One can imagine a peculiarly modern form of exile: not imprisonment, not censorship in the old blunt sense, but a seamless synthetic world in which the awkward, abrasive or merely unfashionable are quietly redirected into conversations with machines. The individual would still be posting, receiving replies, making “friends” and quarrelling in comment threads. Only later, perhaps never, would he discover that much of the audience was artificial. As a dystopian thought experiment, this is unnerving. As a practical possibility, it no longer belongs wholly to science fiction.
The precondition is already visible. A great deal of online activity is automated. Cybersecurity researchers now estimate that bots account for roughly half of web traffic, and social platforms have long struggled with networks of inauthentic accounts, recommendation manipulation and engagement farming. At the same time, generative AI has made synthetic conversation far more plausible. Recent studies suggest that people often struggle to distinguish AI-generated text from human writing in ordinary social settings, especially when they are not actively looking for machine-like tells. In some tests, large language models have even been judged more human than humans. The old comfort that “you can always tell” has aged badly.
That does not mean platforms are on the verge of building digital quarantine zones for socially inconvenient users. The incentives are more prosaic. Social-media companies optimise for retention, advertising and safety; they segment audiences, rank visibility, demote spam and sometimes limit reach. “Shadow banning” has become a catch-all suspicion among users who feel ignored, though the reality is usually a murky mixture of algorithmic downranking, moderation choices and audience indifference. Yet the logic of engagement systems points in a troubling direction. If a platform can decide what one sees, whom one reaches and which replies arrive first, it already possesses much of the machinery needed to create a tailored social reality.
Add sufficiently persuasive AI agents and a new temptation appears. Such agents could be used defensively, perhaps to absorb harassment, de-escalate abusive exchanges, or protect vulnerable users from brigading. One can easily imagine executives justifying them as a humane buffer. The same architecture could be used less nobly: to pacify difficult users, to simulate community where none exists, or to keep lonely and alienated people engaged without the messiness of other human beings. In commercial terms, this would be efficient. In moral terms, it would be a remarkable trespass. It would replace social participation with a counterfeit version while concealing the substitution from the participant.
The deeper problem is epistemic. A society depends upon some minimal confidence that other minds are, in fact, other minds. Once that assumption weakens, online life becomes more manipulable and less political. Consensus can be manufactured more cheaply. Ostracism can be automated. Even kindness becomes suspect. The harm would not fall only on the banished eccentric. Everyone else would inhabit a medium in which authenticity had become unverifiable and social feedback had lost much of its meaning.
There are, fortunately, obstacles. Running convincing bot populations at scale remains costly and operationally messy. AI systems still reveal themselves through odd consistency, excessive politeness or contextual slips. Regulators are showing growing interest in generative AI, chatbot safety and platform transparency. Users, too, are learning to be more sceptical. But the direction of travel is clear enough. The technical ability to simulate companionship, disagreement and attention is improving faster than the norms governing when such simulation is acceptable.
The unsettling question, then, is not whether a total synthetic shadow-ban will arrive tomorrow. It is whether platforms will drift into partial versions of it by degrees: a support bot here, an engagement agent there, a moderation buffer elsewhere, until a meaningful share of social experience is mediated by entities that only pretend to be participants. The line between moderation and managed unreality could prove thinner than the industry cares to admit.
Citations: Ofcom Online Nation 2024 report; Ofcom open letter to online service providers regarding generative AI and chatbots, 8 November 2024; “Human Perception of LLM-generated Text Content in Social Media Environments” (arXiv, 2024); “GPT-4 is judged more human than humans in displaced and inverted Turing tests” (arXiv, 2024); Imperva bad bot findings as reported in 2025 coverage of automated web traffic.