How AI Is Blurring The Line Between Fantasy And Real Threats

Caitlin Roper started seeing herself in nightmares that looked like social-media posts. Pictures of her dangling from a noose or caught in flames began circulating on X and other networks. They were grotesque — but what made them unbearable wasn’t just the violence. It was the small, chilling detail: the blue floral dress she was wearing in the photos was the exact dress she actually owned. These images weren’t careless trolling. They were deliberately produced, AI-crafted likenesses meant to frighten.

ALSO READ: Google vs Disney: YouTube TV Blackout Explodes Into A Public Fight

This isn’t the same old harassment dressed up in new words. Generative models now let someone take a single portrait and spin out terrifying, realistic visuals. Where once a determined harasser might have needed technical skill or access to video tools, now a few prompts and the right model are enough. As University of Florida law professor Jane Bambauer warned, “Anyone with no skills but bad intent can now use these tools.”

The pace at which the technology has become usable is what alarms experts. Digital simulations of violence have existed for years — courts and communities have seen edited or staged media before — but the difference today is speed and scale. Channels and feeds that once hosted user-made fakes now sometimes carry dozens of lifelike clips: women being harmed, judges portrayed as victims, audio that mimics familiar voices. One troubling thread is how quickly those clips can be assembled and shared before platforms react. In some cases, users have even relied on chatbots to get step-by-step guidance for real-world harm.

Platforms and the teams that run them are feeling the pressure. OpenAI’s Sora text-to-video feature, for example, has been singled out after users produced hyper-realistic violent content; the company says it applies guardrails and moderation, but critics contend those protections can be worked around. X removed some posts that targeted Roper but left others — and for a time, when she publicly complained about the abuse, the platform temporarily suspended her account instead of the harassers’. That mismatch between user harm and platform response has left many feeling exposed.

Another, subtler danger is how this tech bolsters “swatting” and other hoax emergencies. Synthetic audio and voice-cloning tools let callers imitate victims or officials convincingly; a fabricated distress call can sound authentic enough to trigger an armed response. In one instance, a school district in Washington was locked down after officials received an AI-generated report of a shooter. “How does law enforcement respond to something that’s not real?” asked Brian Asmus, the district’s safety chief.

ALSO READ: Looking Beyond Chrome? These Privacy-First Browsers Put You Back In Control

The result is a shift in how harm is experienced online. Harassment used to be words on a screen or anonymous insults in a comment thread; now it can be moving, personalized imagery showing you harmed. That visceral quality changes the stakes. For victims, the images aren’t abstract threats — they feel immediate and real. Roper captured that fear plainly: “These things can go from fantasy to more than fantasy.” As generative tools improve and spread, the worry is that this form of targeted intimidation will only grow more damaging — and harder to police.