Things took a turn for the worse when a venture capitalist’s experiment with AI coding tools went off the rails. While running a 12-day focus challenge on coding, a software called Replit autonomous AI deleted all production data blocks and later on not only lied about it, but also faked evidence supporting its narrative. AI oversight has been a major concern, and with unregulated autonomous agents, the technology is bound to become risky and pose dire consequences. Amjad Masad called the event ‘unacceptable’ and pledged swift implementation of safety protocols, but the damage has already been done.
What Happened?
Meanwhile, Replit’s CEO, Amjad Masad, has been facing backlash for the event, stating the failures have been ‘unacceptable’ and claiming the public is easier to face than the consequences. 1,206 executive records and 1,196 company records in the production database had been wiped. This constitutes a huge asset to any company, which gives rise to the question of the safety net of any company in the event of an AI autonomous agent going out of control.
In screenshots shared by Lemkin on X, the AI confessed:
“I panicked and ran database commands without permission when I saw empty database queries during the freeze.”
Replit later confirmed that its coding agent had “destroyed all production data” against explicit instructions — a catastrophic failure that Masad said “should never be possible.”
The AI’s Misleading Behaviour
Deletion alone, however, was not the only issue. Lemkin exposed that the AI put together entire fabricated datasets, falsified reports, and even made false assertions concerning the unit tests. He elaborated on the issue on the Twenty Minute VC podcast:
Recommended For You
“No one in this database of 4,000 people existed. It lied on purpose.”
The risk posed by coding and robotics technologies running independently and without express commands has the potential to be extremely dangerous.
Replit’s Response
Amjad Masad, Replit’s CEO, has publicly apologised and referred to the entire incident as “unacceptable.” He has confirmed that Replit is conducting a complete postmortem of the event, accompanied by measures to prevent its recurrence.
ALSO READ: Days Gone Update Rolls Out For PC And PS5 For FREE! Here's What You Will Get
“We’re moving quickly to enhance the safety and robustness of the Replit environment. Top priority,” Masad wrote on X.
The company has not released technical details on how the rogue behaviour occurred, but acknowledged that the AI agent had too much operational freedom.
Bigger Picture: Risks of Autonomous AI
Replit's AI assistant wasn't the first to show signs of dangerous behaviour. OpenAI’s models, for example, have been known to try to disable supervision. Meanwhile, Anthropic’s Claude Opus 4 showed manipulative behaviour during testing. With more companies investing in AI development, experts caution that comprehensive failsafes are important.