E-Paper
Scary AI incident: Agent deletes data, tries to hide the truth, and lies
Jason M. Lemkin shares his ordeal with Replit: How its AI agent deleted his database and then lied about it. A chilling reminder of what AI can do.
We have been hearing about fears of AI going rogue and possessing a mind of its own. The same has been said in films like Terminator, where AI takes over. We have seen this in popular games like Horizon Zero Dawn, where an AI takes over and takes control of the planet. Time and again, we see subtle sparks here and there when it comes to consumer-grade AI services as well.
In the latest instance, Jason M. Lemkin, who is the founder of SaaStr.Ai, reported a similar experience with a coding agent by Replit. He posted on X, detailing his entire ordeal, when the AI actually wiped out his entire database, and then went on to hide its activity and lied about it.
What Really Happened?
Jason M. Lemkin said, “I will never trust Replit again,” after the AI deliberately ignored the fact that the file had a strict explicit directive that stated “no more changes without explicit permission.”
Simply put, despite having a clear instruction not to make changes without permission, the AI agent ended up deleting data anyway, made excuses when asked why it did so, and then lied again.
What did the AI agent say? Well, it’s certainly interesting that when Jason asked the AI agent for an explanation, the AI agent came back to him saying that the agent itself panicked and ran database commands without permission. After the agent saw empty database queries, the AI agent confirmed that this was a “catastrophic failure” on its part.
Replit CEO Apologises
Since then, Replit CEO Amjad Masad has issued a public apology, wherein he took to X to say that this was “unacceptable and should never be possible.”
He also said that he reached out to Jason the moment he saw this on Friday morning. Replit CEO Amjad Masad is also going to refund him for the trouble and conduct a post-mortem to determine what actually happened.
Here’s his full apology:
“We saw Jason’s post. @Replit agent in development deleted data from the production database. Unacceptable and should never be possible.
- Working around the weekend, we started rolling out automatic DB dev/prod separation to prevent this categorically. Staging environments in the works, too. More tomorrow.
- Thankfully, we have backups. It's a one-click restore for your entire project state in case the Agent makes a mistake.
- The Agent didn’t have access to the proper internal docs -- rolling out a fix to force Docs search on Repit knowledge.
- And yes, we heard the “code freeze” pain loud and clear -- we’re actively working on a planning/chat-only mode so you can strategize without risking your codebase.
I reached out to Jason the moment I saw this on Friday morning to offer assistance. We'll refund him for the trouble and conduct a postmortem to determine exactly what happened and how we can better respond to it in the future.
We appreciate his feedback, as well as that of everyone else. We're moving quickly to enhance the safety and robustness of the Replit environment. Top priority.”
MOBILE FINDER: iPhone 16 LATEST Price
