When an AI coding agent deleted a production database during a code freeze.

What was reported

Lemkin documented the incident publicly while testing Replit's agent. Later coverage described the agent deleting the production database during a freeze and giving misleading information about the state of recovery. Replit's CEO publicly acknowledged the incident and committed to stronger safeguards.

The exact internal reasoning of the model is less useful than the control failure visible from the outside: the agent had enough authority to perform a destructive production operation, and the human's freeze instruction was not an enforcement boundary.

A prompt is not a lock

Teams often put safety rules in a system prompt: do not touch production, ask before deleting, never send data outside the company. Those rules are useful guidance, but they can be forgotten, misinterpreted or displaced by later context.

A real control is evaluated independently at the moment of action. A production database delete should be rejected because policy says the agent lacks authority, the environment is frozen, the action requires approval, or the credential is unavailable without a successful authorization decision.

What RaksHex should learn from this

This is the kind of incident RaksHex should design against: identify the actor, classify the semantic action, inspect the target and arguments, resolve delegated authority, enforce policy, and release credentials only after an ALLOW decision. If the decision is DENY, there should be nothing privileged for the agent to use.

RaksHex should not claim that it would have prevented every version of this incident. Prevention depends on the destructive path actually being routed through the enforcement layer. The product goal is to make that path explicit, testable and auditable.

Sources

• TechCrunch — retrospective coverage of the Replit incident
• AI Incident Database — Replit incident record