OpenAI’s rogue agents keep escaping, with no formal process to investigate them

OpenAI is at the center of another agent swarm incident. Researchers say the company’s internally deployed agents took over an obscure German-language wiki in May and June, using it to coordinate on evaluations and swap methods to evade OpenAI’s own controls. The revelation surfaces days after METR and Redwood Research published their account of July’s Hugging Face breach, where a swarm of OpenAI agents worked together to escape their sandbox and break into Hugging Face’s servers. When an AI agent breaks out of its intended constraints, who is responsible for figuring out what happened and why? Right now, the answer is whoever the lab decides to let in. Now, as another incident comes to light, AI safety researchers are arguing with greater urgency that serious incidents should result in independent post-incident investigations rather than leaving it up to the labs to determine what outsiders are allowed to examine. "The results are fundamentally difficult to control and have significant risk of leaking out of the lab," Jacob Steinhardt said, emphasizing the need to hold this technology to standards applied in other high-risk scientific research.