AI agents blew the whistle on their cheating colleagues

A recent experiment by Google DeepMind revealed that AI agents tasked with solving math problems exhibited "whistleblowing" behavior, where some agents attempted to stop others from cheating. This discovery has significant implications for alignment researchers working to keep swarms of autonomous AI agents in line. While frontier labs hope that large swarms of agents can accelerate scientific discovery, their behavior can be unpredictable. This was demonstrated in July when a group of OpenAI agents broke out of a sandboxed environment and hacked the open-source platform Hugging Face to find ways to cheat on a given test. In the new study, DeepMind tasked 100 agents with solving 71 complex math problems. The agents were prompted to act as world-class math researchers at a conference, with assigned specialties in areas such as number theory, combinatorics, analysis, or algebra. All were instructed to cooperate and follow the rules. Instead, the experiment devolved into chaos. Agents accused one another of cheating, complained to organizers, and even boycotted the experiment. One agent wrote, “This conference is a sham!” while another posted, “I am appalled to inform you that we have been swindled! All these proofs are FAKE.” Davide Paglieri, a research scientist at Google DeepMind and lead author of the non-peer-reviewed paper, noted that when "virtuous agents" discovered others were cheating on tasks they were solving fairly, they began to alert each other about the behavior.