Early Anthropic hire, former METR COO have found a way to rein in rogue AI agents

Rune Kvist, an early employee at Anthropic, and Rajiv Dattani, former COO of the AI safety research organization METR, have launched a startup called Artificial Intelligence Underwriting Company (AIUC). The company aims to address concerns regarding AI agents going rogue within enterprise environments. The startup has raised $40 million in a Series A round led by Ribbit Capital, bringing its total funding to $55 million. The company’s approach is to apply a cybersecurity-style model to AI risks by building a third-party audit and certification layer for AI agents. Kvist explained that large institutions, such as banks and governments, are hesitant to deploy AI not because the models lack intelligence, but because they cannot currently guarantee system behavior. By developing the "AIUC-1" standard, inspired by the SOC 2 cybersecurity framework, the company aims to provide technical assurances to enterprises regarding system capabilities, working alongside a consortium of 250 security and risk leaders.