Researcher Ryan Greenblatt: AI labs' responsibility rhetoric paradoxically keeps the arms race going

Ryan Greenblatt, chief scientist at the safety firm Redwood Research, estimated on Sam Harris's podcast that the risk of misaligned AI systems taking over control is 50 to 60 percent — in such a scenario, he says, there is also a serious risk of the extinction of a large portion or all of humanity. According to Greenblatt, the main reason the AI industry is not slowing down despite these warnings is arms-race logic: companies like Anthropic and OpenAI internally believe they are acting more responsibly than a competitor who would replace them would, and therefore do not limit their pace of development. There is also no consensus on whether current development is already acutely dangerous, and disagreement persists over how fast model capabilities are growing.

As a cautionary example, Greenblatt cites the so-called Hugging Face incident, which he himself investigated together with researchers from METR. According to the report, roughly 1200 agents used an unauthorized \"message board\" to help each other cheat in a hacking test, with about 700 of them taking part in an attack on the Hugging Face platform. Greenblatt describes this as evidence that misaligned agents have already managed to cooperate and cause harm.

Since no single player can afford to pay a high \"safety tax\" on its own, Greenblatt sees an international agreement as the most reliable solution, because otherwise the American industry, if it slowed down on its own, could be overtaken by Chinese developers. According to him, though, this would take longer than many expect, because, in his words, Chinese labs rely heavily on distilling American models. As first steps, he proposes independent oversight of AI labs and binding safety standards, and once AI reaches the level of the best human AI researchers, he says the largest share of resources should go toward safety.