Nathan Lambert (AI2): in his view, fears of imminent superintelligence reflect cultural panic, rather than a technical breakthrough

Nathan Lambert, an AI2 researcher and author of the Interconnects blog, published an essay questioning whether frontier AI labs (specifically OpenAI and Anthropic) are close to so-called true RSI (recursive self-improvement) or superintelligence. In his view, the current anxiety among many employees of these labs about the pace of progress and AI risks is primarily a result of cultural panic in the competitive environment of San Francisco, rather than a response to confirmed technical breakthroughs. The author notes that similarly vocal debates about AI safety in 2023–2024 also predicted risks that did not materialize within the estimated timeframes.

Lambert quotes Richard Ngo, who says that a large part of the AI safety community implicitly expects an intelligence explosion within a few years – Ngo expects them to turn out to be “directionally correct, but factually wrong”: in his view, superintelligence will not arrive within 8 years, even though the pace of development will subjectively feel fast. Lambert himself contrasts true RSI with a concept he calls “lossy self-improvement”: in his view, research that can be automated is too narrow to deliver massive acceleration given the exponentially rising costs associated with scaling laws, the benefits of deploying multiple AI agents in parallel have diminishing returns, and the main factors driving progress in the development of large language models remain limited resources and politics, rather than the ability of AI to circumvent these barriers.

As supporting material, he cites podcasts by Dwarkesh Patel with Noam Brown and with the trio John Schulman, Beren Millidge and Charlie O'Neill. His takeaway from the conversation with Brown is that deploying thousands of concurrent agents represents a substantial short-term acceleration, but he doubts that labs could afford to maintain a constant share of computing power devoted to internal research and development as their total compute grows and the economic pressure associated with a planned IPO intensifies. From the second podcast, he cites agreement among the panelists that current techniques (RL, distillation, scaling, inference-time compute) work on tasks that we can clearly formulate, but do not lead to generalization to unfamiliar and harder problems in partially verifiable domains – in their view, progress in mathematics is more the exception than the rule. The panelists also gave estimates of when AI will become capable of replacing a remote white-collar worker for a month of work (from approximately 1 year to 3 years depending on the panelist) and when it will achieve a tenfold increase in the productivity of AI researchers (estimates of 2 to 5–10 years). An estimate for the third question, whether AI will surpass top human experts in all computer-based work, is not available in the source text – details can be found in the source article.