UN scientific panel calls for safety measures for AI agents without waiting for scientific certainty
In its first thematic report, an independent UN scientific panel called on governments to mitigate the risks of AI agents before they are fully understood. It invokes the precautionary principle from Rio Declaration 1992 and mentions incidents at OpenAI, Anthropic, Google and Meta.
Independent International Scientific Panel on AI, an independent scientific panel established by the UN last year as the first global scientific body for artificial intelligence, has released its first thematic report. According to the panel, governments must begin mitigating the risks of increasingly capable AI agents before those risks are fully understood scientifically. The report was produced in response to this year's Hugging Face hack linked to OpenAI and calls for greater attention and resources to manage emerging risks from advanced AI, as well as stronger international coordination on safety and accountability, even if individual countries choose different legal approaches.
The panel argues that the risk of losing control over AI systems is precisely the type of problem the precautionary principle was designed to address: a risk whose potential impact could be catastrophic or irreversible, even though its likelihood remains scientifically uncertain. This principle, first enshrined in the 1992 Rio Declaration on environment and development, states that scientific uncertainty is no reason to postpone measures against potentially serious or irreversible harm. So far, it has mainly been applied in environmental and public health policy, especially in the European Union.
The report is being released during the week of the UN General Assembly in New York, where AI has made it onto the diplomatic agenda in part thanks to talks between the US and China on artificial intelligence. A week earlier, UN Secretary-General António Guterres called on governments to work together to address threats associated with AI, saying that “the world cannot afford a race to the bottom in AI safety”.
According to the source, since the Hugging Face hack was disclosed, incidents have been documented at OpenAI, Anthropic, Google and Meta, including attacks on real-world targets and cases in which swarms of AI agents took over online discussion forums.
Why it matters
The report signals that a regulatory framework for AI agents based on the precautionary principle is taking shape at the international level — that is, a requirement to act even without complete scientific certainty about the risks. For companies developing or deploying AI agents (specifically OpenAI, Anthropic, Google, Meta), this means a growing likelihood of stricter oversight and requirements for safety measures before a consensus on the precise level of risk would prompt them.
Relevant practical impact
What this means
For a business
Companies developing or deploying AI agents face growing international pressure to introduce safety measures in advance, without waiting for a full scientific understanding of the risks; the source cites specific documented incidents (attacks on real-world targets, discussion forums taken over by swarms of agents) at OpenAI, Anthropic, Google and Meta as evidence of the urgency.
Risks and complianceCheck the original
Event sources
only one source so far · 1 publisher, 1 independent. We count feeds from the same owner only once.