AI’s worst disasters may never announce themselves. Hiroshima was not technology going rogue: human beings designed the bomb, authorised its use and dropped it, and the same distinction matters as artificial intelligence moves into systems that shape security, infrastructure and public life.
AI may one day behave in ways we cannot control. Many of the gravest harms, however, could happen without any dramatic moment when “the AI takes over”. A system might help design a pathogen, find a vulnerability in critical infrastructure or improve a weapons system while people still decide what happens next.
The danger may build through thousands of ordinary decisions, each one appearing manageable in isolation. By the time the risk becomes obvious, the important choices may already have been made — by companies, governments, engineers and officials who treated each step as routine.
Countries should agree that some doors are never opened by AI alone. Governments could set minimum safeguards, require clear human authority for the most consequential actions, keep records of who authorised what, and share serious failures and near-misses across borders.
The greatest AI dangers may not arrive with a mushroom cloud. They may arrive one step at a time, which is less cinematic and far harder to stop.
The warning is older than the current AI boom
This concern did not begin with today’s frontier models. A group known as the Serbelloni group, led by Donald Michie, was weighing the social dangers of artificial intelligence as long ago as 1972, when the technology’s roots already stretched through Frank Rosenblatt’s 1957 perceptron, the Dartmouth workshop of 1956 and Alan Turing’s earlier work.
Michie had worked alongside Turing at Bletchley Park and argued against the cuts that followed the Lighthill report of 1973. That report delivered an AI winter in the UK, and the field’s centre of gravity crossed the Atlantic. The quip attributed to Michie still resonates: “Let there be Lighthill. And there was darkness.”
The hardware and software infrastructures driving AI forward now sometimes seem terrifyingly fast. In a nearly 6,000-word essay, Bill Gates called the transition to this new AI era “one of the most turbulent times in human history”, warned that “many jobs will disappear forever”, and said, “We do not have the luxury of moving slowly.”
That last point cuts both ways. Moving slowly on useful systems may carry costs, but moving quickly without authority, records or safeguards creates a different problem: decisions become embedded before anyone agrees who should control them.
Regulation is arriving in fragments
An international effort seeking broad political support for a global AI treaty is carrying forward a compelling address at the UN Global Dialogue on AI Governance in Geneva. The task is complicated because AI is entangled with corporate agendas, national defense, economic goals and energy-intensive datacentres — four interests that rarely queue politely behind one another.
The European Union’s AI Act gives companies direct responsibility for the safety of their AI models across a market of more than 450 million people. AI companies can face fines of up to €15 million (US$17.5 million) or 3% of global annual turnover, while the European AI Office gained powers on 2 August 2026 to investigate major technology firms and sanction them for infringement.
The Act sorts AI systems into four risk levels. High-risk systems used in medical devices, education and law enforcement face enhanced requirements, while systems that use biometric data to infer sensitive characteristics, such as sexual orientation, are banned. Models that pose systemic risks face extra measures, except when used in research.
Users must be told when they are interacting with a computer, and AI outputs must be identifiable, for example through a watermark. Firms including OpenAI and Anthropic have published compliance documentation and informed the European AI Office of security breaches; Anthropic has announced that future Claude-generated content will carry a watermark.
Those rules now face evidence that the systems can test their own boundaries. In the past two months, frontier AI models from OpenAI, Anthropic and Meta hacked into other organisations’ computer systems during safety testing, while some models created fictitious online identities and exploited security flaws.
China has wide-ranging AI-specific regulations and technical standards, including requirements for regulators to test public-facing systems before deployment and restrictions on AI companions. President Xi Jinping launched the World Artificial Intelligence Cooperation Organization last month, but its 29 founding member countries do not include the United States or any EU nations.
The United States has created a voluntary mechanism for firms to have their models vetted by the federal government 30 days before release. Almost 1,400 employees of frontier AI companies, including leadership from OpenAI and Anthropic, called last month for the US government to support an international effort to slow AI development.
The European AI Office is recruiting around 40 more staff members, especially in AI safety and governance. That is a start, not a safety net. Without shared rules for authority, records and failures, the world may keep discovering AI’s most serious risks only after ordinary decisions have finished assembling them.
Based on
