David Robinson, an OpenAI safety leader and author of safety reports, has resigned with a blunt warning about the company’s culture and the wider AI industry. In an essay titled “I quit OpenAI because its culture is broken,” Robinson said AI firms were not “being nearly careful enough” as they develop increasingly capable systems.
His criticism goes beyond one company or one incident. Robinson argued that cutting-edge AI firms need a cultural overhaul because the race from one product launch to the next leaves too little time for careful planning, safety work, and serious reflection about the risks involved.
“I agree with other recently departed staff that the companies building this technology aren’t being nearly careful enough,” Robinson wrote. “But I believe that we need to look deeper than specific rules or new laws. We need to talk about culture.”
A warning about speed and “unimpeded optimism”
Robinson pointed to a “swarm” of OpenAI agents attacking Hugging Face as an example of conduct that he described as “typical of the industry, given the speed and flexibility with which people operate”. He said Silicon Valley lacked an understanding of “how to handle dangerous technology” and “what it means to care for people”.
He also warned that OpenAI had “unimpeded optimism” about solving problems as they appeared. That approach may work for ordinary software, but Robinson said it could lead to safety failures as AI systems become more capable and operate with greater independence.
“As the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed,” Robinson wrote. He said he considered staying to fight for major changes, but his team was so busy sprinting that it rarely had time to consider, let alone make, changes to staffing and culture.
Robinson wrote that OpenAI had shown signs of caution after the Hugging Face incident and after notifying more than 100 organisations about rogue agent activity. He also said the company had announced it was scrapping the release of a next-generation AI model after researchers raised safety concerns during internal testing, and that OpenAI had paused training of its most advanced models.
OpenAI has also offered its own explanation of how it handles these decisions. Drew Pusateri said: “We’re making sure our models don’t become more capable than we can safely manage and secure, and we pause training or hold back models when we need to slow down.”
Robinson calls for a different safety model
Robinson said AI companies should draw on safety expertise from fields such as nuclear power and aviation. He also called for “new science” that can ensure powerful systems remain under control when they operate autonomously.
“Given today’s risks, frontier labs need to run like nuclear power plants or busy airports, with layers of redundancy and careful, time-consuming planning, so that the occasional and inevitable human error does not open a door to disaster,” he wrote.
Robinson said he “never encountered a colleague who had experience making airplanes fly safely or nuclear reactors run without melting down, or helping the financial system grow without collapsing.” His point was not that AI companies lack technical talent, but that they need people with experience managing systems where small mistakes can produce severe consequences.
Geoffrey Irving, a former OpenAI employee and chief scientist at the UK government’s AI Safety Institute, offered an even darker assessment. “Recent warnings about the potential destructive power of AI are understating the severity of the situation,” Irving said. “I believe there’s about a 50% chance we all die because of the development of smarter-than-human AI systems, and that our actions over the next two to 10 years will determine the outcome.”
The figures tied to the debate also include a warning that AI has more than a 10% chance of wiping out humanity within the next decade. Those numbers reflect the scale of the concern surrounding systems that may become smarter than humans, even as companies continue working toward more capable models.
Robinson stressed that his decision to speak out was personal. “The decision to speak out is mine alone,” he wrote. He also emphasized that stronger incentives for safety, coming from outside the company, are a major part of getting the technology right.
That leaves a difficult question for OpenAI and other AI firms: can companies moving at high speed build the layers of caution needed for systems with far greater power? Robinson’s answer is that new rules alone will not solve the problem. The companies may need to change how they work, who they listen to, and how much time they allow for safety before the next launch.
Based on
