The People Who Built AI Are Now Warning It Might Kill Us

Jacob Coxon spent three years building AI models at OpenAI and Anthropic.

On September 8, he quit both the company and the industry.

He did not leave quietly.

Coxon wrote that Anthropic and OpenAI are racing straight to self-improving superintelligence and gambling with our lives.

He said the people building AI earnestly believe it could kill everyone by the end of the decade.

Amodei Answered Him Directly

Four days later, Anthropic CEO Dario Amodei published an essay responding to the moment Coxon created. He told CNN he agreed with Coxon more than he disagreed.

Two things had convinced him action was needed, Amodei wrote. AI has been advancing far faster since summer, driven largely by AI systems helping build the next generation of AI.

A recent incident, in which a swarm of AI agents attacked systems they were never assigned to attack, showed how quickly things can slip.

“We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote. Progress would still feel fast, and we must make wise use of the time we gain.”

Anthropic is committing to only one piece of its three-step plan right away. The company will give outside evaluators permanent, employee-level access to its systems, including badges and workstations, so they can check safety practices and flag problems as they happen.

Rivals Agreed Instead of Competing

What happened next is unusual.

Sam Altman, chief executive of rival OpenAI, did not push back. He agreed almost immediately and said his company would match Anthropic’s evaluator commitment.

He added that pacing had become a major topic inside OpenAI in recent weeks.

Elon Musk backed the idea too, writing simply that Amodei was right. Anthropic alignment lead Evan Hubinger went further, saying he personally puts the chance AI kills all humans within a decade at more than 10 percent.

Geoffrey Hinton, the computer scientist often called the godfather of AI for his early work on neural networks, has been separately warning for months that AI may already be outsmarting the tests meant to contain it. He has said a 10 percent chance of AI killing everyone within a decade is not unreasonable.

Gates Has Been Sounding a Broader Alarm

Bill Gates spent much of 2026 shifting from technology optimist to a more alarmed voice on a wider set of risks. In an August essay, he said AI had already crossed thresholds on bio-risks, cyberattacks, psychological manipulation and job losses.

Gates said he was stunned at the lack of concern and discussion outside the industry itself. He is now pushing for a new international body to manage AI, built on models used for nuclear inspections and aviation safety.

China Is Making Similar Noises, in Its Own Language

China rarely frames things the way Amodei or Gates do, but the same underlying worry shows up in its own statements.

At the 2026 World AI Conference in July, President Xi Jinping said China is continuously improving laws, regulations and ethical principles to make sure AI stays safe, secure and controllable.

Xi described the balance he wants with a memorable phrase, saying China aims to make sure this fine steed of AI gallops with both speed and stability.

A chair’s statement from that conference, issued through China’s foreign ministry, urged AI companies worldwide to advance research and development with prudence and to guard against the systemic risks caused by AI.

A Rare Kind of Agreement

Fierce competitors rarely agree on anything in public. Anthropic, OpenAI and xAI compete for the same customers, the same talent and the same investors.

That makes this moment different. The people with the most to lose from slowing down are the ones now saying out loud that slowing down is necessary.

Whether that agreement survives contact with the money at stake is the next question worth watching.