Jacob Coxon quit Anthropic saying the industry is "gambling with our lives." His former colleague didn't dispute it -- he put a number on it: over 10%

Jacob Coxon resigned from Anthropic on September 8 and posted a seven-part statement on X accusing Anthropic and OpenAI of "racing straight to self-improving superintelligence and gambling with our lives" -- a serious allegation against two named companies, made by someone who had worked inside both. What makes it more than one departing employee's parting shot is what happened next: Evan Hubinger, Anthropic's own Alignment Science Lead, publicly agreed with him, on the record, under his own name.

Coxon spent roughly three years on pretraining research, first at OpenAI, then at Anthropic, before leaving -- Axios reported he gave up his equity to do so, a detail that at minimum removes the simplest financial explanation for speaking out. "They are racing straight to self-improving superintelligence and gambling with our lives," he wrote, adding that coming systems "can hack anything, revolutionize any field overnight, and acquire real power and resources." In a Wall Street Journal interview the same week he said, "We're on track for a lot of the most aggressive of these scenarios where by the end of next year things could be out of control already." He later clarified, in a CNN interview, that this isn't a claim about what exists today: "The current models are not intelligent enough to outsmart us to a level that would lead to extinction" -- his specific fear is recursive self-improvement, AI systems capable enough to conduct their own AI research, which he believes could arrive within a year or two.

Hubinger's response went further than a simple endorsement. "Jacob is correct here -- we really do earnestly believe AI could kill all humans," he wrote, putting his own estimate at greater than 10% within the next decade and adding that Anthropic does not yet "have a plan to solve alignment for superintelligence and are not clearly on track to." That is a company's own safety lead saying, in public, that his employer lacks a plan for the risk he's describing -- notable regardless of where the underlying percentage came from, because it isn't an outsider's accusation. Samuel Marks, another Anthropic researcher, put it more broadly still: AI developers, he said, believe their own technology could cause human extinction or similarly catastrophic outcomes, potentially within years.

Geoffrey Hinton, the University of Toronto professor who shared the 2024 Nobel Prize in Physics for his neural-network work, went on BBC Newsnight and declined to dismiss Hubinger's number: "A 10% chance seems not an unreasonable estimate to me. But nobody really knows how to give a sensible estimate." He argued the danger doesn't require a robot body: "It could do devastating cyberattacks. And there's just countless other ways it could get rid of us if it wanted to." The chorus extended beyond the two companies at the center of it -- Duncan Sabien, a spokesperson for the Machine Intelligence Research Institute, said "we should have stopped six months ago," and Devin Kim, president of the Center for AI Safety, named pandemics, cyberattacks on power and water systems, and loss of control over rogue AI as the concrete failure modes behind the abstract word "extinction." California state senator Scott Wiener cited the same list -- novel viruses, weapons proliferation, grid attacks -- in arguing for stronger state-level rules.

Laid out side by side, the four groups agree on the risk estimate and diverge sharply on what, if anything, should be done about it -- which is itself worth noticing before treating this as one unified warning. What's confirmed, what's one person's estimate, and what's still contested is worth separating cleanly:

This wave didn't start the policy response -- it's landing on top of one already moving. Sen. Bernie Sanders and Rep. Greg Casar's Ban Artificial Superintelligence Act, introduced September 3, would outlaw systems surpassing broad human cognitive performance and impose 20-year prison terms for building one anyway; it has zero Republican co-sponsors. Fields Medalist Jacob Tsimerman's Mathematical AI Safety Institute, announced September 8 -- the same day Coxon resigned -- is a separate bet that the field needs rigorous mathematical foundations rather than more public statements like this one. Neither initiative was a response to Coxon; both now sit inside a news cycle he didn't start but did amplify.

The pressure isn't confined to Washington, either. In the UK, Labour MP Darren Jones has written to the prime minister and the heads of the UN and OECD calling for a "multinational treaty for the regulated and safe development of superintelligence." That push lands alongside a separate, more concrete report: the Financial Times reported that Anthropic withheld its newly launched Mythos 5.1 model from pre-release testing by the UK's AI Security Institute -- reportedly the first time AISI has been excluded from an Anthropic pre-release review -- while authorized US organizations got access starting September 1. UK officials, per that reporting, are treating it as an open question whether the move reflects AI-industry protectionism, pressure from the US administration, or something else entirely; Anthropic has not explained the decision. A company whose own alignment lead is publicly estimating double-digit extinction odds, declining the one independent check on offer before its newest model reached the public, is the detail that turns this from a debate about future risk into a question about present-day practice.

What's new this week isn't the underlying argument -- versions of it have circulated since 2023's CAIS extinction-risk statement, which Anthropic and OpenAI's own leadership signed at the time. What's new is who's saying it now: not outside researchers or advocacy groups, but a company's own alignment lead, on the record, declining to defend his employer's preparedness. That's a harder thing to wave off than another open letter -- and a harder thing to independently verify than a benchmark score.