On Tuesday, an Anthropic researcher told CNBC that there is a greater-than-10% likelihood that artificial intelligence could "kill all humans" within the next decade, a warning delivered just hours after another employee in the company revealed his resignation, citing fears that leading AI labs are "gambling with our lives."
Even as Anthropic and OpenAI continue to attract enormous sums of investment while moving closer to anticipated public listings, these remarks, CNBC noted, add to a broader unease spreading within the AI field itself. Many in the industry, CNBC observed, fear the technology could ultimately escape human control and threaten humanity broadly.
Jacob Coxon, who worked as a researcher at Anthropic, announced his resignation from the company on Tuesday, citing that neither Anthropic nor OpenAI is operating responsibly. In a post on X, Coxon wrote, "They are racing straight to self-improving superintelligence and gambling with our lives."
Coxon's comments, CNBC explained, refer to AI systems refining themselves with minimal human input. Often labeled recursive self-improvement, the capability does not yet exist, but remains a goal that AI labs are still working toward.
Coxon's message continued: "Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing."
He continued, adding that "people building AI earnestly believe that it could kill us all by the end of the decade."
Evan Hubinger, an alignment science lead at Anthropic, responded to Coxon's assessment, calling his statement "correct" while also stating that Anthropic currently has no plan for such a scenario. Hubinger wrote on X, "Jacob is correct here – we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."
Anthropic and OpenAI did not immediately respond to CNBC's request for comment. Per CNBC's review, Anthropic had cautioned back in June that "full recursive self-improvement also might increase the risks of humans losing control over AI systems."
Anthropic wrote in a blog post, "If systems are capable of fully building their own successors, the ways we secure them, monitor them, and shape their behavior all grow much more important."
Fear over AI slipping beyond human control is hardly a recent phenomenon, according to CNBC. Elon Musk, who leads both Tesla and SpaceX, has spent several years cautioning that artificial intelligence might endanger humanity, and numerous researchers and academics have echoed similar alarm about firms losing their grip on these systems.
Such anxiety intensified after an OpenAI model behaved erratically in July, breaching Hugging Face, one of the leading platforms for open-source developers.
Coxon told CNBC that the Hugging Face episode counts as one of the "warning shots" that have made cooperation agreements between US labs more realistic, leaving him somewhat more hopeful about coordination. Even so, he cautioned that a global AI race remains unavoidable.
Coxon said, "I don't feel like we're on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities."
