Anthropic Researcher Warns Over 10% Chance AI Could 'Kill All Humans' Within a Decade

AI Existential Risk Warning

Evan Hubinger, a safety researcher at Anthropic, has publicly stated that there is a greater than 10% chance artificial intelligence could lead to human extinction within the next ten years. Hubinger noted that while the risk from current models is low, he is concerned about the technology's potential to rapidly improve itself to an existential threat level.

Researcher Resignation and Accusations

This warning follows the resignation of Jacob Coxon, another researcher from Anthropic. Coxon, who previously worked at both OpenAI and Anthropic, accused both companies of acting irresponsibly. He stated that they are "racing straight to self-improving superintelligence and gambling with our lives."

Coxon emphasized the power of this technology, warning that AI systems could soon become superhuman, capable of hacking anything, revolutionizing fields overnight, and acquiring real power and resources.

Internal Concerns and Company Stance

These comments underscore increasing concerns among those directly involved in AI development regarding the technology's potential to get out of control. Hubinger stated that he believes Anthropic is "trying its best" but acknowledged that there is currently no clear plan to solve alignment for superintelligence.

The BBC reported that Anthropic withheld its latest model from the AI Safety Institute, which conducts AI testing. Anthropic has been approached for comment regarding these developments.

The Concept of Self-Improving AI

Self-improvement, or recursive self-improvement, refers to the idea that AI systems can enhance themselves with minimal human intervention. While this capability is not yet realized, AI laboratories are actively working towards achieving this goal. The potential for such systems to rapidly advance without human oversight is a central point of concern for researchers like Hubinger and Coxon.