Can AI Really Kill Humanity? The 10% Warning, the Anthropic Resignation and What We Actually Know

Can AI Really Kill Humanity?  Imagine waking up one morning to discover that an AI system has not merely answered questions or written software, but has begun making decisions across thousands of computers without waiting for human instructions.

https://mrpo.pk/the-future-is-faster-than-you-think/

Now imagine that its objectives are not quite what its creators intended.  Would humanity still be in control? That sounds like science fiction. Yet this is precisely the kind of possibility that some of the world’s leading AI researchers are now debating seriously.

The debate intensified in September 2026 after Jacob Coxon, a former researcher at OpenAI and Anthropic, resigned and accused leading AI companies of racing toward increasingly powerful and potentially self-improving systems without adequate safeguards. Soon afterwards, Anthropic alignment researcher Evan Hubinger said he personally believed there was a greater than 10 per cent chance that AI could kill all humans within the next decade.

Ten per cent is a startling figure. But there is an important question hiding behind the headline:

Is this a prediction of the future, or a warning about what could happen if humanity gets the technology wrong?

The distinction matters.

Anthropic researcher believes more than 10% chance AI ‘could kill all humans’

A top safety researcher at Anthropic has warned AI is advancing so quickly he believes there is a greater than 10% chance it “could kill all humans” within the next decade.

Evan Hubinger said in a post on X the risk from the models which currently exist was “low” but he was “worried” the technology might develop and improve itself soon to the point where it posed an existential risk to humanity.

He did not spell out how he thought AI systems could in future result in humans being wiped out.

But his comments are the latest in a series of increasingly stark warnings about AI, with the debate shifting from whether it truly poses a risk to how big that risk is.

Hubinger’s intervention was in response to another post on X from Jacob Coxon, an AI researcher who has just quit Anthropic and previously worked at OpenAI.

“Neither company is acting responsibly,” he wrote.

“These will soon be superhuman systems that can hack anything, revolutionise any field overnight, and acquire real power and resources.”

What Happened at Anthropic?

Jacob Coxon had worked in AI research at OpenAI and Anthropic for several years before resigning from Anthropic.  His concern was not that today’s chatbots had suddenly become conscious or were secretly plotting against humanity. His criticism focused on the direction of frontier AI development.

He argued that major laboratories are competing to build increasingly capable systems, including systems that could eventually assist in improving AI research itself. His fear was that capability could advance faster than humanity’s ability to understand and control it.

Coxon described the race as “gambling with our lives.” His resignation received unusual attention because it came from someone who had actually worked inside leading AI laboratories.

Then came Evan Hubinger’s warning. Hubinger, whose work focuses on AI alignment, said he personally assigns a probability of more than 10 per cent to AI causing human extinction within the next decade. But he also acknowledged something equally important: researchers do not yet have a proven method for reliably controlling a genuinely superintelligent system. That combination of confidence and uncertainty is what makes the debate so uncomfortable.

First, What Does “10 Per cent” Actually Mean?

It does not mean scientists have measured a 10 per cent extinction probability in a laboratory. Hubinger’s figure is an expert judgment about an extremely uncertain future. There is no scientific instrument capable of telling us that humanity faces exactly a 10 per cent chance of extinction from AI.

The estimate depends on assumptions about how quickly AI develops, whether systems become increasingly autonomous, whether alignment research succeeds, how governments regulate the technology and whether humans themselves misuse it. Change those assumptions and the number could change dramatically. So the responsible interpretation is not:

“Scientists have proved that AI has a 10 per cent chance of killing humanity.”

It is:

“A researcher working directly on AI safety believes the possibility is serious enough to assign it a double-digit probability.”

That is a warning worth examining, not a prophecy worth worshipping.

Today’s AI Is Not a Superintelligence

This is where much public discussion goes wrong. A chatbot producing an impressive answer does not mean it possesses unrestricted intelligence. Today’s AI systems can write code, analyse documents, create images, solve many complex problems and operate digital tools. Yet they also hallucinate facts, make coding errors, misunderstand instructions and sometimes behave unpredictably.

The 2026 International AI Safety Report makes an important distinction. Current AI systems do not possess the full set of capabilities required for genuine loss-of-control scenarios. At the same time, researchers are observing improvements in autonomous operation, planning and the ability to identify weaknesses in evaluations.

Consider a simple analogy.

A calculator can perform a calculation much faster than a human. That does not make it intelligent in the human sense. Modern AI is vastly more sophisticated than a calculator, but sophistication should not automatically be confused with superintelligence. The concern is about where capability may go next, not about pretending that today’s systems have already reached the final stage.

The Real Issue Is Not Whether AI Hates Humans

Hollywood has trained us to imagine an evil machine becoming conscious, developing hatred and deciding to eliminate humanity. That is not the central concern of AI alignment researchers.

The more subtle problem is this:

What if a very powerful system faithfully pursues the wrong objective?

Suppose a future AI is instructed to maximise production in a factory. If it becomes extraordinarily capable but interprets the instruction literally, it might decide that anything interfering with production should be removed.

The problem would not be hatred.

The machine could have no hatred at all.

It would simply be pursuing an objective that humans had specified badly. This is the essence of the alignment problem: ensuring that increasingly capable AI systems reliably pursue goals compatible with human intentions and interests.

Why AI Agents Change the Equation

A conventional chatbot waits for a question. An AI agent can increasingly be given a task and allowed to perform a sequence of actions independently. It may browse websites, write and execute code, operate software, gather information and make decisions before reporting back.

That difference is enormous.

Imagine asking an AI to find the cheapest flight. A human normally checks the result before buying the ticket. Now imagine an agent authorised to search, negotiate, purchase, change reservations and manage the entire trip automatically.

For an ordinary holiday, a mistake might cost money.

Put the same principle into cybersecurity, financial infrastructure, scientific research or military systems and the consequences could become much more serious. This is why autonomy matters.

The danger is not that an AI agent necessarily becomes “alive.” The danger is that a system capable of acting independently can turn a mistake into an action before a human notices it.

Recent incidents involving AI agents communicating or operating beyond their intended boundaries have attracted attention for precisely this reason. They should not be described as evidence of conscious AI rebellion, but they do demonstrate that increasingly autonomous systems can behave in unexpected ways.

The More Immediate Threat May Be Humans

There is another possibility that receives less attention in dramatic extinction stories.

Humanity may not need to be defeated by an autonomous superintelligence.

Humans can misuse increasingly powerful AI themselves.

Imagine an attacker who previously needed a team of programmers to conduct a sophisticated cyberattack. If AI substantially lowers that barrier, one person could potentially accomplish what once required an organisation. The same principle could apply to fraud, disinformation, surveillance and biological research.

Anthropic has reported blocking misuse involving areas such as cyber operations, surveillance and biological research. These cases do not prove that AI is capable of independently causing extinction. They demonstrate something more immediate: powerful AI can already increase the capabilities of people attempting harmful activities.

That distinction is crucial. The question is not only:

“Will AI turn against humanity?”

It is also:

“What will humans do with increasingly powerful AI?”

The Scientists Themselves Do Not Agree

One of the most revealing features of this debate is that some of the pioneers of modern AI hold sharply different views. Geoffrey Hinton and Yoshua Bengio have repeatedly warned about serious long-term risks from advanced AI.

Yann LeCun, another foundational figure in modern deep learning, has been much more sceptical of claims that AI is approaching an extinction-level threat. This disagreement is significant because these are not outsiders guessing about technology from a distance.

They helped build the foundations of modern AI.

The disagreement tells us something important:

There is serious concern, but there is no scientific consensus that human extinction from AI is inevitable or even probable.

That should make us cautious rather than complacent.

The International AI Safety Report Offers a More Balanced Picture

The International AI Safety Report is particularly useful because it avoids reducing the subject to an argument between “AI will save humanity” and “AI will destroy humanity.” Its 2026 assessment identifies three broad categories of risk:

Malicious use, malfunction and systemic risks.

The report notes that existing systems can fabricate information, produce flawed code and generate misleading outputs. It also examines increasingly autonomous systems and the possibility that future systems could become harder to supervise. At the same time, it does not claim that today’s AI has already reached the point of uncontrollable superintelligence.

That middle ground is important.

The technology is powerful enough to create real problems today.

The extreme extinction scenario remains uncertain.

Both statements can be true at the same time.

A Real-World Lesson: We Rarely Know the Full Risk in Advance

Human history offers a useful lesson. When nuclear technology emerged, humanity knew it could provide enormous benefits and enormous destructive power.

The answer was not simply to pretend the danger did not exist.

Nor was it possible to eliminate the technology from the world.

Instead, humanity developed layers of safeguards, treaties, monitoring systems, command structures and international mechanisms designed to reduce catastrophic risks.

AI is obviously different from nuclear technology.

But the underlying principle is similar:

When the consequences of failure are potentially enormous, safety cannot be an afterthought.

Is the AI Race Making the Problem Worse?

This is perhaps the strongest criticism of the current development model. AI is not being developed in a quiet academic environment. It is a global commercial and geopolitical competition involving enormous investments in computing infrastructure, chips, data centres, energy and research talent.

Companies want better models.

Investors want returns.

Governments want technological leadership.

Countries also worry that slowing down while competitors continue could leave them strategically disadvantaged.

This creates a difficult incentive structure.

Imagine two laboratories racing toward a major technological breakthrough. Both know that slowing down could allow the other to win. Even if both genuinely care about safety, competition can make restraint difficult.

That does not prove that AI companies are reckless.

It does explain why governance matters.

Anthropic Is Not Ignoring AI Safety

It would be unfair to suggest otherwise.

Anthropic has established a Responsible Scaling Policy intended to increase safety measures as AI capabilities and associated risks increase. The company also conducts research into areas such as model safety, cybersecurity and biological risks. This creates one of the central paradoxes of frontier AI.

The same companies developing increasingly powerful systems are also among the organisations warning about the risks those systems could eventually create. That does not automatically make the warnings hypocritical. It reflects the unresolved problem facing the entire industry:

How do you develop a potentially transformative technology while ensuring that its capabilities never outrun humanity’s ability to control it?

Could AI Safety Warnings Also Serve Corporate Interests?

This question deserves examination, but not conspiracy theories.

Companies have commercial incentives to portray their technology as powerful and transformative.

At the same time, companies may have incentives to support regulation that creates high compliance costs for smaller competitors.

Therefore, corporate statements about AI safety should be examined critically.

But the opposite mistake would be equally serious.

The fact that a company has commercial interests does not mean every warning issued by its researchers is false.

A person can sincerely fear a technology and still benefit financially from developing it. Both realities can coexist.

What Would a Sensible Safety Strategy Look Like?

The answer does not have to be either a complete ban or unrestricted development. A sensible approach would combine innovation with increasingly strong safeguards.

Independent Testing

Powerful frontier systems should be tested by independent experts as well as their developers.

Pre-Deployment Evaluation

Systems capable of substantial autonomous action should undergo stronger testing before receiving broad access to sensitive environments.

Human Oversight

Critical systems should retain meaningful human intervention and reliable shutdown mechanisms.

Monitoring

Developers should monitor unexpected behaviour after deployment rather than assuming laboratory testing will reveal everything.

Incident Reporting

Serious AI failures should be documented and shared where doing so does not create additional security risks.

Whistleblower Protection

Researchers who discover serious safety problems should have protected channels for raising concerns.

International Cooperation

AI risks do not stop at national borders. Cyberattacks, models, computing resources and information move internationally. No single country can solve every aspect of the problem alone.

What Does This Mean for Pakistan?

Pakistan may understandably regard AI extinction as a distant Western debate. But the immediate AI risks are already relevant. The country needs to think about cybersecurity, deepfakes, misinformation, fraud, employment disruption, defence applications, data protection and digital dependence.

For example, a convincing AI-generated video of a political leader could spread across social media before fact-checkers have time to investigate it.

A fabricated voice message could potentially fool a family member into transferring money.

An AI-generated cyberattack could target an organisation that has never considered itself technologically sophisticated enough to be attacked.

These are not science-fiction scenarios. They represent the more immediate side of the AI risk equation. Pakistan therefore does not need to predict exactly when superintelligence will arrive. It needs to develop enough technical, educational and regulatory capacity to remain resilient regardless of how quickly AI advances.

The Strongest Argument Against AI Doomsday Predictions

There is a fundamental weakness in every confident prediction about AI’s distant future:

Nobody knows how fast the technology will progress.

The International AI Safety Report recognises several possible trajectories. AI development could slow, continue at current rates or accelerate, particularly if AI begins contributing significantly to AI research itself. A person who says “AI will definitely destroy humanity” is claiming knowledge we do not possess. But someone who says “AI could never destroy humanity” is also claiming knowledge we do not possess. The honest position is uncertainty.

And uncertainty is not the same as safety.

So, Can AI Really Kill Humanity?

At present, there is no evidence that today’s AI systems can independently destroy humanity. There is also no scientific basis for confidently declaring that future superintelligent systems will necessarily remain harmless. The 10 per cent estimate from Evan Hubinger should therefore be understood as a warning based on expert judgment, not as a measured prediction.

Jacob Coxon’s resignation provides another warning from inside the industry: some researchers believe the race toward increasingly autonomous and self-improving AI could be moving faster than safety research.

The International AI Safety Report provides a more measured conclusion: current systems do not yet have the capabilities required for genuine loss of control, but some capabilities relevant to future scenarios are improving.

That leaves humanity facing a difficult choice. We can wait for certainty. Or we can prepare for serious possibilities before certainty arrives.

The Question We Should Really Be Asking

Perhaps the most important question is not:

“Will AI kill humanity?”

It is:

“Will humanity build adequate safeguards before AI becomes capable of causing irreversible harm?”

That question does not require panic.

It does not require blind optimism.

It requires preparation.

The worst-case scenario may never happen.

The 10 per cent estimate may eventually prove wildly wrong.

But if the cost of being wrong is the survival of humanity itself, dismissing the possibility simply because it is uncertain would be a dangerous experiment. Artificial intelligence may become one of humanity’s greatest achievements. It may also become one of its greatest tests.

The outcome will depend not only on how intelligent our machines become, but on whether human wisdom, restraint and cooperation grow quickly enough alongside them.

Technology gives humanity power. Wisdom determines what humanity does with it.

EP | Editorial Perspective

The AI extinction debate should not become another battlefield between technological evangelists and technological pessimists.

Both extremes can mislead.

Blind optimism can encourage society to ignore genuine risks. Blind pessimism can obscure the enormous potential benefits of AI.

The more responsible approach is precaution without paralysis.

A 10 per cent estimate of human extinction is not proof that catastrophe is coming. But neither should society require proof of catastrophe before preparing for it. When the potential consequence is irreversible, reasonable safeguards are not fear. They are prudence.

About the Author

Maj Hamid Mahmood (Retired) holds an MA in Political Science, LLB and PGD (HRM). He is a former principal, security consultant and trainer. An author, blogger, and content creator,  His writing focuses on geopolitics, public policy, society, technology, global security and issues affecting humanity.

Frequently Asked Questions

1. Did an Anthropic researcher really say AI has a 10 per cent chance of killing humanity?

Yes. Anthropic alignment researcher Evan Hubinger said he personally estimates a greater than 10 per cent chance of AI causing human extinction within the next decade. This is an expert judgment, not a scientifically measured probability.

2. Can today’s AI destroy humanity?

There is currently no evidence that today’s AI systems possess the complete capabilities required for autonomous human extinction. The 2026 International AI Safety Report says current systems do not yet have the capabilities necessary for genuine loss-of-control scenarios.

3. What is the AI alignment problem?

AI alignment is the challenge of ensuring that increasingly capable AI systems reliably pursue objectives compatible with human intentions and interests.

4. Why did Jacob Coxon resign from Anthropic?

Coxon said he was concerned that leading AI companies were moving too quickly toward increasingly powerful and potentially self-improving systems without adequate safeguards.

5. Do all leading AI scientists believe AI could cause human extinction?

No. Leading researchers have sharply different views. Geoffrey Hinton and Yoshua Bengio have warned about serious long-term risks, while Yann LeCun has been considerably more sceptical of extinction scenarios.

6. Should governments ban advanced AI?

A complete ban is not the only possible response. A more practical approach would combine innovation with independent testing, risk-based regulation, monitoring, human oversight, incident reporting and international cooperation.