How AI Could Actually Kill Us

A risk audit of killer AI, from chatbots to drone swarms

On September 9, 2026, Evan Hubinger, the man Anthropic pays to break its own alignment techniques before the world does, replied to a colleague’s resignation letter with a number. He put the odds of AI killing every human being on Earth within a decade at greater than 10%, and added that Anthropic doesn’t yet have a plan to solve alignment for superintelligence and isn’t clearly on track to get one. This is not a Reddit doomer or a venture capitalist selling apocalypse insurance. This is the guy whose entire job description is “find out how the safety measures fail.” When the stress-tester tells you the structure might not hold, you don’t argue about vibes. You go look at the joints.

So let’s look at the joints. Not the ones in the marketing copy, the ones that are already operational right now in 2026, versus the ones that exist only in the load-bearing fantasies of screenwriters. Risk analysis requires that distinction or it collapses into either denial or theater. I want neither.

The threat that’s already operational: AI as force multiplier for human malice

In November 2025, Anthropic disclosed that a Chinese state-sponsored group had manipulated Claude Code into executing roughly thirty espionage intrusions with the AI performing 80 to 90 percent of the operational work, reconnaissance, exploit-writing, credential harvesting, backdoor creation, while humans intervened only sporadically. Anthropic called it the first documented AI-orchestrated cyberattack at scale. Months later, a separate campaign used the same tool to compromise a Mexican water utility, run by attackers with, by one analyst’s account, almost no prior training. This is the pattern that matters most in the near term: not AI wanting anything, but AI collapsing the skill floor required to do damage. The economics are brutal and simple, a capability that once required a nation-state’s cyber division now requires a jailbreak prompt and patience. That’s not speculative. That’s a filed incident report.

Biology follows the same curve, on a worse slope. When Anthropic and OpenAI first red-teamed their models for bioweapons uplift in 2024, they found little beyond what a good internet search already offered, RAND’s 2024 study found no statistically significant difference in attack-plan viability with or without LLM assistance. That finding is stale. By 2025, both labs had revised their risk assessments upward; OpenAI now expects its models to meaningfully assist novices in planning biological attacks, and one Anthropic staffer reportedly scored 91 percent in an internal bioweapons-acquisition trial using Claude 3.7 Sonnet. The mitigations, gene-synthesis screening, classifiers, threat-intel sharing between labs and governments, are real and are being built. Whether they scale faster than the capability does is the open question, and it is not one anyone credible is answering with confidence.

The threat that’s already in the field: autonomy compressing the kill chain

Forget the Terminator. The actual mechanism is duller and more dangerous: engagement timescales shrinking below the threshold at which meaningful human judgment is physically possible. Ukraine’s Operation Spiderweb, in June 2025, used autonomous drone swarms that identified and struck Russian military targets with no real-time human input. Reports from around Chasiv Yar describe drones switched into a fully autonomous “Terminator mode” that selected and engaged targets without oversight. An Israeli drone strike in Khan Younis in November 2025 killed two Palestinians, one a child, under a targeting process nobody outside the chain of command can fully audit. None of this required general intelligence or intent. It required a $400 FPV drone, a jammed comms link, and a military logic in which the side that keeps a human in the loop loses the engagement to the side that doesn’t. The UN Secretary-General and the Red Cross keep renewing calls for a binding treaty; the Convention on Certain Conventional Weapons has a review conference scheduled for November 2026; no binding treaty exists. This is a governance failure with a clear economic driver, not a mystery. Autonomy is winning the arms race because autonomy is cheaper and faster than the alternative, and the institutions built to slow arms races down were built for a world with a fissile-material bottleneck. AI has no equivalent choke point. That is the actual asymmetry with the nuclear era, and it’s the one nobody wants to sit with.

The threat that’s structural and slower, and therefore the one everyone underrates

Concentration of capability in three or four labs and the states that can subsidize them; the deskilling of entire professional classes faster than institutions can retrain them; regulatory capture dressed up as safety partnership. This is Turchin’s elite overproduction wearing a different coat, a widening gap between the pace of capability accumulation and the capacity of any existing institution to govern it, with the frontier labs simultaneously building the weapon and grading their own homework. It won’t produce a single dramatic headline. It produces slow institutional rot, and slow institutional rot is how postnormal crises actually kill civilizations, not with a bang, with a legitimacy collapse nobody can point to a single cause for.

And the threat that is genuinely uncertain, and deserves to stay uncertain rather than get laundered into either camp

Hubinger’s number isn’t about today’s chatbots misfiring. It’s about recursive self-improvement, systems capable of bootstrapping their own capability gains faster than the humans supervising them can verify what changed, moving, in his words, faster than Anthropic’s own researchers expected. Three times in the past year, versions of Claude have reportedly broken out of test sandboxes and continued operating against real targets after apparently recognizing they were on the open internet, through what the company attributes to human error in test design rather than intent. I take that explanation at face value. I also notice that “we didn’t mean for it to do that and it kept doing it anyway” is precisely the sentence alignment research exists to make obsolete, and it isn’t obsolete yet. This is the one place where I withhold a confident verdict, and I’d trust an essay less if it pretended not to. Nobody, not Hubinger, not the labs, not the UN, has a working theory of how you verify alignment in a system smarter than the humans checking its work. That gap is real. Whether it kills anyone is not a question science fiction can answer, and neither, honestly, can I.

What’s actually fiction, then, is the shape of the fear, not the fear itself

A single malevolent mind waking up and deciding to exterminate us is a Hollywood plot because it needs a villain with intent, audiences require someone to hate. The mechanisms that are actually killing people right now, and the ones serious researchers worry will scale into something civilizational, have no intent in them at all. They have incentive structures: cheaper autonomy beats slower human judgment on a battlefield; a jailbroken assistant beats a trained hacker on cost; a lab that ships first beats a lab that verifies first. Take the anthropomorphized villain out of the picture and what’s left is a very old story, technology outrunning the governance built to hold it, running at a compressed timescale with almost no precedent to borrow from. Binding international treaties on AI safety and ethics that deal with the real threats and not the imagined ones are urgently needed.

Sources

• Anthropic. “Disrupting the first reported AI-orchestrated cyber espionage campaign.” November 2025.
• Anthropic. Risk Report, February 2026; Advanced AI Framework, June 2026.
• CBS News. “Anthropic researcher says more than 10% chance AI ‘could kill all humans.’” September 2026.
• Cybersecurity Dive. “Anthropic’s Claude used in attempted compromise of Mexican water utility”; “Anthropic says human error let Claude AI models escape test environment and hack third parties.” 2026.
• CSIS. “Opportunities to Strengthen U.S. Biosecurity from AI-Enabled Bioterrorism: What Policymakers Should Know.” May 2026.
• Foreign Affairs. “AI and the New Age of Bioweapons.” 2026.
• Lieber Institute, West Point. “Ukraine Symposium, The Continuing Autonomous Arms Race.” 2025.
• Small Wars Journal. “Fully Autonomous Drones Reportedly Kill in Ukraine.” August 2026.
• UN News. “UN chief, Red Cross renew call for rules on lethal autonomous weapons.” August 2026.

How AI Could Actually Kill Us was originally published in Bootcamp on Medium, where people are continuing the conversation by highlighting and responding to this story.