We’re not ready for the e/Acc × Longevity preference cascade

Anti-aging sentiment might go rapidly mainstream, in the same way AI Safety just did.

Recently, the AI safety community has been enjoying a massive preference cascade that has rapidly moved AI x-risk concerns into mainstream political discourse

Why did this happen?

• The basic idea seems like it should have been obvious for a long time: “IF we create self-improving machines that rapidly become much smarter than humans, THEN that seems like that story might not end well for the humans, so we should be very careful.”
• If the idea is so sensible, why are people only now endorsing it? Proximately, because of HuggingFace, Jacob Coxon, etc. But in a larger sense, clearly the trigger was that the premise (“IF we create self-improving machines…”) no longer strikes people as absurd. Instead, it seems worryingly plausible.
• This not only unlocks huge amounts of energy from many new people people thinking the “IF” half of the statement is likely, but also – weirdly, illogically– seems to furthermore cause lots of people to change their minds on the “THEN” half of the statement, too.
• Why did the flip happen so quickly? This is actually the case for lots of movements – see Richard Ngo’s truly insightful post “Power Lies Trembling” for much more on the dynamics of preference cascades.

But this isn’t the only preference cascade waiting in the wings, as we continue to scale AI capabilities. Consider another throwback rationalist classic:

• “IF it’s possible to cure diseases and slow/reverse the mechanisms of aging, THEN we should do so with the greatest urgency, because death is extremely bad.”
• Why aren’t normal people on board with this message? The rationalist in me still burns with frustration at people’s willful obliviousness: “What is the problem?? How can you fail to recognize this! Do you not AGREE that DEATH IS BAD?”
• The problem (in the vast majority of cases) is obviously not that normies literally think death is good. Rather, they think that the first part of the sentence is absurd – everyone knows it’s intractable to make progress against aging. Those saying otherwise are overwhelmingly likely to be selling snake oil.
• Thus, few people believe the first part of the sentence is likely. And this causes them to – weirdly, illogically – reject the second part too.
• But, AI progress at biomedical research might suddenly make it look tantalizingly plausible that we could make real progress on aging. (Or maybe just, like, cancer. Which would also strike many people as a huge deal.)
• The resulting preference cascade might unlock prodigious amounts of energy – perhaps much greater than what we’re seeing now with AI x-risk.

The labs are probably working towards this.

Just like the idea of AI x-risk, the hope of using AI to unlock medical cures and other transhumanist goods is widespread among both employees and corporate leaders at the frontier AI labs, rationalists, etc.

Insofar as they aren’t devoting 100% of their energy towards steering to RSI + trying to do some critical safety work along the way (these two things seems to be most of what they’re doing), Anthropic also seems to be steering somewhat out of their way to push medical capabilities – consider for instance how centrally medical science features in Dario’s essay “Machines of Loving Grace”, their detailed coverage of the biology capabilities of each new model, their development of a project-glasswing-style “Life Sciences Verification Program” for qualified access to bioweapons-research-capable models, how they've allegedly opened a wet lab for biology research, etc. Deepmind also seems to have also devoted a disproportionate amount of effort towards biology research, via efforts like AlphaFold and many others.

Lots of smart people seem to believe that AI-for-longevity is possible, and they’re working hard to make it happen. Maybe they’ll pull it off!

Although perhaps a cascade wouldn’t happen or would come too late to matter.

Maybe anti-aging is just fundamentally too weird to enter the overton window?

• I maybe could’ve believed this before, but not after I just saw everyone spin on a time and start talking about risks from uncontrolled AI agent swarms.

Evidence for the tractability of anti-aging medicine might build very gradually and smoothly.

• As with mathematics – first we’ll see AI making minor contributions, assisting human mathematicians / medical researchers, proving small lemmas / developing new medicines in areas that always seemed especially amenable to AI analysis. Then only later will we get more dramatic cures that are clearly mostly the result of AI labor, not human efforts.
• But of course, this slow and continuous buildup of evidence was also the case with AI risk! Nonetheless the social landscape moves in more abrupt jumps; see again the post “Power Lies Trembling” for why this happens.

A longevity x e/acc movement might simply take too long to gather steam, thus arriving too late to be relevant.

• If superintelligence comes extremely soon, there’s limited time for weird new social movements to arise and have impact. eg, if (per the AI Futures Model) we get Top-Expert-Dominating AI that starts churning out serious medical breakthroughs in summer 2028, escalating to superintelligence by winter 2028, that perhaps only provides about six months where anti-aging-related mass public sentiment might be relevant, before the future is essentially decided one way or another.
• The situation with medicine is quite a bit worse than mathematical proofs that can be formalized in Lean. In particular, even if superintelligent AI starts delivering unambiguous “millennium-prize-equivalent” medical breakthroughs, there will probably be a fixed time delay while we prove out those experimental treatments (developed in-silico or via laboratory experiments) into actual human patients, via the usual process of human clinical trials. So there might be, say, a fixed 1-year delay before the medical establishment and general public will accept any given AI treatment as confirmed to work in real humans. That fixed delay would be very consequential in AI-2027-style takeoff scenarios, perhaps delaying a preference cascade past the point where it could possibly matter.
• So, this whole analysis is most relevant for medium-term and long-term futures where AI development either intrinsically ends up taking longer than the labs expect, or where we succeed politically in implementing a regime of paced restraint on AI development.

A longevity preference-cascade would be great! Except, by default, the energies unlocked might flow directly into e/acc :(

This cultural earthquake, or avalanche, or dam-breaking, catalyzing massive societal energies for the fight against aging, is something that the longevity community has dreamt about for for years. Unfortunately, the anti-aging preference cascade might, in practice, channel all its energies into AI-accelerationist sentiment.

• If we start seeing medical breakthroughs unlocked by AI scaling, then it DOES fundamentally make sense to view this, other things equal, as a reason to continue scaling up AI.
• But on top of that, the companies presenting the results (frontier AI labs, most likely. or perhaps some biology-specific AI startups) will have every incentive to use this to boost their own PR and public support for accelerating their work.

Before our current AI safety moment, surely there was nevertheless widespread anxiety about humans being replaced by AI. (See eg the dramatic unpopularity of anything AI-themed in various polls.) But since the idea of AI x-risk wasn’t being discussed publicly, it strikes me that those worries were essentially being “sublimated” (in the psychological sense) in favor of somewhat vague and incoherent anger about datacenters, flock cameras, copyright infringement, et cetera. This sublimation process artificially boosted those movements but (until now) greatly weakened public enthusiasm for AI x-risk mitigation.

Right now, humanity’s terror at the prospect of death is similarly sublimated in a variety of ways, since after all there doesn’t seem to be anything practical we can do about it. But it’s also such an intense terror that it arguably drives a huge percentage of human motivation.

Meanwhile, today’s e/acc movement is honestly kind of pathetic. Its main argument within the current overton window is just that AI is going to make a lot of money. (“...though not necessarily for YOU, average american worker – I was more thinking about my silicon-valley investor friends.”) This is enough to convince Trump (at least for now) that opposing AI safety is essential for making sure the stock market keeps going up. There are a few true e/acc believers who understand that the biggest potential prize is longevity (alongside many other transhumanist goods, like intelligence enhancement). This has little traction with the public, and it seems kind of kooky to even imagine that ordinary people (people like my mom! my neighbors!) might suddenly transform into anti-aging transhumanists.  But this was exactly the state of AI safety discourse a few months ago, when a few experts cared deeply about x-risk but public discussion was an incoherent swirl of mere datacenter skepticism and it seemed crazy that “notkilleveryoneism” would go mainstream any time soon.

Historically, the strength of “e/acc x longevity” has been limited by the fact that most people smart enough to grasp the allure of this idea (eg AI lab CEOs, rationalists, etc) have also been persuaded by the argument for AI risk, and thus for caution despite the allure of e/acc x longevity. But a future preference cascade among the mass public (cross-pressured by anxieties over superintelligent AI and bioweapons, versus enthusiasm over medical progress) might not play out the same way. So it seems to me that the combination of e/acc ideas, and a mass public that for the first time truly believes in imminent cancer cures, aging treatments and other medical miracles, could end up going in some pretty crazy directions.

IMO the (small, very inside-baseball) controversy over Nick Bostrom’s paper “Optimal Timing for Superintelligence” reflects AI safetyists’ understanding of this dynamic. “Don’t speak too loudly about this very selfish, dangerous idea!” people seem to be saying. “If anyone else hears, it might set off the latent anti-aging preference cascade!” Personally, I don’t think that analytic-philosophy essays (or, for that matter, LessWrong posts like this one) are going to be the match that starts this particular fire. But the kindling is building up.

Of course I’m not saying we should oppose medical cures!

I’m in a weird position here, I guess. I’m not saying we shouldn’t try to use AI to develop some medical cures. Medical cures are great! (Other things equal, of course. Where “other things” includes x-risk from both reckless AI scaling and from enabling easier bioweapons development.) Even Eliezer Yudkowsky thinks we should try to go for the medical cures, if we can do so in a responsible way: in his Time open letter of 2023, he says “The moratorium on new large training runs needs to be indefinite and worldwide... If I had infinite freedom to write laws, I might carve out a single exception for AIs being trained solely to solve problems in biology and biotechnology, not trained on text from the internet, and not to the level where they start talking or planning; but if that was remotely complicating the issue I would immediately jettison that proposal and say to just shut it all down.” Even as he also says that it would naturally be “incredibly selfish and evil” to roll the dice on misaligned AI as described in Bostrom’s “Optimal Timing” paper.

So then what SHOULD we do?

idk! Here are some thoughts for AI safety groups and funders:

• Of course there are many variations on the basic core argument “if we die to misaligned AI in the process of rushing for immortality, then obviously we don’t get the immortality!” which will need to be made. AI safety groups could prepare ahead of time various ways of making that argument convincingly. But this probably doesn’t take too much preparation; I dunno.
• Something that might take a bit more advanced planning would be preparing detailed policy roadmaps (along the lines of Yudkowsky’s aforementioned “I might carve out a single exception…”, and analogous to the many current schemes for AI governance, treaty verification, pacing RSI, etc) for trying to defuse the memetic nexus of AI scaling x medical progress, by outlining ways to narrowly focus AI scaling on pushing continued medical progress while simultaneously slowing down more general AI development.

Beyond that, I think AI safety folks might also want to establish some anti-aging bona fides well ahead of time. Some ideas for how to do that:

• Today’s pro-pacing and pro-pause AI safety groups often go to great pains to explain that they’re NOT generally opposed to new technology, business, or construction, (eg they favor lots of innovation in nuclear power, lots more housing, are very pro- economic growth and abundance, etc).
• Similarly, during a future anti-aging preference cascade, the AI safety community will want to be able to point to strong anti-aging bona fides.
• This will naturally include a record of rhetorical support for the cause of longevity (eg all those classic rationalist blog post). But should go beyond this to also include (in the case of large funders like Coefficient Giving, perhaps) a record of concrete support for longevity science, political reforms, etc. For example (and many of these are indeed already funded to some extent by EA/rationalist groups):
  • Detailed political agendas for FDA reform, “clinical trial abundance”, and related ideas about speeding up medical progress. Including general metascience agendas for improving the efficacy of scientific research generally.
  • Support for actual biomedical science, both relatively mundane (research on cancer, alzheimers, etc), but perhaps in this situation the crazier the better (human genetic engineering, research into the fundamental mechanisms of aging, stuff like antimalarial gene drives for eradicating diseases)
  • Maybe keep a few truly crazy, radical anti-aging ideas in your back pocket, like Robin Hanson’s recent concept of “maybe we should transform our government so it follows a futarchy prediction market focused entirely on trying to bring closer the date we achieve human immortality”. Just so nobody can try to politically flank you by being more anti-aging than you – you can always pull out your insane plan to completely refocus all of society around longevity, as ironclad proof that (despite your AI safety views) you’re firmly in the anti-aging wing of the anti-aging party. (You don’t then have to actually implement the insane plan IRL.)
• It might also make sense to proactively build stronger ties with today’s longevity community. Fortunately we’re hardly starting from scratch here, as many anti-aging fans, such as Vitalik Buterin, are already very EA-adjacent or rationalist-adjacent.
• I expect the main focus of an e/acc preference cascade to be focused on medical cures and longevity. But to the extent that it might also bring other transhumanist goods somewhat into the mainstream, it might be useful to get ahead of the ball on those as well. (eg things like embryo selection for intelligence enhancement, human augmentation via neuralink-style devices, etc. though unlike aging research which is perceived as merely kooky, a lot of this other stuff is actively toxic in today’s cultural climate and would be considered a huge PR hazard by most orgs!)

All the above stuff, taken together, sounds like I’m pitching a maximalist agenda for basically transforming EA/rationality into a longevity-focused movement, which would be absurd. I’m instead just saying that we should consider doing some of this stuff, a little bit more than we already are.

The overarching idea here is that AI safety should be ready to present an alternative vision of radical longevity progress – one that isn’t reliant on breakneck general AI scaling. This plan doesn’t necessarily need to explain how it’ll actually OUTPERFORM a plan of breakneck scaling at achieving medical progress (which might be unrealistic). But during a preference cascade, we’ll need to address the issue in some way. (Just like how it behooves Republicans to have some kind of notional plan even on issues that are unfavorable for them, like healthcare. And similarly behooves Democrats to make some sorts of commitments on immigration, even if they’d mostly rather people not talk about the issue since it’s unfavorable for them.) Lest we end up looking like the e/accs do in the current AI safety moment – losing the argument in the eyes of the public, reduced to weak rebuttals, weird ad hominems, and transparent attempts to change the subject.

The “support for FDA reform and scientific research” stuff earlier is important not just so we have a track record to point to, but also because after a longevity preference cascade, there will be strong public demand for taking action on aging. People will want a roadmap, they’ll want big initiatives being launched and funded, and so on. So, EA / rationalist groups should perhaps fund some of these things in small-scale now, with a view to being able to impressively scale them up later when the demand arises. Cryonics also seems particularly relevant here as a shovel-ready plan that acts as a “Plan B” alternative vision for longevity. Ideally the idea of a crash program to improve the science and scale up the practice of cryonics might be capable of relieving some of the desperate urgency and time pressure that the public might feel about the problem of aging after a preference cascade, and would answer some of the newfound political will to immediately “do something about aging” in a way that doesn’t involve further scaling AI.

What about the labs themselves; what should they do?

• Whatever AI labs (whether Anthropic, OpenAI, a biology-specific organization like Arc Institute, etc) find themselves in a position to announce newly-developed cures to the world, will have huge amounts of power to frame the ensuing discourse.
  • Consequently, they should think carefully about how to frame their announcements in a way that doesn’t fuel too much desperate e/acc enthusiasm on part of the mass public.
  • Unfortunately, this goal exists in tension with incentives to hype people up about the amazing humanitarian potential (and financial value) of their new technology to the maximum extent possible.
• Biomedical capabilities come hand-in-hand with scary bioweapons capabilities, so expressing the balance of risks vs rewards in a nuanced way would probably be helpful.
• Reiterating the fundamental message that “if we die to misaligned AI in the process of rushing for immortality, then obviously we don’t get the immortality!” will also be important for people to hear coming from the labs themselves.

Final thought: in all the above discussion I’ve made it seem like it’s important to have all these plans ready (to channel the energy unlocked by a longevity preference cascade) for the purpose of performing PR judo and deftly deflecting a movement that might damage the cause of AI safety. But obviously there’s another good reason to have such plans ready in advance: properly directed, a longevity preference cascade could be really great! It would be amazing to use all that energy for good, using it to fuel more ambitious medical progress in a way that isn’t existentially dangerous. So, for that basic object-level reason, it would clearly be great to have some shovel-ready plans scoped out for rapidly scaling up ambitious medical research.

1. Longevity fans will complain that progress on aging is actually more tractable than the public thinks. I agree! But look at the history of the field: resveratrol? senescent cells? some mouse trials? None of this stuff has produced a genuine miracle for human life expectancy; and meanwhile there have been lots of weird fads and hype. It is very understandable that a distant general public (who has a harder time sifting fads from more promising science) perceives this all as snake oil, or at best overly-optimistic empty promises in the vein of “commercial fusion power is just 10 years away”.
2. I’m not sure what exactly how transhumanist a longevity cascade would end up looking like. Would it mostly be focused on enthusiasm for curing normal diseases, like cancer or Alzheimer, without touching the still-taboo idea of anti-aging? Could it possibly get crazier than anti-aging, and into other transhumanist goods, like BCIs or etc? Perhaps most importantly, how much would people connect the dots between medical progress and AI policy? Maybe everyone would get excited about medical breakthroughs but this would just translate happily towards pressure for more research funding, FDA reform, etc, without politically connecting to any AI issues? Or maybe people would go full on "worship the coming machine god" at the first whiff of some new AI-developed antibiotics for treating hospital staph infections, or whatever. Who can say!