What Decides How Much AI Changes the World

People keep asking why I left AI research. I didn’t.

I focused on what made prior AI predictions wrong. When I wrote down the research agenda I thought had to exist, the shortest name for it was Abundance.

Here is how I got there.

The forecast that changed my mind

When ChatGPT came out, a lot of thoughtful people thought releasing it was reckless. The model could write fake news articles in seconds. Personalized propaganda would flood the internet and we’d lose the ability to tell what was true.

That did not happen.

The forecast assumed people believe what they see. We don’t. We believe people we trust, and things our side already believes. Only about 1% of users see most fake content online, and mostly those who went looking for it. The share of people with conspiratorial tendencies is roughly stable across history.

Flooding the supply of fake content does not create demand for it.

Something sat between the AI capability and what actually happened in the world. I started calling these societal frictions.

Why almost nobody maps these

It’s hard.

You usually can’t get the answer from one expert. Disinformation journals publish people who accept the field’s premise, which is that supply is the problem. I only learned how much the demand side mattered by leaving the field and reading psychology and the history of conspiratorial thinking and media.

You can’t get it from the news. Most elite universities now give automatic full scholarships to American families earning below the median income. The bottom 40% of households qualify for a Pell grant that covers community college tuition. You would not know that from coverage written for high-income readers.

You can’t get all of it from being inside a field. A homebuilder can list every thing blocking construction: permits, neighborhood associations, zoning boards. But the builder doesn’t see the incentives that make politicians keep those in place.

Getting misinformation wrong taught me to be humble about what else we were getting wrong. AI safety wasn’t doing this work, partly because it takes setting aside the instinct that superintelligence makes the details moot.

So I made a list of the sectors that matter most to people’s lives, and started working through them.

Bioweapons

I started here because I’d spent years on biosecurity.

Say a model tells you exactly how to pull anthrax out of soil. You still have to sort through the billions of bacteria in a tablespoon of dirt, then test whatever you isolate to find out whether you’ve got a dangerous strain or a harmless one.

Every bioweapon is different, but the pattern is similar. Hurting a lot of people requires doing something physically very hard, like milling a toxin fine enough to hang in the air without destroying it. Stitching DNA together is hard. Turning that DNA into a living virus is much harder, and it needs controlled materials you can’t order without an institution.

I mapped the full end-to-end sequence, physical steps included, instead of starting from what advocacy organizations were saying. I then understood what biologists were saying all along: a lone attacker with no institution will have an extremely hard time because biology is weird.

Most bioweapon attacks come from people inside institutions or governments. For them the work is difficult but doable. AI doesn’t change much for the people who can already do it.

Which is why I now watch lab automation more closely than model capability.

Drugs

We could have a tuberculosis vaccine. The science is already here.

Approval takes years and hundreds of millions of dollars. For all those years the company has to stay solvent without selling anything. Only a firm with deep reserves or many drugs can wait that long. Thus, a big filter on which drugs get made is whether a large company is interested.

Tuberculosis patients are mostly poor, so this is a bad return on investment.

The same economics also makes new antibiotics unviable. Companies with new working antibiotics keep going bankrupt, because doctors know to use them as a last resort.

Clinical trials are another gate. A cheaper alternative is learning from patients already being treated, which requires standardized records shared across providers. American hospitals can’t read each other’s files. That is a big reason the UK ran the COVID vaccine trials.

People ask whether AI can replace trials by simulating a human body. That depends on whether bodies are simulatable. There are reasons to doubt it. Weather forecasting fails past about ten days because tiny errors compound until the simulation drifts away from reality. A body has more moving parts than a storm.

Health records are a very promising target for AI. Even there, whether the bottleneck is technical, legal, or something else is unclear.

Science

Scientists are rewarded for volume of papers, which rewards incremental work. Grant cycles do the same, running 12 to 18 months.

Breakthroughs come from the opposite behavior: overturning consensus, funding work that looks unpromising, and running enormous multidisciplinary efforts like the Human Genome Project. mRNA went unfunded for years because it seemed too weird!

AI already helps scientists work faster. Whether it produces breakthroughs depends on whether these structures change, or on AI bypassing academic science altogether.

The sectors people depend on daily

Which sectors shape people’s lives most?

Housing is 33% of household spending, transportation is 17%, food is 13%. Working parents spend 10 to 20% on childcare. Healthcare is 18% of GDP.

AI is not poised to transform all of these first. But obstacles I found will also hit AI-first companies.

• Existing medical providers can veto new clinics. In about 30 states, opening or expanding requires a certificate showing the community needs it. Incumbents get to weigh in. An AI-first clinic still has to open a building.
• In some cities, daycares are legal on less than 4% of the land. I don’t expect robots to be watching kids anytime soon. If they ever do, they’ll still meet zoning boards.
• Many states won’t let nurse practitioners do what they trained to do, because the doctors’ lobby fought it. AI-powered services will meet that lobby.
• Waymo is kept out of DC and other cities, though its cars are in far fewer serious crashes than human drivers.

My next medical deep dive is on how healthcare prices get set. If the doctors’ lobby sets them, prices won’t necessarily fall when automation makes the work cheaper.

Everything I have written so far (except the bioweapons section) is the core of abundance.

This work is undersupplied

To know how the world is going to change, you have to know how it works now. Not how headlines describe it, or how a politician frames it, or how your own field talks about it. Those are all simplifications.

To know what to do about the housing shortage, someone had to know both the national numbers and the hyperlocal details of how one particular building gets approved.

This research is narrow when you do it. People deep in AI safety tell me it’s nice that my Substack gets attention, then ask why I write about daycare when the world is ending. But one of the headline talks at Effective Altruism Global 2024 conference made the case for sociopolitical research too. I’m not alone.

AI research shouldn’t always look like AI research. Some of it should be about the world.