Who Tells the Billionaire He’s Wrong?

TL;DR: Technical brilliance is not human wisdom. As AI gives a small group of technology leaders more influence over how the rest of us work and live, one of their most important skills may be recognising what they don’t understand — and keeping people around who are willing to tell them so.

Mark Zuckerberg has been thinking about humanity again. In August, he published a 6,500-word manifesto, The Future Is for Everyone, about superintelligence, liberty and individual empowerment. Meta’s goal, he says, is to put personal superintelligence into the hands of billions of people.

There are ideas in it I agree with. I would much rather have technology CEOs thinking about the consequences of what they build than ignoring them. But reading Zuckerberg on human agency also made me think about Silo, one of my favourite TV series of recent years.

That may seem like a strange connection. It isn’t.

Smart enough to build it

Silo is about thousands of people living underground, generations after some catastrophe made the outside world uninhabitable. It is a great science-fiction mystery, but what interests me most is the question underneath it: who gets to decide how other people should live when they believe they know something everybody else doesn’t?

I won’t spoil the story. What I find fascinating is that the people controlling the system don’t necessarily think of themselves as villains. There is a logic behind what they do. They have information others don’t have and believe terrible consequences could follow if they lose control.

That is much more interesting than simple evil. Give an intelligent person enough power, a model of how the world works and absolute confidence that the model is correct, and almost anything can start to look reasonable.

I kept thinking about that while reading Zuckerberg… He still complains about The Social Network, and on the facts he has a point. He has said that the film reproduced tiny details of his life while getting his motivations fundamentally wrong. Aaron Sorkin and David Fincher made a drama, not a documentary.

But the metaphor aged better than the biography.

The lasting idea in the film isn’t that Zuckerberg created Facebook because he wanted girls or social status. It’s that a young man who was brilliant at computers was much less brilliant at understanding people, society or the messiness of human life. He had also shown a willingness to disregard some basic rules of human decency in business and tech. As a young man, he famously called his early users “dumb fucks” for trusting him with their data. And then, very quickly, he acquired enormous power over how people related to one another.

That power eventually extended far beyond Facebook. Instagram, WhatsApp and Meta’s recommendation systems became part of the social infrastructure of billions of lives.

I think about three children I saw recently (several few months ago, really) outside my son’s old school. They were probably 10 or 11, physically standing together, but each was staring at a phone. Nobody was talking or playing. They had access to technological capabilities that would have seemed like science fiction when I was their age, yet I’m not sure the technology was making that particular moment richer.

So when Zuckerberg writes that technology should empower individuals, I don’t disagree. I just think it is worth asking what empowerment means, because he has answered that question differently before. In January 2018 he told Facebook’s product teams to stop optimising for relevant content and start optimising for “meaningful social interactions”, pushing posts from friends and family back up the feed and public content down it.

He said he expected time spent on Facebook to fall, and it did. Publishers spent that year complaining about the collapse in their reach, which is the clearest evidence that he meant it. Then TikTok arrived and proved that recommended video from strangers held attention better than anything your friends had to say, and the feed and what he says changed again.

Intelligence isn’t wisdom

This isn’t only about Zuckerberg. Elon Musk may be a more extreme example of the same problem.

I used to find some of Musk’s ambitions inspiring. Electric cars, reusable rockets and making humanity multiplanetary are ideas on a scale few entrepreneurs even attempt. There was a sense that the companies existed in service of something larger.

Today I find it harder to separate that rhetoric from his pursuit of power, attention and personal importance. Perhaps that’s unfair; none of us can know another person’s motives. But after enough years, actions become more useful evidence than declarations of intent.

Nor do I put Sam Altman and Dario Amodei in exactly the same category. I see more consistency in some of Altman’s thinking, and Anthropic has devoted serious attention to AI safety. But they operate inside the same strange moment in which a handful of technology leaders are no longer merely deciding what products to make. They are making predictions about how millions of us will work, what skills will remain valuable and, ultimately, what role humans will have in a world of increasingly capable machines.

Technical intelligence isn’t enough for that. You need some history, psychology, sociology and politics, but even reading all the right books doesn’t solve the problem. You need contact with people who see the world differently, experience of being wrong, and enough humility to recognise that humans rarely behave as neatly as models predict.

Most importantly, you need people around you who can say no.

The AI-native company meets reality

The corporate version of this problem is already playing out.

For the past couple of years, a seductive vision has spread through technology companies. AI will allow tiny teams of exceptional people to do what once required hundreds. Managers, product managers, junior employees and specialists can be removed. The remaining “builders” will command armies of agents.

Meta tried to move towards something like that. Reuters’ investigation into Project OT describes plans for much smaller “AI-native” teams and scenarios in which many teams could shrink by as much as 60%.

Then came some revealing numbers. Internal code changes were up 220% year-on-year, but changes resulting in new or improved features for users increased only 36%. Major technical and security incidents rose 40%, while time spent firefighting increased 70%.

More output wasn’t the same as more value.

Zuckerberg eventually abandoned plans for a second company-wide round of cuts that had been considered for November. Meta is now talking about “betting on people” again. Andrew Bosworth has acknowledged that the company did an “atrocious” job explaining and implementing parts of its reorganisation and damaged employees’ trust that their expertise would be valued.

Changing your mind when evidence changes is good management. The problem is that organisational experiments don’t happen only in PowerPoints and spreadsheets. They happen to people.

I was laid off in May after almost five years at Cloudflare. Companies sometimes have to eliminate jobs, and I’m not arguing otherwise. What troubles me is the increasingly AI-pilled version of corporate restructuring, where layoffs can start to feel like a pissing contest between tech CEOs. Look, we cut 10%. We did 15%. We’d never done layoffs before, but we did 20%. Headcount reduction itself becomes part of the AI story (although it seems that it maybe wasn’t what some thought it would be): proof that your company is leaner, faster and somehow further into the future.

That’s a dangerous game when decisions about people’s careers are being made on confident assumptions about what AI will be able to do next year, before those assumptions have actually been demonstrated. It’s worse when the cuts themselves help sell the AI narrative to investors, markets and other CEOs, but not more than that.

Once you’ve been on the receiving end — and watched a badly managed process let go of talented people who helped build so much of the company — the language starts to sound different. Efficiency. Flattening. AI-native. Even phrases such as “building for the future by reducing significantly our workforce.” Sometimes they describe necessary change. Sometimes they’re just cleaner words for decisions made with far more confidence than evidence.

Predictions aren’t harmless

Dario Amodei has been particularly confident about what comes next. In 2025, the Anthropic CEO predicted that AI could eliminate half of entry-level white-collar jobs and push unemployment to 10–20% within one to five years. Axios reported the prediction at the time. He is now reportedly saying AI could replace software engineers within six to twelve months.

Maybe he’s right. The progress of AI has been fast enough that dismissing such predictions would be foolish. But the people making these predictions aren’t detached academics forecasting the weather. They run the companies building the technology, and people listen to them.

Students listen too. John Burn-Murdoch recently showed in the Financial Times that US computer-science enrolment and UK applications have fallen after years of growth. We don’t know that AI caused the reversal, but repeatedly telling a generation that software engineers may soon be obsolete surely changes how some people think about studying computer science.

If the forecasts prove wrong, the consequences don’t disappear with the forecast.

The same applies inside companies. A CEO hears that AI agents will replace half a department. An investor asks why margins aren’t improving faster. Another CEO boasts about running the company with fewer people. Soon an uncertain technological forecast has become an organisational strategy.

This is where values become interesting. Values are easy to proclaim when they cost nothing. The real test comes when they cost money, power or competitive advantage.

Who tells the CEO he’s wrong?

I suspect this will become one of the defining leadership questions of the AI era.

Who is in the room when the big decisions are made? Is it the engineer predicting that agents will replace half the company next year, the investor demanding efficiency and the executive who knows that agreeing with the CEO is good for a career?

Or is there someone with enough experience and independence to ask what happens if everyone is wrong?

What work are those supposedly redundant people actually doing? Who understands the customer? Who develops younger employees? Who notices when five brilliant engineers are efficiently solving the wrong problem? Who remembers why the company tried something similar six years ago and abandoned it?

These aren’t arguments against ambition. I want ambitious people building extraordinary things. I want AI researchers attempting what sounds impossible, Musk launching rockets, Zuckerberg spending billions on technologies that may fail. But ambition without limits isn’t wisdom, and intelligence without doubt becomes dangerous once it has enough power to act.

I do wonder what Steve Jobs would be doing at Apple now. He belongs in this essay more than I would like, and not only as a warning. He was as convinced as anyone that he knew what people needed before they did, and he was hardly famous for taking no for an answer. But he also told Walt Mossberg that a company had to be run by ideas rather than hierarchy, because “the best ideas have to win, otherwise good people don’t stay.” Apple employees even had an annual award for the person who stood up to him best. Jobs knew about it, and apparently enjoyed it.

He also arrived at technology from an unusual angle for the industry, through design, typography and a stubborn interest in how a thing felt in your hand (I kind of think he would have made some better decisions in terms of human use of the iPhone than Tim Cook). Whether any of that would have survived this particular moment, with these incentives and these markets, I have no idea. I do wonder.

I kept thinking about Silo while writing this, because the series is largely about what happens when a small group becomes certain they understand the system better than everyone living inside it. I’m writing about that separately, so I’ll leave it here.

Being smart enough to build the system is one thing. The harder question is who is left in the room to tell you when you are no longer wise enough to decide what happens inside it.

More on Silo soon.

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论