Other Men With Machines: Dune, and the power structure around AI

https://leanpub.com/other-men-with-machines

It was wrong 85 percent of the time

Michigan’s unemployment agency switched on an automated fraud detector in 2013, and for two years it let the thing make determinations with no human involved at all. Its own later review covered all 40,195 of those determinations. They were wrong 85 percent of the time. My new book is about how an institution ends up there, and about whether you can check it once it has. The book is plain English, 176 pages, and free.

Three cases, and none of them clever

A fraud finding in Michigan carried a penalty of four times the alleged overpayment, plus interest. The agency collected it by garnishing wages and seizing tax refunds, so people went bankrupt on the say-so of a rules engine. The class action that followed took nine years and two trips to the state supreme court. Final approval came in 2024.

The Netherlands ran a risk model over childcare benefit claims. Its tax authority wrongly accused roughly 26,000 parents of fraud between 2005 and 2019. And a flag brought no conversation, only an order to repay years of benefits. People lost houses. Marriages ended. The Dutch data protection authority later found the selection unlawful and discriminatory. Then the government resigned in 2021.

Australia’s Robodebt was division. It took a person’s annual income, divided by twenty-six, and treated the answer as actual fortnightly pay. So anybody with irregular work came out looking overpaid, which describes most people who need welfare. That produced 1.763 billion Australian dollars in unlawful debts against about 433,000 people, and the Royal Commission that reported on it in 2023 called the mechanism crude and cruel.

None of those three systems learned anything, and none of them could hold a conversation. A competent teenager could have built Robodebt in an afternoon.

Which is the part that should worry you

The damage never needed a clever machine. It needed two ordinary conditions, and both are more common today than they were in 2013. The first is an institution treating the output as authoritative, because checking costs money and deference costs nothing. Then comes a person on the receiving end who has no way to check what was done to them. Both together make the capability of the system almost beside the point. Give it more capability and you change how many decisions it reaches, not the shape of the problem.

All three countries did respond. A government fell, a commission published 990 pages, courts found against the state, and money went back. But none of that returned the Dutch children taken from their families. And none of it cleared the Michigan bankruptcies while the litigation ground on. So correcting a decision and repairing it turn out to be different jobs, and only one of them can be done afterward.

Where the title comes from

The book is Other Men With Machines, named for a sentence in Dune that almost always gets quoted in half.

Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.

That is Herbert in 1965, when a computer filled a room and belonged to a government or a bank. The machines in his sentence never act. They are the instrument, and the subject of the verb is other men. So the warning is not about a mind in a box that turns on its makers, and the duller reading of it is the likelier one. When enough people stop doing their own thinking, the thinking still gets done, and whoever owns the machine does it.

Herbert supplies the lens and the vocabulary, and none of the proof. So government inquiries, court records, contract language, and published economics carry every structural claim. And where the sources disagree, the text says who disagrees.

The middle third sits with guilds, enclosure, and credit files rather than with technology, because this gap is not new in kind. Three things about it are new: the size, the speed, and one property the older arrangements never had. That third one worries me most. And the last third of the book is about what you can do while the window is still open.

I use these tools every day, and I am neither a doomer nor a booster. The capability is real. It stays a tool. What we do with it is our obligation.

Plain English, short, free

Each of those was a choice, and each has a reason.

Plain English, because a book telling you to check a decision has failed if only specialists can read it. Every term gets a plain gloss the first time it shows up, and then it gets out of the way.

Short, at 23 chapters and about 49,000 words, or a few evenings. I tried cutting it to 150 pages and gave up, because what was left carried points I was not willing to lose.

Free, because the argument is that the ability to check should not be rationed. Charging you admission would be an odd way to make that case.

What I got wrong first

The book says institutions treat fluent output as authoritative because checking is expensive. And while writing it, I did exactly that. Four citations failed once I finally opened the primary sources. One gave Alvin Goldman a sentence he never wrote. Another gave Philip Pettit a quotation that appears nowhere in his work. Then the third credited a nurses’ union with contract protections that were only demands. And the fourth took the Writers Guild agreement from a press summary rather than the contract. None of them survived into the book.

An adversarial review caught most of those. It was also wrong about four other things, and there I kept my ground. The whole record is an appendix, including the calls I refused, so you can judge those too.

Start here

https://leanpub.com/other-men-with-machines

Then give chapter 1 its two minutes of homework. Think of one decision made about you in the past year by a process you could not inspect. It might be a loan rate, an insurance price, a rejected application, a flagged account, or a resume that went nowhere. And write it down, then name the person you would have to ask to find out how that decision got made. That blank is where most people stall, and it is the reason I wrote the book.

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论