What people in AI really think about regulation

This article is an on-site version of our Swamp Notes newsletter. Premium subscribers can sign up to get the newsletter delivered every Monday and Friday. Standard subscribers can upgrade to Premium here, or explore all FT newsletters

Last week, I attended a high-level, closed-door AI conference in California. Guests included senior leaders across the sector, as well as several of their corporate clients and non-profit types trying to figure out how to make the technology less scary and economically disruptive: polls show that young people in particular are turning against AI.

It was a timely moment to be there, given the recent hoopla over rogue AI attacks. The previous day, President Donald Trump had met the industry’s top CEOs and proclaimed that the way forward was “tremendous self-regulation” on the part of the AI companies themselves. So, in one session at the conference, I asked the audience whether they thought that approach would work. Nobody raised their hand.

Good luck to Republicans selling a message to midterm voters that even tech bros don’t buy. Later, in another smaller session about cyber security, I got even more worried. Everyone expressed concerns about how exponentially fast the technology was moving, and how hard it was to even understand what was human and what was AI. As one participant pointed out, there are now 108 non-human actors for every human online, and verifying proof of (human) life is close to impossible. “We are in an existential mess when it comes to even understanding what is human online anymore,” said another.

It reminded me of when ChatGPT first appeared almost four years ago. Shortly afterwards, I got a call from a source who told me: “It’s going to be all about trust; you’ll only be able to trust what is nearest to you — what you can actually see, hear, touch.”

If that turns out to be true, it would only further the trend towards political and economic regionalisation and localisation that we are already seeing. One cyber security expert noted that clients were asking for increasingly individualised, bespoke security systems, because they don’t even trust other companies in their industry that they might be dealing with during daily business. Everyone knows AI isn’t secure and nobody knows how much the other guy has done to combat challenges inside his own business. So, the individual drawbridges are going up.

This means that the tribalism and fragmentation we are seeing in politics may be coming for business, too. Add in Chinese open-source applications (which are cheaper but arguably less secure) and the problem is amplified.

So, what is the solution? What can government do to really fix the problem? According to several people I spoke to, the answer is “nothing”. Part of the problem is that, unlike cutting-edge technologies of the past, AI didn’t start with the military — it was born in the private sector (albeit via other technologies like the internet itself that were funded by government). Unlike, say, nuclear weapons, which are largely owned and run by the military (and thus can be regulated via top-down international agreements), AI is everywhere.

Recommended

Of course, there are about 20 companies that run the internet and they all talk to one another. That seems to be what passes for regulation these days, and it’s clearly not good enough. I think we can safely assume that this administration won’t do better than it has done already.

But I want to pose a question to readers of this note, with a prize. What is the very best that could be done right now both to make AI safe and to keep the US in the game in terms of competing on the technology with China? I would love to hear ideas from you all and the person with the most interesting answer gets to engage in a longer conversation about the topic in a future note.

In the meantime, I want to push the conversation about the regulation into the area of AI and financial markets, about which my friend Andy Green, a former policy counsel at the SEC and Public Company Accounting Oversight Board, and political appointee at USDA, has much to say below. Andy, I’ve written about the risks of tokenising markets, a topic near and dear to you. What risks should investors be aware of here?

Recommended reading

• I was struck by this New York Times piece on homeless people in Portland creating their own purpose-built housing and co-operative communities, rather than staying in state-run shelters. This is an interesting experiment in self-empowerment, one that could also save taxpayer money: the shelter system is the biggest driver of the city’s $170mn budget shortfall. If there were some seed funding available for people to create such villages, I bet the idea would be replicable nationally. It reminds me a bit of the co-operative housing communities I’ve written about, albeit on a humbler scale.
• I loved this New Yorker profile of the writer Barbara Kingsolver. It’s interesting to me that in America, unlike the UK, it’s difficult to be accepted as both a literary novelist and a popular one. The same goes for historians. In Britain, you can both sell lots of books and be serious. In the US market, you often must choose to be one or the other (and are marketed in very different ways as a result). I think this comes down to confidence in culture and language, which I think Britons have more of.
• In the FT, my colleague John Paul Rathbone wrote a lovely piece about swimming the Bosphorus.
• And, at the risk of being self-promotional, I hope readers will have a look at my FT Weekend essay on why the US-China conflict today resembles the Great Game between the British and Russian empires in the 19th century. It’s adapted from my fourth book, Sea Change: America’s New Great Game in the Arctic, which looks at the great power competition in the High North, which very much resembles what’s going on in other areas like the South China Sea and Latin America. It will be published by Crown in the US on October 6, and Swift Press in the UK on October 8.

I was struck by this New York Times piece on homeless people in Portland creating their own purpose-built housing and co-operative communities, rather than staying in state-run shelters. This is an interesting experiment in self-empowerment, one that could also save taxpayer money: the shelter system is the biggest driver of the city’s $170mn budget shortfall. If there were some seed funding available for people to create such villages, I bet the idea would be replicable nationally. It reminds me a bit of the co-operative housing communities I’ve written about, albeit on a humbler scale.

I loved this New Yorker profile of the writer Barbara Kingsolver. It’s interesting to me that in America, unlike the UK, it’s difficult to be accepted as both a literary novelist and a popular one. The same goes for historians. In Britain, you can both sell lots of books and be serious. In the US market, you often must choose to be one or the other (and are marketed in very different ways as a result). I think this comes down to confidence in culture and language, which I think Britons have more of.

In the FT, my colleague John Paul Rathbone wrote a lovely piece about swimming the Bosphorus.

And, at the risk of being self-promotional, I hope readers will have a look at my FT Weekend essay on why the US-China conflict today resembles the Great Game between the British and Russian empires in the 19th century. It’s adapted from my fourth book, Sea Change: America’s New Great Game in the Arctic, which looks at the great power competition in the High North, which very much resembles what’s going on in other areas like the South China Sea and Latin America. It will be published by Crown in the US on October 6, and Swift Press in the UK on October 8.

Andy Green replies

Thanks Rana. Supercharged by little-known tax subsidies and backed by Congressional encouragement if not statutory blessing, regulators are poised to make side-bet markets far more attractive trading venues than the mainline capital markets. Even if the result isn’t devastating economically, the result will certainly be costly — both to worker-retiree investors and to the companies that want to raise money in the stock market.

Let’s zero in on the specifics. One of the most important rules for investors ensures that no matter which broker they choose, or which exchange they go to, they get the best publicly available price in the market or better. We don’t have rules like this for buying cars or lawn furniture. But we do for stocks, because Congress and the SEC decided long ago that integrating different trading centres was essential to making the markets more competitive, efficient and robust.

With stocks and other assets trading 24/7 at nanosecond speeds, the opportunities to rip off even the largest institutional investors and destabilise the markets have never been greater. Yet the SEC has proposed to unwind this integrated market and let exchanges and brokers execute trades far from the market price.

But it gets worse. The SEC is moving forward with plans to allow for “tokenised” stocks. Not to be outdone, the CFTC is moving forward with plans to allow for “perpetual futures” of stocks that would likewise not be subject to traditional stock trading rules, but would be ostensibly linked to the prices of the underlying stocks. The largest futures exchange says they’re not legal, but is instead pushing for single-stock futures. Either way, holographic images of stocks would take investors’ dollars out of the stock market and into side bets. That almost certainly means less money for the stock issuers and fewer governance rights for investors.

Why might side bets pose threats to traditional stocks? First, the trading rules are different. You can obtain much more leverage — ie, risk — with these side bets. Second, tokens, prediction markets and perpetual futures may access more favourable tax treatment, such as avoiding the financial transaction tax that funds the SEC or tapping a “blended” long-term capital gains treatment on even ultra-short-term trades.

The real cost of fragmented capital markets will come due, one way or another. Investors might be surprised to wake up and learn they’re bankrupt from overnight trading that auto-liquidates. Investors who don’t even trade these products might be subjected to margin calls because of token or perpetual futures trading by others. And instead of finding ready buyers and sellers in the stock markets, institutional investors will find their counterparties have left the traditional stock markets for taxpayer-subsidised side-bet markets.

As the CFTC increasingly blurs the lines between capital markets and commodity derivatives markets, it raises the question of why America has two separate market regulators at all. If the markets are going to get all mixed up, then it’ll be hard for the House and Senate Agriculture Committees to say the agencies should not be merged.

In the interim, Congress and the regulators — at least those focused on capitalism and not on gambling — need a better plan. Both regulators should have similar processes for letting their regulated exchanges bring new products to market. While the SEC’s process allows it to reasonably intervene and block illegal products, the CFTC’s processes don’t. That needs to change.

The SEC and CFTC should be equally committed to holding a firm line against the fragmentation of the capital market. One can be a (reasonable) crypto or prediction market enthusiast and still believe in sustaining a transparent and liquid capital market.

Rules that protect participants from abuses help to ensure the markets are there when the economy needs them. For example, there are clearly established rules governing futures brokers’ advertising to retail investors for a reason — but are those claiming to sell swaps to retail customers following them?

What’s at stake isn’t economically small. A handful of cash-rich tech companies are turning over into debt-laden hyperscalers, which may or may not be able to deliver on their AI bets. Reliable stock market plumbing, together with good disclosure and strong auditing, is key to being able to sustain genuine market growth — rather than perpetuate a bubble or worse. Are we ready to stop the fragmentation of our stock market? The decision is up to Congress — and us.

Your feedback

We’d love to hear from you. You can email the team on swampnotes@ft.com, contact Ed on edward.luce@ft.com and Rana on rana.foroohar@ft.com, and follow them on X at @RanaForoohar and @EdwardGLuce. We may feature an excerpt of your response in the next newsletter

Recommended newsletters for you

The AI Shift — John Burn-Murdoch and Sarah O’Connor dive into how AI is transforming the world of work. Sign up

Unhedged — Robert Armstrong dissects the most important market trends and discusses how Wall Street’s best minds respond to them. Sign up