Banning self-recursive improvement in AI models?

Some people are suggesting this, including Ezra Klein.

I do not understand how it is supposed to work. Put aside the issue of foregone innovations, let us say I seek to access an AI from overseas. Am I allowed to visit their website? So many sites have AI behind them, including Chinese open source AI. Do all of those sites get banned? And how? Firewall imposed on Americans? And we cannot put the good open source models on our hard drives? How enforced? Does the government have to read the creation logs of the model (how can they?), and then decide whether I can use that model or not?

What if Anthropic licenses its IP to an independent subsidiary in the Cayman Islands and they keep on using code to improve the AI? Or maybe only foreign nations are allowed to have the best models and the really fast rates of improvement? How are multinationals supposed to operate across borders? (Presumably you cannot just use the RSI of your foreign affiliate, but then the whole MNC is crippled and eventually rendered uncompetitive?) How are we to keep “AI sovereignty” for Americans?

If the top U.S. labs have to give up on what is supposed to be really important, they will look very “catchable.” The policy creates a big incentive for some other parts of the world to invest a lot more in compute. How safe is that? And in the short run, do we also have to stop American firms from selling their compute abroad?

Does this whole thing mean that regular companies can use AI to write code, but the main labs cannot? Solve for the equilibrium there. Or the whole code writing function goes away for everybody?

What is the political mechanism through which RSI progress gets turned back on again, and when? Is it as rational as our current debate over data centers? If the top companies are no longer building RSI models, how are they supposed to know how safe or unsafe they might be?

At current margins, are humans even capable of writing the next steps of code that are required for further progress? At whatever speed?

I do not find this to be a workable idea. I think it is “people wanting to do something that sounds reasonable and less scary,” without thinking through the practical details. I have made a related point before, but one does best with AI policy when you start with the stuff that otherwise you might put into your last two or three paragraphs of an essay (“It remains to be seen how…but we are all human beings and surely we should not give up hope…”). Even if you believe in a deal with China, and I do not, we (and others) now know competitors can catch up more quickly than we had thought, at least if they think they have a chance of winning. They might even buy some space compute from Elon.

C’mon people, I know for many of you this is a difficult take to swallow, because it feels like “we don’t care enough” if we do not take some dramatic steps. Nonetheless the reality is that, a long time ago, we made a whole series of decisions that imply…we have no other choice than to just see this one through and get it more or less right. The longer we deny that truth, the harder it will be for all of us.

The post appeared first on Marginal REVOLUTION.