NYT Editorial Board Comes Out Against Extinction

(Archive link)

The NYT editorial board's article on AI is far better than I'd expected, but at the same time not all I'd hoped for.

The title sets off very well: "Humanity Has Avoided Apocalypse Before. Let’s Do It Again." It is truly excellent to see the extinction threat from loss of control be mainlined.

A quick gloss of their policy requests: an AI Commission in government, licensing requirements for AI companies, an AI "constitution" written by the US Government incorporated into AIs, mandatory watermarks/identifiers on all AI content, mandatory independent testing for AI models before release, and a government agency to investigate accidents. Internationally, they call for tightening export controls, limiting China's access to semiconductors, and ultimately negotiating an international slowdown with China and an international framework for AI oversight.

These are all steps in the right direction—of taking AI seriously. That said, it isn't clear if the licensing is required for training or for selling AIs. The idea that constitutional AI "would ensure alignment with human values" is of course not remotely true. And mandatory testing should apply to all models trained, not all models released, of course, and this is a glaring oversight. But overall these are far more real attempts to grapple with the issues than I had any right to expect.

(They also make a clear implication that it would be irresponsible for Anthropic to go public. I don't particularly see strong arguments for this, could someone explain?)

I was initially saddened to see substantial emphasis on staying ahead of China, but then heartened again to see them say that negotiating an international slowdown would eventually be necessary.

The United States should also seek to negotiate a coordinated A.I. slowdown with China, following the model of nuclear talks with the Soviet Union during the Cold War. In the long term, an international framework for A.I. oversight will be the only way to protect people.

Even given that they're not yet at the point of detailing the proposed international treaty with China, I think they could still have also proposed:

• Mandatory chip location tracking, so governments an tell where large concentrations of compute are.
• Chip activity monitoring, to help verify compliance for a US-China treaty.
• Liability laws for damages from AI, to remotely align the incentives of companies.
• Requiring authorization for training runs, not just model releases
• Mandatory independent testing of all trained models, not all released models
• Mandatory reporting of serious incidents and near misses, together with whistleblower protections
• And possibly government funding for alignment research, though to be fair I seriously doubt whether the government would remotely have the capacity for this to not be eaten by capabilities research

One uncertainty I have is whether they could have done much better on proposals for alignment. I imagine Constitutional AI is the one they picked, not because it's particularly effective, but because it's the only one they could think of and knew how to explain here. But perhaps I am missing some simple alternative.