Third Party Evaluators

TBPN，科技每日秀 Substack 专栏。关注

Happy Wednesday.

We’re live on YouTube and 𝕏.

The current thing in tech and business is Zuck’s response to the AI slowdown debate.

Today’s Lineup

Circle Co-Founder & CEO Jeremy Allaire at 12:00 PM
Theory Ventures General Partner Tomasz Tunguz at 12:10 PM
Rune Co-Founder & CEO William Layden at 12:25 PM
ReynKo Founder & CEO Justin Beroz at 12:40 PM
Footprint Co-Founder & CEO Eli Wachs at 12:50 PM
BackOps Co-Founder & CEO Sean McCarthy at 1:00 PM
Impulse Space Founder, CEO, & CTO Tom Mueller at 1:10 PM

Run of Show

Third Party Evaluators

As the NY Post crashes out over the prospect of METR becoming the “handpicked AI watchdog” and Martin Casado at a16z pushes for the Department of Energy, a small debate has emerged over who’s actually equipped to do this job. In Dario’s We Must Pace the Frontier essay, he only goes as far to say that Anthropic is committing to bringing in METR, he doesn’t really push for METR to be the only option.

These are the two extremes, METR has been deeply embedded in the AI safety discourse since inception, with some team members articulating concrete visions relevant to this exact moment years ago. The Department of Energy oversees nuclear weapons and… I can’t really name any high profile DoE employees who post thoughtful AI safety takes on X. But that’s the benefit of course, the government bureaucracy is designed to be an arm of the government and have as few links to private entities they oversee as possible. Revolving doors do pop up regularly though, and there are thresholds for what’s acceptable.

It’s important to remember that, when the Atomic Energy Commission was created to oversee nuclear development, many of the scientists who did forecast the importance and riskiness of the technology were involved, but typically on the advisory side, not as line regulators. Oppenheimer became chairman of the AEC’s General Advisory Committee, which had enormous influence but did not issue licenses. Enrico Fermi and Glenn Seaborg became committee members, with Seaborg later becoming chairman of the AEC itself. Edward Teller is the oddest example. He was originally one of the strongest advocates for the hydrogen bomb (i.e. accelerationism), but he ultimately became the first chairman of the AEC’s reactor-safety committee.

Today, nuclear regulators are talented and hardworking, but they are not tasked with forecasting risks decades into the future. The jobs are diverse: nuclear engineers, mechanical engineers, materials scientists, physicists, health physicists, geologists, probabilistic-risk analysts, cybersecurity specialists, emergency-preparedness experts, lawyers, and inspectors. Pay ranges from $125,000 to $187,000 for many NRC technical staff roles. Enlisted navy sailors manage nuclear reactors aboard submarines at age 22. They then go through roughly a year and a half of specialized nuclear training: Nuclear Field “A” School, about six months of Nuclear Power School, and about six months of hands-on prototype training before arriving in the fleet. Nuclear Power School covers math, nuclear physics, reactor principles, health physics, materials, thermodynamics, electrical systems, and reactor technology.

It’s easy to narrow the aperture for AI safety work to only include a handful of AI researchers who have “real experience training models at frontier labs” but I think that will be hard to digest for many people because of the conflicts of interest. Those folks will have a very important place in the future of AI oversight, but don’t count out the millions of talented Americans who can be quickly trained to help solve hard problems as they arise.

Clip Spotlight: Stephen Colbert’s advice for young creatives: "Go get in trouble."

"Go create something that someone could see. Write something. Go have a reading in a coffee shop. Make a film with your phone. Edit it together. Show it to someone you care about."

"Light a candle under your own ass by saying that 'If no one else will ask this of me, I will ask it of me.'"

"There's no waiting, wishing, wanting, hoping, thinking. That is not how to get better at what we do. You have to go do it. You have to get in trouble."