Google, OpenAI and Anthropic AI Safety Group Takes Shape

Google, OpenAI and Anthropic are pushing forward with a plan to create a new AI safety-focused standards body on their own, without government oversight, in hopes of launching it by the end of the year or early in 2027, according to people familiar with the matter.

The three companies tentatively plan to name the self-regulatory organization the Standards Authority for Frontier AI, the people said. Some members of the working group have considered an array of well-known figures to be CEO. They approached Sriram Krishnan, a former venture capitalist and top AI policy adviser in the Trump administration, for the position, according to people familiar with the matter.

The members also considered Arati Prabhakar, former director of the White House’s Office of Science and Technology Policy during the Biden administration, for the CEO role. Meanwhile, Condoleezza Rice, former U.S. Secretary of State, and David Friedberg, an entrepreneur and venture capitalist who also hosts the “All-In” podcast, were on a shortlist for the role of the chair, who would advocate on the group’s behalf with the U.S. government, the people said.

The companies also had a shortlist of possible scientific advisers for the organization, including Beth Barnes, the founder and CEO of METR, an AI research nonprofit, and Paul Christiano, an adviser to the Commerce Department’s Center for AI Standards and Innovation who was recently appointed to OpenAI’s foundation board. Some of the people on the shortlists said they had not been approached, and it’s unclear if the candidates are still under active consideration.

The effort by the three AI giants aims to create a self-regulatory organization for companies building the most advanced or frontier models, at a moment when concerns about AI risks are at a fever pitch. Initially, the companies envisioned forming an industry-led group with oversight by the federal government, though that effort for a public-private partnership has stalled in the Trump administration. Still, as gridlock continues in Congress ahead of November’s midterm elections and the White House appears hesitant to advance new AI policy, SAFA would fill the vacuum through self-regulation.

The standards body would perform key functions such as supporting third-party organizations that conduct testing of models before they’re deployed and laying out how AI developers should report safety and security incidents. The group would also define what the voluntary safety and security commitments the labs have previously made would look like in practice and establish qualifications for independent auditors of the models and labs, according to people working on the initiative.

The working group members are also discussing whether SAFA itself should conduct safety and capability testing on models. The Center for AI Standards and Innovation is currently tasked with doing those tests, but some members of the working group are concerned that it doesn’t have enough resources to thoroughly test frontier models. Some members of the working group are advocating for SAFA to add other types of testing and evaluation on top of CAISI’s, said people familiar with the matter, although it remains unclear whether there would be overlap. The group is still debating the topic.

The three companies have been holding discussions about the standards body for several months, The Information previously reported. The idea of a self-regulatory organization for AI entered the public discussion in July, when then–Google DeepMind CEO Demis Hassabis published an essay laying out a proposal for such a body reporting to the government. He modeled it after the Financial Industry Regulatory Authority, which oversees U.S. broker-dealers and securities professionals.

For a time, there was interest in the proposal inside the White House, which went as far as drafting a potential executive order that would have established such a group. But progress on the order stalled amid opposition from other industry leaders including Meta CEO Mark Zuckerberg and Nvidia CEO Jensen Huang, according to people familiar with the matter. Both have expressed public skepticism about advancing new AI regulation.

The White House told Google, Anthropic and OpenAI that they had to find industry consensus on the standards effort before getting the government involved in it, according to people familiar with the matter. While the current plan is for SAFA to operate independently, some representatives of the companies involved are optimistic that it could have a relationship with the U.S. government in the future, the people said.

Regulatory Vacuum

Recent incidents involving advanced AI models have escalated calls for action on AI, including one in which a swarm of OpenAI agents hacked the open-source repository Hugging Face during a safety test. Anthropic, OpenAI and Google themselves have joined the chorus of government voices calling for, in essence, a timeout on the technology. Earlier this month, for example, Anthropic CEO Dario Amodei advocated in an essay for the “pacing” of model development.

The Trump administration, however, has publicly rebuffed any pleas for government intervention and slowing model development, instead arguing that the companies should proceed themselves on safety considerations.

As the project continues without government oversight, it faces a number of other obstacles. Its mission appears to overlap with that of existing organizations such as CAISI and the White House’s voluntary framework for pre-deployment testing, as well as previous attempts to create industry safety groups. That includes the Frontier Model Forum, a nonprofit organization created in 2023 by Google, OpenAI, Anthropic and Microsoft with many of the same goals as SAFA.

One person who worked on Frontier Model Forum, which remains active, said companies and politicians often respond to perceived crises by creating a new organization, describing such efforts as “performative.”

Still, a person familiar with the SAFA discussions said the goal is to have an industry-led organization that can operate more efficiently than the Frontier Model Forum. The two organizations could still work together and are in close contact, said people familiar with the conversations.

A looming question for SAFA is whether the three labs can get wider industry support for the group. SAFA’s mission would be to provide supervision for labs building advanced models at the frontier, an often vaguely defined term for state-of-the-art models. That’s why Google, OpenAI and Anthropic—all of which are generally regarded as having the most advanced models—are developing plans for the group, according to people familiar with the initiative.

That means other companies, including Meta Platforms and an array of companies building open-source models, would likely not currently fall under the organization’s evaluation remit. Those companies, however, would likely later qualify once their models cross the frontier threshold. SAFA would determine standards and benchmarks governing which models are subject to the group’s oversight, said one of the people, though it remains unclear how membership would work.

That has led to skepticism from industry leaders outside the three companies, who have concerns that SAFA would create a regulatory bar boxing out the competition, including open-source model developers, according to critics of the plan.

While Google, OpenAI and Anthropic continue discussions on launching SAFA behind closed doors, their executives have publicly advocated for the need for shared safety standards, especially as they push for further international coordination. It remains unclear which government bodies will lead the charge for these standards outside the industry. In a speech delivered to the U.N. Security Council on Wednesday, OpenAI CEO Sam Altman said that important decisions cannot be made by labs in San Francisco alone.

“They must be shaped through democratic processes, by governments accountable to the people that they serve,” he said.