UK ministers explore compulsory AI safety reporting

The UK government is drawing up plans for compulsory safety reporting and monitoring of frontier AI, as the minister responsible for the technology prepares to tell companies to exercise stricter control over autonomous models.

In a speech on Monday Kanishka Narayan will say companies building increasingly powerful and self-directing AI models must do more to deal with risks and build safeguards, according to people familiar with the matter.

He will warn that the government is ready to legislate to increase oversight of the fast-evolving technology, which ministers hope can improve economic growth prospects even as it poses risks to security and jobs.

Mandatory reporting of incidents when AI agents behave maliciously is among regulations being drawn up after recent high-profile cases in which companies acknowledged that their technology had gone rogue, the people said.

This summer an OpenAI agent hacked an Australian health service website, although the incident did not become public until last month. Anthony Albanese, the country’s prime minister, said the US company took “way too long” to inform his government.

It followed a previous incident where OpenAI agents surreptitiously broke out of a testing environment and hacked the AI start-up Hugging Face.

“We cannot leave the risks for somebody else to manage, or rely entirely on developers’ assurances. Nor can we afford to wait for a major incident before building the means to respond,” Narayan is expected to say.

“Where stronger rules, new laws or additional powers are needed to protect the British public, nothing should be off the table.”

Legislation that would force companies to give the UK AI Security Institute (Aisi), the global leader in the testing of frontier models, pre-release access to their latest models is also being considered, after the FT revealed Anthropic declined to do so following pressure from the Trump administration.

UK ministers are concerned that the move might provoke the US president into refusing tech groups permission to share their models, after he dismissed AI safety concerns as a “hoax”.

Ministers are, however, confident that they have persuaded the US not to block sharing of future models with Aisi.

Trump is determined to see US companies press ahead in the race against China to develop the technology, saying previously that “whoever wins AI, wins”.

Narayan is attempting to set out a vision of UK AI sovereignty that would see crucial technologies and data storage housed in Britain, while seeking a handful of world-leading capabilities, including safety, that would offer the UK government leverage in negotiations with the US and others.

Recommended

Speaking last month on the sidelines of Labour’s conference in Liverpool, Narayan said his focus was developing a framework to test models, understand the associated risks, harden the UK’s defences and build AI capability in Britain.

“If regulation or legislation is the mechanism to achieve those ends, I would be the first one in the queue to push for that,” the AI minister said.

In his speech, Narayan will say that about 1,000 cyber security flaws have been identified by a government attempt to use commercial AI models to spot flaws in its own defences.

AI must be used to defend against such risks as part of an upgrade of British state resilience, he will add.

Prime Minister Andy Burnham has said he wants to use the British presidency of the G20 next year to work on “global principles and standards” to ensure safe AI development.

He told the UN General Assembly last month that AI would be “a source of disruption to the way we live and work” and that decisions on its development “must be taken by elected governments”.