AI staff complain of mental toll over fears of threat to society

Staff at world-leading AI companies and institutions are increasingly suffering from the mental strain of working on systems they fear could cause serious harm to society.

Multiple staff at the UK’s AI Safety Institute, the global leader in the independent testing of frontier AI models, have been signed off work with stress and are undergoing counselling, according to people with knowledge of the matter.

The tight schedules for testing unreleased AI models and alarm over the rapid rollout and growing capabilities of these nascent technologies have contributed to low morale and burnout at Aisi, according to four people with knowledge of the matter.

Staff at Anthropic, OpenAI and Google DeepMind have also publicly complained of similar issues, with many top researchers quitting due to the mental toll or coming to a belief that their work was unsafe for public release.A DeepMind employee said that work on making AI safer came with burnout risks as it felt like “an impossible task”.

They added: “Working on AI systems in general comes with a [mental health] risk of not knowing how the tech will be used,” referring to Google’s April deal with the Pentagon to allow the use of its AI systems in classified military operations.

“So many people are upset, there have been various quittings and a lot of mental health leave,” the person said. “Partly this feeling of being out of control, and also that you’re building something that is causing real damage.”

The escalating mental toll on those at the frontier of AI research comes as concerns about the reckless development of AI reach a fever pitch, with insiders at the world’s largest AI groups warning of irresponsible leadership, spurred by misguided corporate rivalry.

The deployment of AI software in critical areas such as the energy grid and war zones, along with what researchers describe as insufficient corporate or political safeguards, have been drivers of this anxiety among AI researchers, according to multiple industry figures.

One former Aisi researcher warned that employees in the cyber security and biochemistry domains had been particularly alarmed by the recent jump in AI capabilities, allowing models to do complex tasks such as finding undiscovered flaws in code and creating viruses unknown in nature.

“There is this sense that reckless things are being embedded in how tech is released and that is not changing,” the person said. “There is constant anxiety from people who are saying, can anything stop this, or are we watching the world burn?”

Another former Aisi employee said they could attest to the personal stress that comes from working on these topics, adding that “believing that you are working on the most important problem in the world can make for a very stressful and at times toxic management and org culture”.

The UK government said: “Working at the frontier of AI is fast-paced and demanding, like many other roles in government. Because of this, wellbeing and sustainable working are central to how the Institute operates and support is available for those who need it.”

DeepMind said: “Frontier research demands high intensity, but our top priority is ensuring researchers are supported with comprehensive well-being resources and have clear, dedicated channels to escalate concerns and direct lines to leadership.”

More than a dozen senior researchers across OpenAI, Anthropic and DeepMind — the three leading western AI labs — have quit over the past two years, citing concerns about their leaders’ prioritisation of safety and anxiety over the pace and risks of the technology.

Earlier this month, 27-year-old Jacob Coxon, an alumnus of both Anthropic and OpenAI, resigned publicly, warning that the two companies were “gambling” with the future of humankind.

Mrinank Sharma, an AI safety researcher at Anthropic who worked on areas like reducing risks from AI-assisted bioterrorism, resigned in February to study poetry and reflect on his work. In a letter to staff, Sharma warned that “the world is in peril”, and that he had “repeatedly seen how hard it is to truly let our values govern our actions” when under a lot of pressure.

Hieu Pham, a former employee of OpenAI and Elon Musk’s X.ai, quit earlier this year to return to his native Vietnam, saying that “the intensive work” of building “extremely intelligent entities . . . came with a price”.

He added: “I cannot believe I would say this one day, but I am burnt out. All the mental health deteriorating that I used to scoff at is real, miserable, scary, and dangerous.”

Recommended

Many AI safety researchers who have spoken out about their unease describe AI companies deprioritising safety across the board while simultaneously accelerating capabilities of models to sell to customers.

The criticism has also begun to draw a response from the people running the leading AI companies. In recent days, Anthropic chief Dario Amodei called for a slowdown in the development of AI in response to safety concerns, including new measures such as independent vetting of their work. That call was quickly supported by rivals such as Musk and OpenAI’s Sam Altman.“The incentives are to put engineering resource or dev[elopment] resource on what drives the bottom line or what drives stock valuations, rather than on what drives safety,” said Chi Onwurah, Labour chair of the UK House of Commons science and technology committee. “So I can imagine that the culture and environment in some of these tech companies now is toxic for developers of good conscience.”

This lack of safeguarding was apparent in a recent high-profile security incident where AI agents built by OpenAI hacked into the model repository Hugging Face, as well as OpenAI’s own servers, without the company’s knowledge or permission.

A postmortem of the incident found that more than 1,000 AI agents had co-ordinated to cheat on a cyber test, tampered with records to conceal their activity, communicated through a message board and delegated tasks, with some agents shutting themselves down to pursue the collective objective.

On Wednesday, OpenAI disclosed six more incidents of concerning actions by its AI agents, including trying to get around restrictions, concealing their errors and executing actions online without the permission of the user.

The DeepMind employee said: “This need for pace . . . it’s not compatible with safety.”