UK statistics agency turns to AI to cut costs and improve data

The UK statistics agency is turning to AI to cut costs and improve data as it tries to fix critical economic outputs and deliver the next census with squeezed finances.

The Office for National Statistics said it expected to save thousands of hours a year with some of its new AI tools, making important efficiencies as it faces further delays replacing faulty jobs data and seeks to cut its outputs by 10 per cent in order to focus resources on core economic data.

James Benford, ONS director-general for economic statistics, told the FT: “There are big quality issues that are still outstanding and we’ve got to implement new standards for GDP and deliver the next census, transformed labour force survey, and statistical business register.”

“Using AI in the production of statistics is helping us do more, but it’s also improving quality,” he added.

The ONS is in the middle of a turnaround effort after deep-seated problems led to a series of issues with its jobs data and other critical economic outputs, forcing the agency to scale back work in areas such as health and crime.

This month it warned that work to replace its faulty labour market survey was “increasingly constraining” its ability to invest in wider statistics improvements.

While the agency has received an extra £100mn a year to support preparation for the next census in 2031, its core funding has been cut by almost a tenth in real terms over the next two years.

“AI is mainly a productivity play. We’re trying to do more things with our workforce . . . and find big efficiency gains that can be reinvested,” said Benford, who was appointed in June last year to help steer the turnaround.

Prime Minister Andy Burnham has doubled down on plans set out by Sir Keir Starmer’s government to increase public-sector adoption of the fast-evolving technology.

Since entering Downing Street last month, he has set up an AI task force chaired by Lord Patrick Vallance, former chief scientific adviser, to “transform public services” and promoted the role of AI minister to cabinet level.

The ONS last year became one of the first national statistics agencies in the world to use AI in the production of official figures, introducing a tool built on Google’s enterprise large language model that classifies jobs and industries from survey responses.

Andrew Banks, ONS lead data scientist, said the tool was estimated to have saved 350 hours a year across two surveys and increased the accuracy of industry classification from 71 per cent to 80 per cent.

Early next year, the ONS plans to introduce an AI scanner to automate the processing of receipts, which are collected from households that take part in the Living Costs and Food Survey.

Banks said, on a conservative estimate, the tool was projected to save 7,500 hours a year by cutting the time needed to review a household’s receipts from three hours to just a few seconds.

“We collect about 90,000 receipts and currently use quite a manual process to extract the data,” he added, noting that removing manual data entry would allow researchers to significantly increase the number of households in the survey.

AI is also expected to improve the new transformed labour force survey (TLFS) by dynamically adding a follow-up question about respondents’ jobs. An initial trial with 1,000 people found that 43 per cent of those receiving the extra question were classified more accurately, with no drop-off in completion rates.

“This has been one of the areas for the TLFS that’s been tricky because it’s digital first,” said Benford. “Normally there is a conversation with an interviewer to figure out the industry code, and we’re trying to mimic that using AI.”

Benford said AI was also being rolled out across the workforce, with more than 5,000 of the ONS’s 5,980 employees using standard tools and 120 data scientists using coding assistants each day.

“The important thing is we’re trying to do more things with our workforce rather than trying to shrink the workforce,” he added. “There’s so much for them to do, we’re just going faster.”