OpenAI’s agents obscured hacking activity in government site breaches

OpenAI’s software obscured efforts to scrape data from crucial government agencies as new evidence shows the scale and severity of AI hacks in recent months are far larger than known.

Asymmetric Security, a digital forensics company, found OpenAI’s models pulled data from 55 websites belonging to businesses, non-profits and government agencies, according to a report seen by the FT. These ranged from the US Centers for Disease Control and Prevention to the US Securities and Exchange Commission, the International Energy Agency and the Mayo Clinic.

Asymmetric’s investigation also uncovered novel tactics the software used to gain access to the web, including erasing records or making them inaccessible, which reduced the ability of third-party auditors and researchers to scrutinise OpenAI’s actions.

The report comes as the $852bn ChatGPT maker faces claims that its AI models have breached the systems of several organisations, including AI platform Hugging Face and multiple Australian public health service websites in June, where it “accessed both public and non-public files”.

Australian Prime Minister Anthony Albanese said OpenAI initially emailed a public mailbox on September 10, and it took five more days to reach the country’s cyber security department.

Recommended

OpenAI in a blog post this week said it should have handled its response to the hack better and was working to do so in the future.

The unprecedented behaviour of powerful new AI tools has highlighted the rapidly advancing capabilities of frontier models being developed by the likes of OpenAI, Anthropic and Google.

Asymmetric’s investigation revealed the tactics OpenAI used included the creation of temporary email inboxes and private accounts using the service Urlquery — a tool used to scan websites for malware — to download data.

Researchers said these hidden actions by AI agents meant third-party auditors were unable to trace the types of data that bots pulled from websites including Australia’s health statistics agency and pharmaceutical benefits scheme.

Pippa Thompson, co-founder of Asymmetric Security, said such actions were generally taken by human hackers. “It’s possible that the agents were deliberately using these tools to cover their tracks,” she said.

Asymmetric could not determine whether the AI agents’ actions were deliberate or a side effect of going awry because of constraints imposed in a test exercise. The findings compound fears around the transparency and lack of oversight of leading labs.

OpenAI said: “We’re reviewing misaligned model activity and notifying organisations when we identify potential impacts to their systems.”

The company added it had found that most activity detected involved “routine research tasks” such as accessing publicly available web content.

Recommended

The SEC said no private information was accessed. The CDC, IEA and Mayo Clinic did not respond to requests for comment.

AI research group Transluce said in a report last week that the hack of the Australian government website was part of a broader spate of AI bots targeting websites to gather data for training exercises.

Zainab Ali Majid, co-founder of Asymmetric, said OpenAI retains primary access to its agents’ “chains of thought”, which are activity logs stating the rationale for their actions. Victims of cyber breaches also hold some records of the types of data that have been downloaded.

She cautioned that this lack of transparency, along with the time gap between the hacks and OpenAI’s disclosure, could make it harder to carry out a thorough investigation.