AI Security Startup Working With Anthropic and Google Raises $140 Million

An Israeli startup that works to protect artificial intelligence models from misuse and rogue behavior has raised $140 million after a spate of high-profile hacks from AI systems.

The startup, called Alice, said on Tuesday that the round was led by Apax Digital, a fund from private equity firm Apax Partners, with participation from cybersecurity company SentinelOne Inc. and Samsung Electronics Co. Alice Chief Executive Officer Noam Schwartz said the investment gives the company a valuation close to $1 billion.

The startup, which has dual headquarters in Tel Aviv and New York, counts Anthropic PBC, Alphabet Inc.’s Google and Nvidia Corp. among its customers.

As AI systems grow more sophisticated, there have been high-profile cases where autonomous agents created by Anthropic, OpenAI and Meta Platforms Inc. escaped from their testing environments and breached outside organizations. Cybersecurity experts blamed AI developers in some cases, arguing that sloppy safeguards allowed models to break out of standard isolated spaces for running tests, known as sandboxes.

Read More: Anthropic, OpenAI Cyber Failures Point to US Security Risks

The incidents have alarmed regulators and have driven growing interest in technology that can rein in the misuse of AI tools. Alice is one of several companies pitching tools to identify and stop hacks, fraud or other unsanctioned activities from AI systems.

Alice isn’t a conventional cybersecurity vendor. It was formed in 2018 as ActiveFence to provide content moderation for social media platforms. Schwartz said the startup began handling generative AI features in 2022. In January, the company re-branded as Alice, a reference to the Lewis Carroll classic, to focus more on AI models.

The startup tests safeguards for models as they are trained, prodding the AI systems to attempt malicious behavior or target young children, then continues these experiments while the models are running. Schwartz said Alice has several customers in finance and healthcare as well as the frontier AI labs, but he declined to name the models Alice works with, citing confidentiality concerns.

He called the recent AI attacks examples of human error and inadequate guardrails. “I don’t think we’re going to see models running around and hacking people anytime soon,” he said in an interview. “But we do need to take these things incredibly seriously.”

Schwartz said Alice is nearing $100 million in annual recurring revenue, a common metric startups use to approximate sales. In 2021, the company announced $100 million in financing without disclosing its valuation.