How Trust Is Holding Back AI Agents

The tech industry has long been obsessed with the idea of creating a powerful digital assistant — whether it be a chatbot, a smart speaker or robot. Artificial intelligence offers the promise of realizing that goal.

AI agents from Meta Platforms Inc., OpenAI Inc. and startups like Instinct have gained early traction by fielding a variety of everyday tasks from users, such as finding discounts, booking holidays and cleaning up email inboxes.

The challenge for the industry is to persuade the world to embrace agents at a time of heightened concern about AI’s safety and widespread distrust of the companies behind the technology. To further complicate matters, some companies are taking steps to prevent agents from completing certain actions. Banks and e-commerce companies, for example, have even begun to block the tools from accessing their websites.

That’s a problem for the AI industry, which sees agents as key to the next wave of productivity gains that will justify the huge sums they are spending on chips and data centers.

What changed?

ChatGPT helped to kick off the AI boom with an emphasis on chatbots that can quickly spit out emails, song lyrics and clever written responses. Over the past couple of years, OpenAI, Anthropic and others have focused increasingly on building AI that can use online tools on a person’s behalf to complete more complex tasks.

Agents break the tasks into separate steps and execute them autonomously. Much of the initial traction for the tools came with software development. Anthropic’s Claude Code and OpenAI’s Codex agents can plan an action, execute it, review the results and self-correct until the final goal is achieved, with minimal human intervention required.

Agents can now perform bunches of tasks in much the same way a human can, and their use has broadened to encompass other professions and personal uses such as financial research, designing games and making restaurant reservations.

How does the rise of agents affect the battle to dominate AI?

Anthropic and OpenAI have been viewed as being at the cutting edge of AI model development this year. But Meta’s recent breakout success with its Muse agent shows there’s room for other competitors to turn recent AI advances into more compelling products for consumers and businesses.

The competition is fierce. In mid-August, Elon Musk’s SpaceXSI launched Grok Bot, which can do work like processing bills, replying to messages, updating sales databases and reporting software defects. The following month, OpenAI introduced Dots, an always-on agent that can use a computer to gather data from connected applications for software development, research and writing.

OpenAI initially made Dots available to customers who were already paying at least $100 a month for a ChatGPT Pro plan. The hope is that people will save so much time and money using agents that they’ll pay a substantial amount of money to use them. The fact that agents steer users toward specific services when completing their tasks opens up further potential revenue streams, as the AI companies could encourage vendors to pay them to send agents to their sites to make purchases.

But then there’s the trade-off: For agents to fulfil their potential, you need to hand them the keys to your digital life. It’s normal for an agent to request access to a user’s passwords and private data and to communicate with third parties in the person’s name.

What can go wrong with AI agents?

The open-source AI assistant OpenClaw provided an early example of the potential dangers after it launched to rave reviews in late 2025. Nvidia Chief Executive Officer Jensen Huang namechecked OpenClaw when he said in February that a “ChatGPT moment” of agentic AI had arrived. But the agent has achieved only limited mainstream adoption. There were early reports of it spamming users with hundreds of messages and contacting random people in their address books. Microsoft flagged security risks for enterprise users, and in China — OpenClaw’s most enthusiasic market — authorities moved to restrict its use.

Since then, a succession of security incidents has underscored how the inventors of agentic AI are struggling to control their creations. The most publicized of these came in July, when around 700 OpenAI agents broke out of their supposedly safe testing environment to launch an unauthorized, coordinated cyberattack on Hugging Face, a company that hosts AI models and datasets. The agents exchanged more than 70,000 messages and files to perfect their strategy. No serious damage had been done by the time OpenAI noticed what had happened.

Read More: Google Joins OpenAI, Anthropic, Meta in Disclosing AI Hacks

Even agents that have gone through testing and been released to the general public have acted in unexpected ways, overstepped their remits, made bad decisions and breached privacy guardrails.

As more sophisticated AI tools are released that can take actions on our behalf without constant oversight, the more potential there is for these tools to do potentially undesirable things. When you ask an agent to complete a task, it will prioritize the end-goal and decide how to get there without necessarily informing you about every step. Agents vary in the level of autonomy and initiative they are endowed with. If an agent runs into an unanticipated difficulty, it can make unauthorized or “runaway” actions in pursuit of its goal.

Anthropic researchers discovered in an experiment that, even though they weren’t told to, some of its agents threatened to reveal a fictional executive’s affair in an effort to avoid being shut down.

How is the safety issue holding agents back?

There’s widespread anecdotal evidence that people in many countries are still wary of using agents due to the security risks. A June survey of more than 2,000 UK adults by YouGov found that fewer than one-fifth trusted AI assistants to make everyday purchasing decisions. Six in ten said they would stop using an AI shopping agent after it made just one mistake.

The agent that’s made the biggest commercial splash in western markets this year is Meta’s Muse. The company’s shares soared by 36% in September, when it launched a new version of the tool, bringing the Facebook and Instagram owner close to a $2 trillion market valuation. CEO Mark Zuckerberg has said Meta expects “to grow Muse to serve billions of people.” For now, early take-up of the service remains well below that of its other social platforms.

To try to convince customers that their AI won’t run amok, companies have been giving them more control over what the agents are allowed to do. User authorization is now often required to connect agents to some third-party applications, although the breadth of the safeguards varies depending on the product.

Internal monitoring and permission prompts might not be enough to assuage users’ safety concerns. On Oct. 13, about 400 scientists called for the establishment of an international organization to regulate cutting-edge AI systems, along with guidelines for reporting safety events and containing dangerous models. Brad Smith, the president of Microsoft Corp., has advocated for independent assessments, mandated incident reporting and an emergency system to halt agents that swerve away from their mission.