‘Know Your Agent’ will be the new rule of the AI age

Move aside KYC, there’s a new acronym in town: KYA.

For decades, regulators have forced banks to follow Know Your Customer rules to try to prevent money laundering. But what happens when flesh-and-blood customers turn into inanimate machines? Every individual and business will have to Know Your Agent.

As Ben Macintyre describes in his latest book Redwood about international espionage in post-revolutionary Tehran, managing human agents can be a tricky business. When the MI6 officer Ian McCredie ran the KGB spy Vladimir Kuzichkin, it was hard to know how trustworthy the Russian was, how reliable his information and whether he was acting as a triple agent. But spies reckon human motivations mostly revolve around money, ideology, coercion and ego.

AI agents, on the other hand, have no inherent motivation or intent, can be programmed to do good and bad, can act in inexplicable ways and can have many more layers of complexity and opacity. A whole new agentic ecosystem is rapidly evolving to verify, align and double-check these automated assistants. As with the evolution of the World Wide Web, protocols, standards and norms underlying this ecosystem will shape the future use of the technology.

AI agents, autonomous software systems designed to accomplish multi-step goals with minimum human intervention, are already being deployed at scale by many companies. A report this year from Deloitte found that 23 per cent of companies are currently using them. But 74 per cent intend to do so within the next two years. Meta’s recently launched Muse, a personal AI agent that can order groceries and book travel, is proving highly popular.

This increasing use of AI agents presents three main challenges. The first, and possibly simplest to address, is cryptographically identifying a particular agent when it interacts with others. Several decades of experience in the $37tn online payments market shows how this can be done at massive scale and speed, even if security remains a permanent vulnerability.

The second challenge is making sure agents behave as you want them to, which may be far harder to achieve. Imprecise instructions can lead to unintended consequences, as the mythical King Midas discovered when he turned his daughter into gold. The necessity of setting appropriate guardrails was highlighted by an incident in Australia earlier this year. An IT worker’s AI assistant booked a popular gym slot for him by kicking other members off the waiting list, which is not what he intended. The complications are massively greater when multiple agents interact with each other, as OpenAI discovered when a swarm of 700 of its agents hacked the AI platform HuggingFace.

This highlights the third big challenge: establishing accountability and liability when things go wrong. What happens if an AI agent performs an action that breaks the law or results in physical harm? Or divulges sensitive personal or commercial information? Expect myriad disputes on how to apportion responsibility, keeping lawyers and insurers lucratively employed for years to come.

The overarching issue determining how successfully AI agents are used therefore comes down to governance. But governance depends on many things, including the quality of the underlying data that feeds into these systems. And not all of an individual’s or a company’s principles and practices are contained in that hard data anyway.

Craig Gravina, chief technology officer of the data company Semarchy, tells me that AI agents risk repeating the problems caused by software “anti-patterns” that were identified by programmer Andrew Koenig some 30 years ago. Systems initially appear to provide effective solutions but fail over the long term, leading to high exit costs.

AI agents can be optimised to address one specific problem but can inadvertently cause others (such as scrubbing people from a gym waiting list). Their effectiveness depends on understanding the broader context in which they run. But many of these operating norms lie in the “magic numbers” people carry around in their heads. “Context is so much more than just vocabulary or ontology,” says Gravina.

In that sense, deploying an AI agent is just like hiring a human employee. How do you best train them? What privileges do you give them and when? How much access do you permit to sensitive data?

Like an MI6 handler, you really do need to know your agent.

john.thornhill@ft.com