Google transforms its data center architecture for agent era

Google’s data center team is racing to turn its infrastructure into a well-oiled machine for AI and the onslaught of agents. At this year’s Google I/O, CEO Sundar Pichai shared startling numbers: Google’s data centers processed about 3.2 quadrillion tokens a month, roughly seven times more than the 480 trillion processed in May 2025.

“Multiple agents work together, and now you’ve got millions, billions of users around the world potentially spinning off agents to help them do things,” said Mark Lohmeyer, vice president and general manager for AI and computing infrastructure at Google.

Google’s new data-center blueprint includes updated hardware, software, and orchestration layers to keep always-running agents operational.

In the LLM era, users sent prompts and received responses, and Google’s infrastructure was designed for latency and throughput. But agents could increase inference transactions by up to 100 times non-agentic workloads, Lohmeyer said. Google’s redesigned AI data-center stack has the elasticity for agents to be widely distributed, run for long periods, and make decisions independently.

“We’re delivering new platforms every year, each one optimized for what we think the world is going to need for the age of agents going forward,” Lohmeyer said.

Efficient data flow is key so agents can act, reason, and decide faster.

Google adjusted the Google Kubernetes Engine into an agent-native environment, where agents could be quickly spun up in sandboxes and containers. “From an infrastructure perspective, you need to spin up a bunch of TPUs or GPUs very rapidly. Then you need to be able to run them and spin them back down,” Lohmeyer said.

Google also made drastic improvements to its silicon to support its middleware changes. It recently introduced new AI chips, with the TPU-8t for training, and TPU-8i for inference. The 8t chip has three times more computing power than the previous-generation Ironwood chip. The 8i chip has 384 megabytes of SRAM and 288GB…