From Google to AWS: How cgroups Built the Foundation of Cloud Multi-Tenancy

Background of the cgroups concept in Linux: Why Google created cgroups

Before 2006, Google used to have thousands of servers managed by their legendary tool Borg; a single server was running multiple Google services like Gmail and web search, and because of this setup, Google was experiencing the noisy neighbor problem.

The “Noisy Neighbor” problem occurs when a single application or tenant on a shared physical server monopolizes critical hardware resources—such as CPU, RAM, Disk I/O, or network bandwidth—at the expense of others. In a multi-tenant setup where dozens of applications run side-by-side on the same machine, an unmanaged process experiencing a bug, memory leak, or unexpected spike in traffic will greedily consume all available system capacity. Because standard operating systems historically lacked strict boundary controls between processes, this runaway consumption starves neighboring applications of the resources they need to function properly, causing severe performance drops, high latency, or complete application crashes.

Linux cgroups solves this exact issue by enforcing hard resource quotas and isolation walls for process groups. Instead of letting every process compete freely for all available hardware, cgroups allows system administrators and cloud platforms to assign strict caps—such as pinning a specific service to a maximum of 2 CPUs and 4 GB of RAM. If a particular application suddenly spikes or enters an infinite loop, cgroups throttles or isolates that specific process once it hits its allocated limit. This guarantees that surrounding applications continue running smoothly without experiencing performance interference, effectively neutralizing the noisy neighbor threat in shared cloud environments.

Rohit and Paul to the rescue

Rohit, Seth, and Paul Menage solved the core resource management issue by developing a lightweight, in-kernel framework originally called Process Containers (later renamed Control Groups or cgroups). Instead of creating heavy virtual machines to isolate tasks—which would waste massive amounts of CPU and memory—they built a system directly inside the Linux kernel that groups processes together and enforces strict hardware boundaries around those groups in real time.

To achieve this, they introduced hierarchical process grouping alongside specialized kernel subsystems for each primary resource. They engineered memory controllers to set hard memory caps and trigger out-of-memory handlers specifically for runaway container groups without crashing the rest of the host system. For processing power, they integrated CPU controllers that leveraged the Linux scheduler to allocate CPU time fractions and throttle low-priority tasks whenever high-priority services needed immediate compute power. Additionally, they hooked into the kernel’s block I/O layer to limit read and write speeds, preventing background log processing or disk-heavy batch jobs from freezing the disk for user-facing applications.

By exposing this entire control mechanism through a simple pseudo-filesystem (/sys/fs/cgroup), they allowed operating systems and management systems like Google’s Borg to dynamically tweak, monitor, and restrict process resource consumption on the fly. This breakthrough allowed thousands of isolated processes to run safely on a single machine at maximum hardware efficiency, effectively creating the core foundation for modern containerization and cloud infrastructure.

Decision to open-source it.

1. Hard Limits of Out-of-Tree Kernel Patches

Google relied heavily on customized Linux kernels to manage its massive server infrastructure. Maintaining deep kernel modifications privately (out-of-tree) creates immense technical debt.

• Every time the official Linux kernel updated, Google engineers had to rewrite, rebase, and debug their internal patches to keep up.
• By upstreaming cgroups into the official Linux kernel mainline, the global open-source community took over testing, maintaining, and improving the codebase alongside Google.

2. Upstream Linux Architecture Philosophy

To control hardware resources like CPU, memory, and disk I/O efficiently, cgroups had to hook directly into core Linux subsystems—such as the process scheduler, memory manager, and block I/O layer.

• Linux kernel maintainers (including Linus Torvalds) generally refuse to support proprietary or external hooks for core functionality.
• For cgroups to operate at maximum performance without hacky workarounds, it had to be integrated natively into the core Linux codebase.

3. Open Standards Drive Hardware & Ecosystem Growth

Google recognized that driving down the cost of cloud and server infrastructure required a universal open standard:

• Ecosystem Adoption: When Linux becomes better at resource management, hardware vendors, enterprise distros (like Red Hat and Ubuntu), and software tooling align around those same standards.
• Reciprocal Innovation: Open-sourcing cgroups enabled other organizations and developers to contribute enhancements (such as memory controllers, security extensions, and eventually cgroups v2), which Google then pulled back into its own infrastructure.

How AWS and Azure are taking advantage of cgroups

Both Amazon Web Services (AWS) and Microsoft Azure leverage Linux cgroups to build, scale, and monetize their cloud ecosystems.

Without cgroups, neither provider could offer high-density serverless computing, managed Kubernetes, or multi-tenant container hosting profitably.

Subscribe to our free Newsletter for Deep System Articles, Dont scan huge books for understanding deep concepts; just subscribe and get 3 weekly articles, completely free for lifetime

The form can be filled in the actual website url.

1. Managed Kubernetes (AWS EKS & Azure AKS)

Both AWS Elastic Kubernetes Service (EKS) and Azure Kubernetes Service (AKS) run millions of containers for different customers on shared Linux host nodes.

• Enforcing Pod Limits: When a developer sets resources: limits: { cpu: "500m", memory: "256Mi" } in a Kubernetes YAML file, the underlying container runtime (containerd or CRI-O) translates those exact values into Linux cgroups settings on the host node.
• Resource Prioritization & OOM Handling: If an application on AKS or EKS attempts to consume more memory than allowed, cgroups triggers an Out-Of-Memory (OOM) kill event specifically for that single container, protecting neighboring customer pods on the same node.

2. MicroVM Security & Serverless Isolation (AWS Lambda & Fargate)

AWS built Firecracker, an open-source Virtual Machine Monitor (VMM) written in Rust, to power AWS Lambda and AWS Fargate.

• The Firecracker Jailer: Firecracker wraps every microVM inside a dedicated cgroup jail alongside Linux namespaces and seccomp filters.
• Multitenancy on Serverless: Even if malicious code breaks out of a lightweight virtual machine, the surrounding cgroup boundary prevents it from monopolizing the host’s CPU cores, memory channels, or disk bandwidth.

3. Native Linux Distros & Custom Kernel Optimization

Both hyperscalers maintain their own Linux distributions optimized specifically for cloud hosts: Amazon Linux and Azure Linux (formerly CBL-Mariner).

• cgroups v2 Adoption: Both platforms have migrated their default node OS images to cgroups v2.
• Pressure Stall Information (PSI): By using cgroups v2, AWS and Azure host systems monitor kernel-level resource pressure (CPU, memory, and I/O starvation) in real time. This telemetry feeds into auto-scaling algorithms and bin-packing schedulers to maximize hardware usage across their datacenters.

4. Billing, Metrics, and Resource Accounting

cgroups does not just limit resources; it accurately tracks resource usage over microsecond intervals.

• Granular Billing: Serverless products (like Azure Container Apps, AWS Lambda, or Fargate) charge per millisecond of CPU time and megabyte of memory used.
• Accounting Controller: Cloud providers query cgroup metrics (such as cpuacct and memory.current) directly from the Linux /sys/fs/cgroup virtual filesystem to generate accurate usage metrics for customer billing.