AI Chips’ New Secret Sauce Is Packaging

Making better chips for AI is increasingly about how they are laid out and stitched together—not just cramming together as many transistors as possible on a single piece of silicon.

The shift toward “advanced packaging” in the chip world—a term referring to a variety of strategies for knitting together pieces of silicon into one big chip—has long been building. And it looks certain to be a much bigger driver of chip advances in the future.

The reason is simple: The old ways of making chips better-performing and more energy efficient aren’t working as well as they used to.

For decades, chip advances came almost exclusively from innovations in how transistors were designed and how small they were. A cadence where the number of transistors on a given piece of silicon doubled roughly every two years took hold. The pace of advances became so reliable that it became known as Moore’s Law, named after Intel co-founder Gordon Moore, who first observed it.

In the past decade or so, as transistors became so small that they’re measured in nanometers, that law has come into question. Some chip-makers like Intel struggled to advance their manufacturing at the same pace as before. Others that did reliably advance—notably, Taiwan Semiconductor Manufacturing Co.—made less-enormous leaps in transistor density from generation to generation.

The rapidly rising cost of the most advanced chip-making technology presented another challenge. Well before the AI boom, that helped push the main makers of large central processing units—Advanced Micro Devices and Intel—toward changes in packaging. They moved away from etching circuitry onto a single piece of expensive silicon. Instead, they welded together numerous pieces of silicon, some made with older and less-expensive chip-making technology. As a side benefit, the strategy helped improve manufacturing yields.

AI, though, has kicked the packaging race into a new gear. Cost hasn’t been as big a consideration for Nvidia as it developed the AI chips that have driven the boom. But performance has, and its market-leading chips wouldn’t have been possible to manufacture without advanced packaging.

Nvidia’s chips use technologies developed by TSMC that bring together massive amounts of AI number-crunching power with lots of high-bandwidth memory, all within one advanced package. Constraints on the growth of packaging have been a bottleneck for Nvidia’s sales, although TSMC has converted old chip factories into packaging facilities and increased packaging capacity several times over in the past few years.

More investment in packaging is on deck as companies like TSMC, which reports quarterly results on Thursday, work with customers to push chip-making limits. Future packaging approaches increasingly involve stacking pieces of silicon on top of each other, not just side by side.

The shift toward packaging is making it more of a focus, despite being historically a lower-margin business with relatively little innovation. Chip manufacturers like TSMC are doing more of it themselves, bringing in-house an activity that previously was largely outsourced. And chip manufacturing equipment makers including Applied Materials have been touting packaging’s potential: Chief executive Gary Dickerson said in August that packaging-related revenue would grow more than 70% this year.

Number of operational data centers run by Firmus, an Australian cloud-computing company that scrapped its IPO plans after seeking a valuation of $30 billion.

Anthropic has long held the lead over rival OpenAI among business customers. Its Claude models became go-to tools for AI-driven coding, an area where automation can bring quick and dramatic returns.

There are signs that that lead is eroding. OpenAI’s models still may not be as adept as Anthropic’s in coding or other big business applications. But they have become cheaper following the release of new cost-efficient models in June—an advantage in a world where cost is increasingly critical.

The chart above from OpenRouter shows the spending patterns of 120,000 companies—largely AI-native startups. It suggests allocations between Anthropic and OpenAI have become close to even in the past month.

An Anthropic agent went rogue during testing this summer and submitted a false tip about a homicide to a police website in Philadelphia. The incident, which Anthropic uncovered more than two months after it happened in July, illustrates the variety of sometimes unexpected ways misbehaving agents can cause trouble. The tip in Philadelphia was routed to spam and didn’t prompt any police follow-up.

WSJ AI & Business is a weekly look at AI’s transformation of the business world. This newsletter was curated and edited by Asa Fitch. Reach him at asa.fitch@wsj.com (if you’re reading this in your inbox, you can just hit reply). Got a tip for us? Here’s how to submit.