Deep|Optics: Scale-In Moves Optics Inside the Tray, and the Bandwidth Opportunity Could 10x Scale-Up

Core conclusion

Optics are moving inside the tray, and the memory tier it targets is the largest bandwidth tier in the system, roughly ten times the scale-up bandwidth per GPU and about one hundred times the scale-out. The capital is already moving, with $2 billion each from NVIDIA into Lumentum and Coherent, $3.25 billion from Marvell for Celestial AI, and a revenue ceiling of up to $120 billion on the Google and Marvell warrant. Scale-in, in this note, means die-to-die and chip-to-memory optical links at millimeters to about a meter, which Lumentum’s CEO on August 27 called “the thing to watch over the next 12 to 18 months”. The word is spreading with split meanings since NVIDIA uses it for something else, and Section 3 sorts out who means what. The kicker is Section 7, where our checks point to the largest scale-in exploration running at Google, built around memory pooling, and the warrant’s disclosed scope, “memory interface controllers and near-memory compute”, is what we believe to be its public paper trail. The work is early, and first-generation products start well below the ceiling.

1. Scale-In’s Coming-Out Week

At an investment conference on August 27, Lumentum CEO Michael Hurlston said, “I think the thing that is going to be significant is the scale-in, and that’s the thing to watch over the next 12 to 18 months... Are you going to see it actually go in-tray and serve this high bandwidth connectivity between memory and between GPUs?” He was describing optical links inside trays. Scale-in was also featured in imec’s slides on the first day of Semicon Taiwan, in a talk on 3D integrated optics for chip-to-chip interconnect.

Also on August 27, Marvell said on its FY27Q2 earnings call that its fiscal 2028 revenue outlook for scale-up optics “has increased meaningfully compared to prior expectations”, against the roughly $300 million category framing it gave a quarter earlier, and that it is on pace to make about $1 billion of capacity prepayments to suppliers in fiscal 2027. The parts are the same since scale-up optics uses the lasers, optical engines, packaging, and test flows that scale-in will need, just at longer reach. Scale-up optics ships starting in 2027 while scale-in follows two to three years later, if not more, because those same parts still have to clear tighter thermal, reliability, test, and other requirements within the tray.

On August 19, a week before the earnings call, Google and Marvell announced a partnership that, in the filing’s words, “spans a comprehensive range of custom silicon programs that attach to the TPU ecosystem”. Marvell issued Google a warrant on up to 59 million shares. Most of it vests in 240 equal installments, one for each $500 million of qualifying revenue through fiscal 2033, which is where the $120 billion figure in the press comes from. None of that revenue is committed. The disclosed scope covers AI inference accelerators, storage controllers, network interface controllers, memory interface controllers, and near-memory compute. We believe the last two items refer to the pooled-memory tier behind scale-in, and Section 7 explains why.

At Hot Chips 2026, NVIDIA used Scale-In for a DPU services network, and Meta used it for chiplet-level data movement inside its MTIA 400 accelerator. Scale-in now has at least three different technical meanings.

2. The Definition Adopted in This Note: Optical Links Inside the Tray, From Millimeters to About One Meter

In this note, scale-in refers to optical links inside the tray that serve high-bandwidth connectivity between memory and between GPUs. This includes die-to-die connections between chiplets and compute die, and chip-to-memory connections such as XPU to HBM, over reaches from millimeters to about one meter. Links from the XPU to pooled memory belong to the adjacent pooled-memory fabric, which this note treats separately. The term also appears as scaling-in and, at Marvell, as scale-inside.

AI systems use different interconnect media at different physical layers. Within the package, CoWoS, interposers, and RDL dominate. On the board, systems rely on PCB traces and retimers. Within the rack, copper DAC and AEC cables handle scale-up traffic. Within the data center, pluggable optics dominate, and between data centers, coherent optics dominate. As bandwidth demand per XPU rises, optics enters one layer at a time, and the electrical-to-optical boundary moves progressively closer to the chip.

This also explains why bandwidth requirements rise at each inward layer. Scale-out is measured by network-port bandwidth, scale-up by fabric bandwidth, and scale-in approaches memory bandwidth. The closer the link is to the chip, the higher the aggregate bandwidth per XPU and the more demanding the power and density requirements.

Comparing those tiers requires conversion: a byte is eight bits, so 1 TB/s equals 8 Tb/s, and a link can be quoted one way or both ways. Lumentum’s ECOC 2025 material puts per-GPU scale-up bandwidth at 7.2, 14.4, and 28.8 Tbps for Blackwell, Rubin, and Feynman, counted one way, while NVIDIA quotes NVLink 6 at 3.6 TB/s, counted both ways. The two agree once directions are matched, since Lumentum’s 14.4 Tbps is per direction while NVIDIA’s 3.6 TB/s is the both-ways total of the same link. This note quotes bandwidth one way throughout, and memory-bus figures are aggregates.

The optical boundary moves inward from scale-out to scale-in. Source: FUNDA.

3. Everyone Means the Tray, Except NVIDIA

The optics and memory camp uses scale-in for the tier inside the tray. Marvell’s solutions page positions its portfolio “across scale-in, scale-up, scale-out and scale-across architectures”, and its blogs call the inner tier scale-inside, meaning die-to-die interconnects that move data between compute and memory dies inside XPUs, extending to chip-to-chip links within a tray. Avicena defines it as “AI scale-in (die-to-die and die-to-memory connectivity)” with reach from a few millimeters to one meter. Keysight bounds it at “the silicon and package domain including pre-silicon design, chiplets, memory and die-to-die/package interconnects”. Eliyan frames its market as “AI scale-up and scale-in networks”. LightXcelerate, a Palo Alto startup, states that its optical chiplet supports scale-in, scale-up, and scale-out across UCIe, UALink, PCIe, CXL, and NVLink.

Executives use the word the same way. Coherent CTO Julie Eng, on a Celesta Capital TechSurge panel with Eliyan founder Ramin Farjadrad and Lumentum cloud CTO Matt Sysak, put scale-across and scale-out in deployment now, large-scale scale-up deployment next year, and scale-in after that, describing it as solving “inter-die interconnects with light” when copper runs out in the die-to-die space. She also noted renewed interest in NRZ signaling for short links, “especially scale-up and die-to-die scale-in”.

Eliyan founder Ramin Farjadrad, Coherent CTO Julie Eng, and Lumentum cloud CTO Matt Sysak on the deployment sequence for optics. Source: Celesta Capital TechSurge, Aug 2026.

The research world has picked the word up as well. On the opening day of Semicon Taiwan, imec presented a slide at its ITF Taiwan forum titled 3D integrated optics for chip-to-chip scale-in interconnect, showing wide-and-slow area I/O with low-speed SerDes and a step from pluggable optics to co-packaged optics to an optics layer stacked directly under the XPU and HBM. On imec’s chart, the 3D step pulls away from co-packaged optics by roughly two orders of magnitude in bandwidth density per unit of energy (Gbps/mm per pJ/bit) by 2035. That came just four days after the Lumentum and Marvell statements.

imec at ITF Taiwan, Semicon Taiwan: 3D integrated optics for chip-to-chip scale-in interconnect. Source: imec. Photo by Jukan (@jukan05 on X), August 31, 2026.

At Hot Chips 2026, NVIDIA used the same word for a different layer. Its five-network AI factory model pairs Scale-In with the BlueField-4 DPU, a per-node infrastructure network for agentic CPU capacity, storage, security, and orchestration, quantified at 800 Gb/s per Vera Rubin compute tray, plus 4x1.6 Tb/s of scale-out. NVIDIA also has the pooled-memory tier this note discusses, called Context Memory (CMX), built on BlueField-4, and does not call it scale-in. Meta’s MTIA 400 slide deck uses “Scale-in DMA & streaming reductions” for data movement between chiplets inside the package, over electrical links. Microsoft’s Azure documentation has long used scale-in for removing virtual machine instances in autoscaling. The table below explains each term.

Meta MTIA 400: scale-in DMA and streaming reductions inside a multi-chiplet design. Source: Meta, Hot Chips 2026.

Scale-in as defined in this note, other uses of the term and the adjacent pooled tier. Source: FUNDA, redrawn from public sources. MPU means memory processing unit, covered in Section 7.

NVIDIA sells the pooled tier this note describes, under the CMX name. It has invested $2 billion each in Lumentum and Coherent with purchase commitments, and licensed Groq’s technology for SRAM-first inference. Its current scale-up racks run on copper.

The five networking infrastructures of the NVIDIA AI factory. Scale-In paired with BlueField-4. Source: NVIDIA, Hot Chips 2026.

4. Sizing It: The Memory Tier Is Roughly 10x Scale-Up