AI Infrastructure Summit Day 1
Today is a guest post from Dustin Sklavos, former AnandTech Senior Editor, product manager at Corsair, and technical writer at Tenstorrent. Dustin brings almost two decades of experience writing and working in the industry, managing hardware line revenues north of $100m, and routinely acts as a robust sail in the face of superficial marketing. While looking for an interesting yet new challenge in the industry, I’m glad to have him writing for More Than Moore this week at the AI Infra Summit.
AI Infra Summit Keynote Review
2026’s AI Infra Summit kicked off with three different sets of speakers and a couple of clear themes emerging. If we’re after anything this year at the AI Infra Summit, it’s understanding if the people are saying anything different compared to last year (here’s a hint, they kinda are!).
We should highlight that the main day one keynote was a fireside chat between our own Dr. Ian Cutress and Professor Dave Patterson, Google Fellow and well known by every computer science student from here to Timbuktu. Ian and Dave focused on the key trends limiting hardware development for the future.
Next up was a presentation by Ian Buck, VP/General Manager of Hyperscale at Nvidia, who last year announced the NVIDIA CPX prefill chip (which was summarily canned by the end of the year). This year his focus was on the first NVL72 Vera Rubin benchmark results, especially how they created an architecture built for Agentic AI. Apparently everyone else did as well, so the proof is in the numbers.
Closing out the keynotes was a fireside chat between AI Infra Summit co-founder Ed Nelson and Intel CEO Lip-Bu Tan. We’ll start with Lip-Bu’s comments as they’re worth highlighting.
Lip-Bu Tan and Intel
The conversation with Lip-Bu Tan was predominately about the state of Intel, its advances, and Lip-Bu Tan’s investments as a VC and personal partnerships.
Intel’s fabrication technologies are moving along nicely, especially their EMIB-T packaging - LBT has said in the last earnings call had demand exceeding supply, and given that Intel isn’t using EMIB-T internally yet points to a good external packaging customer. LBT highlighted that Intel’s organization has been “made more efficient.” (Intel saw a mass layoff of ~30% of its workforce in the last year) under his stead as well.
Insights from LBT on the state of AI were thin on the ground however; lip service was paid to AI’s poor PR as an industry, and LBT acknowledged the problem, citing Silicon Valley as a bubble out of touch with the rest of the country/world when it comes to factors such as adoption, however the interviewer nor LBT engaged much beyond stating the issue.
LBT did divulge two particularly interesting pieces of information. First was why Gaudi ended the way it did: Intel had bought Habana Labs, then promptly the leadership left. I’m sure there’s some saying about a captain knowing how the rudder works; ultimately, when I asked Ian, he said it was more to do with the fact that even the architecture of the third generation was designed pre-acquisition, and they had difficulties integrating the front end of that architecture with the universal software stack that Intel was promoting at the time.
Second, Intel is refocusing on software leadership and bringing on a major hire in the near future to drive that initiative. This provides some interesting food for thought: the main CTO under the previous Intel CEO was Greg Lavender, and Greg was very much a software focused chief technology officer. Greg departed Intel roughly the same time Pat Gelsinger did, and Intel went on the hunt for a new CTO for a few months; the role ended with our good Twitter friend Pushkar Ranade. Pushkar has only been in the role for a few months, but is more of a hardware person, so a software guru or ninja would be a role worth filling at Intel.
Memory and Power
The conversation with Professor Dave Patterson and the presentation by Ian Buck addressed, either directly or indirectly, the major challenges facing the AI industry at the data center level. Two of the three biggest bottlenecks people at the show are focusing on are ones people are already familiar with: memory and power (the third being networking). Patterson specifically stated that we were entering a “memory-centric era” and the facts bear that out: while many silicon and IP vendors on the show floor were finding ways to address bandwidth requirements, they still remain a thorny issue second only to actual memory capacity, and the general need for large quantities of memory doesn’t seem to be abating.
On top of this, there are recent reports of rather than using the top capacity 8/12/16-hi memory stacks for HBM4, leading customers might be considering 4-hi HBM memory stacks to help spread the memory around. Using smaller memory stacks means more individual stacks overall for the same bandwidth, at the cost of capacity. Note that this only works with a good scaling solution. Other vendors are insisting that high capacity is still the name of the game.
The other problem with memory has to do with the difficulty in novel forms of memory breaking through into the industry. It transpires that customers don’t want to deal with vendor lock-in of a new startup driven solution; a proprietary memory solution forces them to stick with one vendor. Memory has historically been a commodity product, allowing any company to choose a solution from any vendor. Normally vendor lock-in generates real risk, ergo the only real way for new memory technologies to be widely adopted is to have them fabricated by multiple companies. While it’s a nice sentiment when referencing new technologies, HBM4 is going down the custom route, ending up with vendor specific base die integration, ultimately eliminating the commodity nature - so it’s accepted when the technology is known, not when it’s unproven without benefit.
Power also continues to be an issue that the industry seems to be aware of, and Patterson and Buck both discussed driving efficiency at both the chip and the data center level. Patterson was especially interested in data centers being built in places with ready access to clean energy and actively reducing the carbon footprint of AI data centers, arguing that data centers built closer to urban centers are at the mercy of using power sources that are less environmentally friendly. He also referenced research he has done recently showcasing that yes indeed, the latest chips are often the most efficient, even if there is power creep per chip.
Buck looked at power from a different perspective, eyeing performance per watt as a key metric. Vera Rubin was touted to have been designed with this in mind; regardless of the overall power consumption of Vera Rubin NVL72, Nvidia is citing performance improvements in the order of 30x the number of tokens per watt in areas where the previous generation hardware was practically unable to play in due to spill-over (and these are vendor numbers and the appropriate amount of salt should be taken). They’re also developing technologies built around intelligently maximizing utilization of the provisioned power at data center sites, using their traditional “Max-Q” branding, but this time at the data center level.
Chip Design For The Masses
Professor Patterson also spoke about a paradox in chip design: the constant advances in AI development are necessitating constant advances in the way the processors that handle AI are designed, including a renewed focus on the host CPU, something Ian Buck also emphasized in discussing Vera’s prioritization of single-threaded performance over core count. AI is changing the game at both the chip and the server level, and Ian Buck highlighted that the old paradigm with high core count in data centers is gone - AI agency is latency sensitive and benefits more from the shift Nvidia is making with Vera.
One of the questions posed to David Patterson was what engineers coming out of school should be targeting, and he suggested that even with a pivot towards application specific silicon like GPUs and ASICs, students and engineers should still aim for a generalized understanding of silicon engineering. Despite silicon becoming more and more specialized, especially in the AI era, and AI itself is such a moving target that trying to specialize risks obsoleting your own education. He stated that engineers have to be flexible because they’re dealing with a constantly shifting landscape so as to not been left behind. With AI models pivoting constantly, the engineering has to keep step.
Finally, amusingly enough, when Patterson was asked about his own AI usage, he said the programmers he worked with have been stunned (in a positive way) by the results of the coding agents they use, and he himself uses several different AI agents. But he clarified that multiple agents are required “because AI lies.” It was a cheeky answer but a friendly reminder that AI can be an incredibly powerful tool, so long as it isn’t left unsupervised.
Final Thought
The show here is another couple of days, so expect to see further insights from the event including round up videos over on the TechTechPotato YouTube channel. Day one is usually a wall of players all vying for attention, so day two should allow us to noodle on some of the more interesting topics on the show floor. We’ll just leave you with the following from our good friend at EE Times, Sally Ward-Foxton, who very appropriately captured the final seconds of Ian’s talk with David.
Ian is currently recovering in the Santa Clara Municipal Hospital burns unit.
[The full conversation with Ian and David is expected to be made public after the event]
Thanks for reading More Than Moore! This post is public so feel free to share it.
点赞
收藏
点踩
分享