Marvell Blogs

Marvell Blogs

January 28, 2025

Marvell leaders share how data centers are transforming to accommodate AI

By Kirt Zimmer, Head of Social Media Marketing, Marvell

What do you get when you combine some of the world’s leading technology analysts with incredibly smart subject matter experts? Answer: the SixFive Media video podcast. It’s must-view content for anyone interested in understanding exactly how AI technologies are evolving.

At Marvell’s recent Investor Analysts Day, company leaders were happy to chat with Patrick Moorhead, CEO and Chief Analyst at Moor Insights & Strategy, and Daniel Newman, CEO and Chief Analyst at The Futurum Group. The resulting conversations (captured on video) were enlightening:

How Custom HBM is Shaping AI Chip Technology

Fresh off Marvell’s announcement of a partnership with SK Hynix, Micron Technology and Samsung Semiconductor, Patrick and Daniel dove into the details with leaders from those organizations. The partnership centers around custom high bandwidth memory (HBM), which fits inside AI accelerators to store data close to the processors.

Custom designs alleviate the physical and thermal constraints traditionally faced by chip designers by dramatically reducing the size and power consumption of the interface and HBM base die. Marvell estimates that up to 25% of the real estate inside the chip package can be recovered via customization.

Will Chu, SVP and GM of Custom Compute and Storage at Marvell, says the company estimates that the total addressable market (TAM) for data centers in 3-4 years is $75B. Last year it was $21B. Out of that, Marvell estimates that $40-43B is for custom accelerators.

Attached to that is custom HBM, which alleviates bottlenecks for AI workloads. In Dong Kim, VP of Product Planning at Samsung Semiconductor said, “Custom HBM will be the majority portion of the market towards the 2027-28 timeframe.” As Patrick Moorhead said, “The rate of change is phenomenal.”

A New Nervous System for AI

If you’re interested in the architecture and optimization of a data center – or a set of them – this is a brief master class in the concept of “scale up and scale out” networking.

Nick Kucharewski (SVP and GM, Switch at Marvell) explains that “scale up” is about adding additional clusters of XPU capacity while making it appear as a single computer to a software application. Expanding beyond a rack of clusters to a row of them, interconnect becomes a critical factor. Copper doesn’t have the reach for a whole row, so optical enters the chat.

“The death of copper has been greatly exaggerated,” according to Achyut Shah (SVP and GM, High Speed Connectivity and Phy at Marvell), but copper can’t handle longer distances between clusters. That’s where “scale out” comes in, connecting these XPU superclusters working on LLMs.

Distributed data centers can be a few hundred or thousands of kilometers apart, but high bandwidth connectors make it feel like one seamless cluster. There are some very interesting tradeoffs between bandwidth and latency that this conversation explores.

The Future of Silicon is Custom

The headline of this video says it all. Nigel Alvares, VP of Marketing at Marvell, talked with SixFive about how hyperscalers are looking to differentiate, optimize their total cost of ownership (TCO), and not become a commodity. As they push for more compute power at a lower power footprint, fundamental changes in data center architecture are emerging. Among other benefits, custom silicon reduces the enormous power impacts of data centers being built and networked to manage AI workloads.

Nigel covers the fundamentals of a successful relationship between a hyperscaler and a semiconductor company, and how rare it is for a company to have an IP portfolio that includes all aspects of a data center – compute, storage and networking. Ultimately, cookie cutter solutions just don’t cut it anymore.

Tags: custom computing, ASIC, AI, AI infrastructure

Recent Posts

Archives

Categories

Accelerated Infrastructure is the Foundation of AI