14x HBM Capacity, 3.2TB per XPU: New Answers to the AI Memory Wall
14x HBM Capacity, 3.2TB per XPU: New Answers to the AI Memory Wall
Following the 2026 OCP Global Summit (October 12–15), TrendForce surveyed multiple vendors' new answers to the AI memory wall, with the core direction being to extend the memory hierarchy beyond HBM.
TrendForce 2026 Korea Roadshow takes place in Seoul on November 18. Join us for insights on how advanced packaging is reshaping the foundry industry, the next growth drivers for DRAM and HBM, and the emerging role of LPDRAM in next-generation computing. REGISTER HERE.
Over the past several months, TrendForce has tracked AI infrastructure through its bottlenecks: the packaging, materials, power, and memory constraints that each set a ceiling on how far compute can scale. Memory is the one that keeps coming back. As inference moves toward multi-turn agents and longer contexts, the KV Cache and model parameters that must stay within reach of the GPU keep growing, while HBM remains limited in capacity.
Related report: HBM Industry Analysis - 4Q26
At the 2026 OCP Global Summit (October 12 to 15), several vendors presented new answers to the AI memory wall, focused on expanding the memory hierarchy beyond HBM. SK hynix proposed moving memory into a dedicated rack and stacking NAND beside the GPU. Metrum AI and Solidigm measured what a shared SSD pool does to latency and power. Nallasway and its partners designed an XPU module that connects 3.2TB of LPDDR5X through high-speed SerDes.
These approaches operate at different levels of the system, and sit at different stages, from published specifications to early design. This article breaks down each one, the data behind it, and where it stands.
Related report: NAND Flash Market Bulletin - Nov. 4, 2026
Source: TrendForce Insights Semiconductor Industry Insights · insights.trendforce.com