SK hynix presented several next-generation memory solutions for AI servers at OCP Global Summit 2026, held October 12–15 in San Jose.
Semiconductor Engineering examines how open-weight AI models affect chip architecture, noting the focus is shifting from peak TOPS to memory bandwidth and capacity, sustained throughput, numerical format flexibility, and deployment security.
Researchers from ETH Zurich and NYU published a paper presenting the first comprehensive experimental characterization of power consumption on real HBM2 DRAM chips, with 13 new findings and 6 key conclusions. Read operations draw significantly more current than other operations, and flipping all bank group bus signals increases power by 29.4% over holding them constant. Based on this, the team constructed access patterns that on average consume 23.6% more power than the JEDEC-specified maximum power measurement loop across 36 tested HBM2 chips.
An SK hynix research team published a paper at the 2025 IEEE International Electron Devices Meeting (IEDM) proposing co-optimization of cell, non-cell, and page schemes to improve the energy efficiency of analog compute-in-memory (A-CiM) based on 3D FeNAND.
Semiconductor Engineering collects multiple recent chip technology papers, covering spatially deterministic nucleation of 2D semiconductors from KAIST and other institutions, the Hyper-NA EUV lithography system with a numerical aperture of at least 0.75 from Zeiss SMT and ASML, and BARE-AI from the University of Florida and others, which uses built-in performance monitors to defend against bit-flip attacks in AI hardware.
Marvell has launched its second-generation Structera X 3406 and 3506 CXL 3.2 memory expanders, built on an industry-first six-channel DDR4/DDR5 design that increases channel count by 50% over the previous generation, supports PCIe Gen 6, and allows a single controller to support up to 9TB DDR4 or 6TB DDR5.
Marvell released the second-generation Structera X 3406 and 3506 CXL 3.2 memory expanders, the industry's first six-channel CXL memory expansion products, with 50% more channels than the previous generation, supporting up to 9TB of DDR4 or 6TB of DDR5 per controller and delivering up to 256GB/s of host bandwidth over PCIe 6.0 x16.
Memory supply tightness will persist through 2027, with DRAM lead times at 20 weeks versus a balanced level of 8 weeks; vendors are shifting PC DRAM capacity to server DRAM, and HDD lead times remain at 50 weeks.
In Q3 2026, 80 startups raised more than $8.5 billion combined, with two inference companies each raising $1 billion and nearly $7.5 billion going to 22 companies with rounds of $100 million or more. Investor focus centered on interconnect technology, networking chips and photonics, including co-packaged optics, millimeter-wave plastic waveguides, ferroelectric glass modulators and coherent DSPs for AI optics, while some companies explored replacing constrained HBM with ordinary DRAM or on-chip SRAM.
Research teams from Syracuse University, FAU, and TU Dresden published a paper on RAPID (Row-parallel Arithmetic Processing-In-DRAM), an architecture that performs row-parallel, bit-parallel computation directly inside DRAM.
Researchers from ETH Zürich, CISPA, and NYU have published a paper proposing the Terracotta framework, which simplifies deploying new DRAM technologies through a flexible DRAM interface and a programmable memory controller. The framework includes two components: a custom command extension allowing DRAM vendors to define new commands within a single standardized interface, and a programmable memory controller that system designers can program after tape-out to support new DRAM technologies. The paper is available on arXiv (number 2610.06475).
Semiconductor Engineering collects several new chip technical papers: oxygen-doped WSe2 monolayer ballistic p-type transistors achieve low contact resistance and high current density; UC Santa Cruz explores the design space of backside clock mesh networks for 2nm GAAFET backside power delivery technology.
Nvidia announced that the Shield TV Pro, released in 2019, is getting a $100 price increase effective immediately, bringing the new price to $299.99. An Nvidia spokesperson confirmed to Ars Technica that the adjustment took effect October 2, citing sharply rising industry-wide costs for memory and other components; the non-Pro model has already been discontinued. The Shield remains in production, and new units face the same supply chain cost pressures as current devices.
Through a decade of venture investing, SK hynix has evolved its corporate VC model from "tech sensing" to "path finding," turning internal R&D, mass production and customer resources into assets startups can leverage.
TrendForce's weekly radar focuses on server CPUs: agentic AI has pushed lead times to 25–30 weeks, well above the balanced level of 16–20 weeks. Meta's AI agent Muse reportedly surpassed ChatGPT in downloads early after launch, driving AMD's market cap past $1 trillion for the first time, and at Connect 2026 Meta unveiled the Meta Charm handheld device powered by Muse.
TrendForce has observed that some cloud service providers are adding QLC SSDs to SSD PODs, replacing the default TLC SSD plus HDD configuration. In AI inference, the Prefill and Decode stages repeatedly read and write KV Cache, and as context windows and batch sizes grow, KV Cache capacity needs multiply, prompting vendors to move caching from costly, capacity-limited HBM down to cheaper tiers.
On September 8, SK hynix held its 2026 Future Forum at the SUPEX Center on its Icheon campus, under the theme "The Golden Age of the AI Era: The Moment for AI Memory Solution Creators to Change the World," discussing AX (AI transformation) trends and the direction of memory technology.
In its AI Ecosystem series, SK hynix notes that as AI shifts from pretraining to inference and agentic AI, the system bottleneck is moving from GPU compute to where data resides and how it flows. The memory wall proposed by Wulf and McKee in 1994 is becoming reality: AI accelerators now reach petaflops of compute, but memory read speeds cannot keep up, leaving many compute units idle waiting for data. The response is to increase memory bandwidth or move memory closer to the processor.
At FMS 2026, Marvell unveiled a new-generation AI memory infrastructure portfolio spanning server-level AI storage, rack-level CXL memory expansion and pooling, and cabinet-level optical shared memory.
Marvell will showcase its AI memory and storage portfolio at FMS 2026 on August 4-6 at Booth #805, featuring PCIe 6.0 enterprise SSD controllers, CXL near-memory acceleration and memory pooling switches, CXL memory expansion cards, Octeon DPUs, and Photonic Fabric technology for optical interconnect scaling.
TrendForce analysts explain three paths tech giants are taking to compress the KV Cache through architectural and algorithmic innovation, with Google and DeepSeek respectively reworking attention mechanisms via GQA and MLA.
In the first half of 2026, the core constraint of AI inference has shifted from compute to memory, as surging KV Cache demands meet limited memory supply and become the main bottleneck. For Llama 3 70B FP16, model weights are about 140GiB, but the KV Cache reaches 160GiB at 64K context and balloons to 320GiB at 128K, about 2.3 times the weight size.
As test-time scaling rises, the AI industry's focus is shifting from training to inference, bringing structural changes to the memory hierarchy. In the first half of 2026, NVIDIA and Google respectively launched CMX, Rubin integrated Groq LPX, and the TPU v8i designed for inference (with 72 GB more HBM3e and 384 MB SRAM than the TPU v8t).