当 HBM 不再唯一,AI 内存的新棋局

Category: China Perspective (Bilibili) · Duration: 5 min · ▶ Watch

Speakers: ICVIEWS 半导体产业纵横

Switch language → 中文

Segments (8)

  • 00:00 · HBM技术简介
    • 介绍了高带宽内存（HBM）作为下一代DRAM技术，通过垂直堆叠实现超高带宽，成为AI大模型训练与推理的关键组件。
  • 00:20 · HBM市场格局变化
    • 阐述了SK海力士凭借HBM技术优势，在市场中地位持续攀升，与三星的份额差距拉大，并首次在DRAM领域超越三星。
  • 01:10 · 三星Z-NAND技术
    • 三星计划重启搁置七年的Z-NAND存储技术，作为HBM的替代方案，旨在大幅提升性能、降低功耗和延迟。
  • 01:34 · NEO Semiconductor的3D X-DRAM
    • NEO Semiconductor推出面向AI芯片的X-HBM架构，基于其自研的3D X-DRAM技术，旨在突破传统HBM在带宽和容量上的瓶颈。
  • 02:29 · Saimemory的全新堆叠式DRAM
    • 由软银、英特尔和东京大学联合创立的Saimemory正在研发全新堆叠式DRAM架构，目标是成为HBM的直接替代者或更优方案。
  • 02:55 · 存内计算（PIM）技术
    • 介绍了存内计算（PIM）技术，通过将计算单元集成至数据存储位置，从根本上解决冯·诺依曼架构的数据移动瓶颈。
  • 03:33 · 华为UCM推理记忆数据管理器
    • 华为发布UCM技术，通过管理KV Cache来优化大模型推理过程，实现高吞吐、低时延，降低对HBM的依赖。
  • 04:05 · 未来AI内存展望
    • 总结认为，未来AI内存市场将呈现架构多样化，多种技术并存，共同构成一个异构、多元的层级体系，以适应不同应用场景。

Memory Facts (8)

  • [00:02] HBM (High Bandwidth Memory) is a type of DRAM that vertically stacks multiple DRAM dies.
    • Layers
  • [00:13] HBM’s 3D architecture provides bandwidth far exceeding traditional GDDR memory.
    • Bandwidth
  • [01:20] Z-NAND performance is close to DRAM, with system latency much lower than traditional SSDs.
    • Latency, Performance
  • [01:25] Z-NAND is based on an improved V-NAND design, using a 48-layer structure and operating in SLC mode.
    • Layers, Mode
  • [01:44] 3D X-DRAM is described as a ‘3D NAND-like DRAM’.
    • Architecture
  • [02:14] X-HBM can achieve a 32K-bit data bus and a single-chip capacity of 512Gbit.
    • Bus width, Capacity (Gbit)
  • [02:20] X-HBM’s bandwidth and density are 16x and 10x that of existing memory, respectively.
    • Ratio (x)
  • [02:45] Saimemory’s technology aims to double storage capacity and reduce power consumption by 40-50%.
    • Capacity, Power

Bottleneck Claims (4)

  • [01:45] Traditional HBM faces bottlenecks in bandwidth and capacity.
    • Evidence: The video introduces 3D X-DRAM as a technology that breaks through these bottlenecks.
  • [01:51] Traditional DRAM has 2D scaling limitations.
    • Evidence: 3D X-DRAM is presented as a solution that breaks this 2D scaling limit.
  • [01:54] Horizontal expansion of memory leads to area and power consumption problems.
    • Evidence: 3D X-DRAM’s vertical stacking is shown as a solution to this problem.
  • [02:59] The traditional von Neumann architecture suffers from a data movement bottleneck, where system performance is limited by data transfer delays.
    • Evidence: The video explains that processing units and storage are separate, causing delays.

Predictions (4)

  • [04:20, Future] The future AI memory market will not be a simple binary of ‘replacement vs. being replaced’.
  • [04:31, Future] The AI computing and memory domain will not see a single winner that completely replaces HBM.
  • [04:41, Near Future] The era of a single memory solution dominating high-performance computing is ending.
  • [04:44, Future] The future AI memory landscape will be a heterogeneous, multi-level system with different technologies for different scenarios (HBM for training, PIM for high-efficiency inference, etc.).

Key Technologies (6)

  • HBM (High Bandwidth Memory): Vertically stacks DRAM dies to achieve ultra-high bandwidth for AI accelerators.
  • TSV (Through-Silicon Via): Provides vertical electrical connections between stacked dies in HBM.
  • Z-NAND: Samsung’s high-performance, low-latency NAND flash technology, positioned as an alternative to HBM for certain workloads.
  • 3D X-DRAM: NEO Semiconductor’s technology that uses a 3D NAND-like structure for DRAM, aiming to overcome traditional DRAM scaling limits.
  • PIM (Processing-in-Memory): Integrates computation directly into memory to reduce or eliminate the data movement bottleneck between CPU/GPU and memory.
  • UCM (Unified Cache Manager): Huawei’s technology for managing KV Cache in AI inference, optimizing performance and reducing reliance on HBM capacity.

Companies Mentioned (9)

SK海力士 (SK Hynix) · 美光 (Micron) · 三星 (Samsung) · NEO Semiconductor · 软银 (SoftBank) · 英特尔 (Intel) · 东京大学 (University of Tokyo) · Saimemory · 华为 (Huawei)

Notable Quotes (3)

HBM替代方案的创新版图呈现出“架构哲学的多样性”，而非单一技术的迭代。 — Narrator @ 04:24

取而代之的将是更复杂、分散化且贴合应用场景的内存层级结构。 — Narrator @ 04:35

单一内存解决方案主导高性能计算的时代正在落幕。 — Narrator @ 04:41

Key Topics

AI Memory · Semiconductor · HBM (High Bandwidth Memory) · DRAM · NAND Flash · Processing-in-Memory (PIM) · System Architecture · Market Competition · Technological Innovation

Takeaways

  • HBM is the current standard for high-performance AI memory but is facing increasing competition from alternative technologies.
  • SK Hynix has established a dominant position in the HBM market, surpassing competitors like Samsung.
  • Companies are developing diverse alternatives to HBM, such as Samsung’s Z-NAND and NEO Semiconductor’s 3D X-DRAM, to address cost, power, and scaling limitations.
  • New system architectures like Processing-in-Memory (PIM) and Huawei’s UCM are emerging to tackle the fundamental data movement bottleneck, reducing reliance on pure memory bandwidth.
  • The future of AI memory is not a single-winner market but a heterogeneous ecosystem where different memory technologies are optimized for specific tasks (e.g., HBM for training, PIM for inference).
  • For China, there is a significant gap in domestic HBM technology, making the development of alternative memory architectures a strategic priority to support its domestic AI industry.