What you’ll learn:
- What memory attributes best serve today’s AI data center and edge storage applications?
- Why DRAM and HBM underserve AI inference workload.
- How NAND-based High Bandwidth Flash prepares designers for the memory-centric era of AI compute.
Artificial intelligence is on a relentless march across the computing landscape. While about one in seven data centers today is equipped to host AI workloads, that’s expected to approach 70% by 2030.1 AI is migrating from hyperscale to enterprise data centers and out to the network perimeter, where edge AI applications are projected to generate nearly $66.5 billion by the end of the decade.2 The fuel for the new computing era is data — staggeringly large volumes that must be fed at high speed to demanding and rapidly scaling AI computing infrastructure.
These vast content repositories overwhelm conventional storage structures and bring an inherent architectural weakness into sharp relief. Data center memory (DRAM and specialized high-bandwidth memory known as HBM) is increasingly struggling to keep pace with the growing demands of large AI models in terms of density, storage capacity and scalability.
At the same time, hyperscale computing manufacturers are contending with rising DRAM and HBM production costs, design complexity, and energy consumption. The challenge is even more daunting in enterprise data centers and edge AI applications, where a proportionately smaller physical footprint renders them ill-equipped to absorb rising memory costs and power use.
Another pressing issue is introduced by AI inference, which is now the dominant AI workload and has different data-management requirements than AI training. Inference stores large — and growing — AI models, and HBM and DRAM-based memory have shown they lack the capacity and cost scalability to keep up with these new demands. Given these distinctly different memory characteristics, an opportunity exists for a memory technology optimized specifically for AI inference.
Why DRAM and HBM Underserve AI Inference Workloads
To understand why DRAM and HBM alone are suboptimal for long-term AI deployment, consider the following drawbacks.3 These began as small fissures, but if uncorrected, they will expand over time to undermine the foundation of next-generation AI-centric storage.
- Density penalties: DRAM capacity scaling has stalled while the need for higher capacity to address AI inference is growing.3
- Mismatched for AI inference: The advantage of DRAM's low latency and random-access characteristics aren’t relevant for AI inference, where access patterns are deterministic and more latency-tolerant thanks to techniques like data prefetching.3
Attributes of an Optimized AI inference Memory Architecture
These fault lines run beneath a $120 billion DRAM industry4 that’s eager to retain its hold on the data center, given that spending by hyperscale providers on AI infrastructure could reach $6.7 trillion by the end of the decade.5
What if it’s time to make a clean break and design a new memory from the ground up that meets the needs of the application, rather than the other way around? An AI-tuned storage-class memory would have the following attributes:
- Larger and scalable memory capacity provisioned for inference workloads
- Higher memory density (GB/mm²)
- High bandwidth to meet the requirements of AI inference
- Lower system-level power consumption
- Cost-effective metrics ($/TB)
High Bandwidth Flash Takes Aim at the AI Data Center
High Bandwidth Flash (HBF) is a disruptive new memory architecture, purpose-built to drive the next generation of AI computing. HBF meets the capacity, energy, throughput, and scalability requirements of advanced computing and data-intensive applications.
Compared to HBM, HBF provides higher capacity and memory density with comparable bandwidth that better aligns with AI inference trends (Fig. 1). As a persistent storage medium, HBF also retains data when power is lost and it’s thermally stable, supporting high operating temperatures.6
To realize these advantages, HBF effectively redesigns NAND flash by optimizing for high bandwidth and inference memory characteristics. The use of CMOS bonded array (CBA) wafer technology further enhances energy efficiency and bandwidth (Fig. 2).
HBF Reimagines NAND Flash for AI Applications
Compared to conventional NAND flash, HBF’s use of parallelism, advanced logic scaling, and custom stacking techniques helps deliver lower latency and significantly higher read bandwidth, enabling large language models (LLMs) to stream data at near-DRAM speeds.6
HBF also includes support for large KV caches to efficiently handle long, complex user prompts and customer- and domain-specific data that helps improve AI inference accuracy.
Extending Memory-Centric AI to the Enterprise and Network Edge
Because HBM isn’t generally available for use in edge and mobile environments due to density, cost, and power penalties, the value of larger memory capacity for handling more complex AI inference problems is realized with HBF. This opens the door for edge devices like smartphones that can make real-time decisions to manage a variety of sophisticated tasks. Thanks to its persistent memory, HBF supports the ability to seamlessly retrieve old context from previous queries to solve new problems.
The advantages of HBF extend to enterprise-level computing, where the user base is much smaller than hyperscale data centers and large GPU clusters supported by HBM are too costly. By adopting HBF-enabled accelerators, smaller enterprises can potentially fine-tune large, pre-trained models for domain-specific uses.
Optimized Memory Removes Obstacles to AI Computing Growth
All around us, data centers and edge AI devices operate autonomously, supporting tasks that range from tonight’s dinner recipe to groundbreaking scientific discoveries. Routine tasks like website hosting and enterprise data management are giving way to intelligent workloads that generate actionable insights using machine learning, deep learning, and data analytics.
It’s time to reconsider how data center and edge memory are provisioned to manage large-scale inference models that make predictions and generate outputs. Compared to HBM, HBF has a clear capacity advantage while delivering the high throughput required by AI inference applications.6 As a scalable new system memory technology, HBF helps reduce performance bottlenecks and accelerates time-to-insight for AI applications in modern data centers and edge networks alike.
References
- Srivathsan, M. Sorel, P. Sachdeva, with A. Bhan, H. Batra, R. Sharma, R. Gupta, and S. Choudhary, McKinsey & Company, “AI power: Expanding data center capacity to meet growing demand,” Oct. 2024.
- Grand View Research, “Edge AI Market Size, Share & Trends Analysis Report By Component (Hardware, Software, Services), By End-use Industry (Consumer Electronics, Smart Cities, Automotive), By Region, And Segment Forecasts, 2025-2030.”
- Legtchenko, I. Stefanovici, R. Black, A. Rowstron, J. Liu, P. Costa, B. Canakci, D. Narayanan, X. Wu, Microsoft Research, “Managed-Retention Memory: A New Class of Memory for the AI Era,” Cornell University, Jan. 2025.
- Fortune Business Insights, “DRAM Market Size, Share & Industry Analysis…,” Feb. 2026.
- Noffsinger, M. Patel, P. Sachdeva, with A. Bhan, H. Chang, and M. Goodpaster, McKinsey & Company, “The cost of compute: A $7 trillion race to scale data centers,” Apr. 2025.
- HBF Fact Sheet, Sandisk, “Sandisk Unveils The Future Of Memory Architecture For AI Introducing: High Bandwidth Flash,” July 2025.
About the Author
Alper Ilkbahar
Chief Technology Officer and Executive Vice President of Memory Technology, Sandisk
Alper Ilkbahar is Chief Technology Officer and Executive Vice President of Memory Technology at Sandisk, overseeing NAND technology development, next-generation technologies, corporate research functions, and market development.
Comment About the Article
To join the conversation, and become an exclusive member of Electronic Design, create an account today!
Leaders relevant to this article:
Facts Only
* AI workloads are expected to approach 70% of data center hosting by 2030.
* AI inference stores large, growing AI models that challenge the capacity and cost scalability of DRAM and HBM.
* DRAM's low latency is not relevant for AI inference where access patterns are deterministic and latency-tolerant.
* DRAM capacity scaling has stalled while AI inference needs are growing.
* High Bandwidth Flash (HBF) aims to meet the capacity, energy, throughput, and scalability requirements of advanced computing.
* HBF redefines NAND flash by optimizing for high bandwidth and inference memory characteristics using CMOS bonded array technology.
* HBF supports large KV caches for handling complex user prompts in AI inference.
* HBF retains data when power is lost and is thermally stable.
* The value of HBF is realized in edge and enterprise applications where HBM penalties are too high.
Executive Summary
Artificial intelligence is driving a massive data demand that is challenging current memory infrastructure, as data centers are expected to host AI workloads increasingly. Memory solutions like DRAM and HBM are struggling to meet the density, capacity, and scalability requirements for large AI models and inference workloads, while simultaneously facing rising production costs and energy consumption from hyperscale manufacturers. This situation creates a need for new memory architectures optimized specifically for AI inference, aiming for higher density, better bandwidth, lower power use, and cost-effectiveness compared to current systems.
High Bandwidth Flash (HBF) is presented as a disruptive architecture designed to address these needs by optimizing NAND flash for high bandwidth and inference memory characteristics. HBF offers higher capacity and density than HBM while maintaining comparable bandwidth, possesses the benefit of persistent storage qualities, and supports features like large KV caches. This capability extends beyond data centers to edge AI and enterprise applications where larger capacity is needed but HBM faces constraints due to cost and power penalties.
Full Take
The narrative establishes a tension between the established memory paradigm (DRAM/HBM) and emerging computational demands from AI, suggesting that current solutions create architectural bottlenecks rather than enabling progress. The core implication is that achieving scalable, energy-efficient AI compute requires a fundamental redesign of memory systems, moving away from application-agnostic metrics toward application-specific optimization.
The transition proposed—from DRAM/HBM to HBF—is framed as necessary for realizing memory-centric AI everywhere, particularly in latency-sensitive edge scenarios where HBM's limitations (density, cost) are prohibitive. This positions HBF not merely as an incremental storage upgrade but as a foundational shift toward memory designed around inference characteristics: density, bandwidth, persistence, and lower system power.
A crucial pattern to observe is the positioning of HBF against HBM: HBM excels in low-latency access for training contexts, while HBF targets the requirements of large-scale, high-throughput data handling inherent in inference and persistence. The argument skillfully leverages existing memory cost constraints to open the door for a new technology that solves performance bottlenecks across the entire AI stack. The missing piece of inquiry is whether this shift truly decouples system design from generalized performance metrics or if it simply relocates the optimization challenge to define what "optimized" means for heterogeneous workloads. What infrastructure investment will solidify this memory-centric architecture globally?