AI Chip Memory Bandwidth 2026 — PatSnap Eureka
AI Chip Memory Bandwidth: Breaking the Memory Wall
Computational throughput in modern AI accelerators now far outpaces data supply rates. This landscape maps the patent and literature signals across compute-in-memory, 3D stacking, chiplet packaging, and DMA optimization — the four approaches reshaping AI chip design.
Why the Memory Wall Defines AI Chip Performance
The memory bandwidth challenge in AI chips arises from the fundamental mismatch between processing unit throughput and data supply rates — a tension documented across a dataset spanning 2017 to 2025. As patent landscape analysis consistently shows, this bottleneck — not raw compute — is the dominant performance and energy efficiency constraint in modern AI accelerators.
Two broad architectural philosophies dominate responses to the memory wall: bringing compute to memory — eliminating data movement by embedding arithmetic inside or adjacent to memory arrays (compute-in-memory, processing-in-memory, near-memory computing) — and increasing bandwidth density — using 3D stacking, chiplet packaging, high-bandwidth memory (HBM), and new interconnect topologies to widen the data pipe between processor and memory.
Sub-domains identified across retrieved records include: SRAM-based CIM, RRAM/ReRAM-based analog CIM, DRAM-based processing-in-memory, STT-MRAM and SOT-MRAM CIM, flash-based CIM, 3D-stacked memory (HMC, HBM), chiplet-based memory disaggregation, near-memory FPGA acceleration, photonic memory interconnects, and DMA bandwidth optimization for AI chips.
Google's foundational TPU paper established that on-chip memory capacity and utilization — not raw compute — is the dominant performance lever, noting a 28 MiB software-managed on-chip memory as central to the TPU design. This insight anchors the entire field's subsequent trajectory. Explore the full patent record on PatSnap Eureka.
Patent & Literature Signals at a Glance
Visualising the distribution of innovation across technology clusters and geographies based on PatSnap Eureka dataset records spanning 2014–2025.
Technology Cluster Distribution by Approach
SRAM-based CIM leads as the most extensively represented cluster, followed by resistive NVM CIM and 3D stacking.
Geographic Distribution of Innovation Records
The US leads with the broadest assignee base; China shows accelerating industry-driven filings from 2024.
Innovation Maturity by Era: 2014–2025
Four distinct eras from foundational research to active productization, with a clear inflection point in 2020 when RRAM, chiplet, and PIM-DRAM approaches proliferated simultaneously.
Four Clusters Reshaping AI Chip Memory Architecture
Each cluster addresses the memory wall through a distinct mechanism, from in-array arithmetic to physical die stacking. PatSnap's domain intelligence maps the overlap and white space across all four.
Compute-in-Memory (CIM) with SRAM
Standard 6T or augmented 8T SRAM arrays are modified to perform multiply-accumulate (MAC) operations in-place, eliminating off-chip data movement for weight-activation products. Key contributors include Purdue University (IMAC, X-SRAM), Illinois Institute of Technology (8T ultra-low power), and Rice University (CAP-RAM charge-domain MAC with programmable bit-width for CNN inference).
MAC operations in-array · No off-chip movementResistive Non-Volatile Memory CIM (RRAM/ReRAM, MRAM, PCM)
Non-volatile memories store AI weights in dense analog arrays and perform matrix-vector multiplication in-situ, potentially delivering orders-of-magnitude bandwidth efficiency gains over von Neumann architectures. Stanford University's fully integrated RRAM-CIM chip, Tsinghua's RRAM-CIM trend analysis, and National Taiwan University's SOT-MRAM CIM architecture using distributed arithmetic represent the frontier here.
Analog in-situ MVM · Non-volatile weight storage3D Stacking, Chiplet Packaging & Near-Memory Computing
Addresses bandwidth by physically reducing the distance between logic and memory dies through through-silicon vias (TSVs), 2.5D interposers, and chiplet disaggregation with advanced packaging. Carnegie Mellon's FPGA+HBM near-memory computing demonstrates large speedups on genome analysis and weather prediction. Arizona State's SIAM chiplet simulator benchmarks device, circuit, NoC, NoP, and DRAM access end-to-end. Esperanto Technologies' ET-SoC-1 with over 1,000 RISC-V processors targets ML recommendation workloads.
TSV · 2.5D/3D packaging · HBM integrationDMA Bandwidth Optimization & AI Chip Characterization
An emerging patented area focused on software and model-driven methods to measure, predict, and optimize DMA bandwidth utilization on deployed AI chips. Shanghai Enflame Technology Co., Ltd. (Suiyuan Intelligent Technology) holds two 2024 active CN patents using trained mathematical models with Maximum Likelihood Estimation and Maximum A Posteriori methods to infer DMA bandwidth under different data-transfer scenarios for diverse AI chip types.
MLE · MAP estimation · Production bandwidth managementWho Is Driving AI Chip Memory Bandwidth Innovation?
Innovation is widely distributed across academic institutions and startups rather than concentrated in a few large assignees, with Google and Intel as the most prominent industry players with directly cited works.
| Geography | Key Assignees | Primary Focus Area | Notable Signal |
|---|---|---|---|
| United States | Google, Stanford, Georgia Tech, MIT Lincoln Lab, Purdue, Rice, CMU, Auburn, UT Austin, Arizona State, Esperanto | TPU architecture, RRAM-CIM, HMC, SRAM-CIM, FPGA+HBM, PIM inference | Most heavily represented geography; broadest assignee base |
| China | Tsinghua University, Fudan University, Shanghai Sharetek, Beijing Jiaotong University, Shanghai Enflame (Suiyuan) | RRAM-CIM trends, non-volatile TCAM, chiplet PIM, photonic AI chips, DMA optimization | 2024 active CN patents from Enflame/Suiyuan — productization signal |
| Korea | Seoul National University, Kwangwoon University, SYNOPSYS Seoul, Sun Moon University | Tensor core memory scaling, M3D CIM SRAM, AI chip memory system, edge AI benchmarking | Growing academic output in GPU memory hierarchy |
| Europe | ETH Zurich, University of Bologna, CEA-LETI Grenoble, Inria/IRISA, University of Amsterdam, Politecnico di Milano | PIM-DRAM, analog IMA, wireless-on-chip AIMC, FPGA memory layout, IMC device overview | Strong academic depth; Bologna wireless-on-chip is unconventional frontier |
| Intel (EP) | Intel Corporation | Cross-point memory compute architecture for AI tensor operations | Major IP activity from established semiconductor incumbent — bellwether for industry-scale commitment |
Monitor Chinese Industry Participants in Real Time
Enflame/Suiyuan's 2024 active patents signal China is moving beyond research toward deployable bandwidth management IP.
Five Frontiers Identified in 2022–2025 Filings
Based on the most recent filings and publications in this dataset, five emerging directions are identifiable — ranging from near-term productization to 5+ year horizon bets.
Model-Driven DMA Bandwidth Optimization
The two 2024 active CN patents from Shanghai Enflame Technology Co., Ltd. represent a new class of AI-chip-specific bandwidth characterization technology, using trained mathematical models with Maximum Likelihood Estimation and Maximum A Posteriori methods to predict and optimize DMA performance — a clear productization signal.
Analog In-Memory Computing at Scale
Applied Materials' fully integrated SoC with scalable RRAM tiles and Stanford's RRAM-CIM chip push toward commercial readiness for analog CIM. These represent the transition from single-macro demonstrations to full system-level integration targeting production AI workloads.
Wireless On-Chip Communication for AIMC Tiles
The University of Bologna's proposal to use on-chip wireless interconnects to supply data at sufficient bandwidth to analog CIM tiles represents an unconventional approach to the on-chip bandwidth bottleneck — addressing the data supply problem without physical wires between tiles.
Photonic Memory Interconnects
Nokia's optically connected memory architecture for disaggregated data centers and photonic AI chip research from Beijing Jiaotong University signal photonics as a longer-horizon bandwidth solution. Technology investors should treat these as option positions requiring 5+ year horizons, while monitoring Intel's cross-point memory CIM patent activity as a bellwether.
What This Landscape Means for R&D and IP Strategy
The memory wall is the dominant AI chip design constraint. Across this entire dataset, from Google's 2017 TPU to 2024 DMA patents, the consistent message is that memory bandwidth — not compute — limits AI chip performance and energy efficiency. R&D investment strategies must weight memory architecture at least equally with processing unit design. Explore the PatSnap analytics platform for patent landscape mapping.
CIM is moving from academic research to silicon. Multiple fully integrated CIM chips are now demonstrated — Stanford RRAM-CIM, Applied Materials ReRAM SoC, China Electronics Technology Group RRAM core. IP strategists should map the growing patent thicket in SRAM-CIM (6T/8T variants), RRAM crossbar, and MRAM CIM, where overlapping claims from U.S., Chinese, and European institutions are accumulating rapidly. The PatSnap Trust Center outlines how IP data is secured and managed.
Chiplet and 3D packaging is the near-term commercial pathway. For teams unable to wait for analog CIM maturation, chiplet-based disaggregation (2.5D/3D) and HBM integration offer a proven bandwidth scaling path, as validated by SIAM simulator work and commercial designs like Esperanto's ET-SoC-1. Advanced packaging IP is a key competitive moat. See how PatSnap customers use these insights for competitive advantage.
Chinese industry players are accelerating productization. The 2024 active patents from Enflame/Suiyuan Intelligent Technology targeting DMA bandwidth optimization, combined with academic output from Tsinghua and Fudan, indicate that Chinese industry is moving beyond research toward deployable bandwidth management IP — a signal for competitive monitoring. The IEEE and WIPO provide additional context on global IP trends in semiconductor technology.
AI Chip Memory Bandwidth — key questions answered
The memory bandwidth bottleneck — the so-called "memory wall" — has emerged as the defining constraint on AI chip performance, as computational throughput in modern accelerators now far outpaces the rate at which data can be supplied from memory.
Two broad architectural philosophies dominate responses to the memory wall: (1) Bringing compute to memory — eliminating data movement by embedding arithmetic inside or adjacent to memory arrays (compute-in-memory, processing-in-memory, near-memory computing); and (2) Increasing bandwidth density — using 3D stacking, chiplet packaging, high-bandwidth memory (HBM), and new interconnect topologies to widen the data pipe between processor and memory.
Compute-in-Memory (CIM) with SRAM is the most extensively represented approach in this dataset. Standard 6T or augmented 8T SRAM arrays are modified to perform multiply-accumulate (MAC) operations in-place, eliminating off-chip data movement for weight-activation products.
Google's foundational TPU paper established that on-chip memory capacity and utilization — not raw compute — is the dominant performance lever, noting a 28 MiB software-managed on-chip memory as central to the TPU design.
The United States is the most heavily represented geography, with key assignees including Google, Stanford University, Georgia Institute of Technology, MIT Lincoln Laboratory, Purdue University, and Esperanto Technologies. China has a growing presence with two 2024 active CN patents from Shanghai Enflame Technology Co., Ltd. (Suiyuan Intelligent Technology) on DMA bandwidth optimization — among the most recent filings in this dataset.
Based on the most recent filings and publications (2022–2025), five emerging directions are identifiable: (1) Model-Driven DMA Bandwidth Optimization; (2) Analog In-Memory Computing at Scale; (3) Wireless On-Chip Communication for AIMC Tiles; (4) Photonic Memory Interconnects and Photonic AI Chips; and (5) ROM-Based CIM for Large-Scale Neural Networks.
Still have questions? Let PatSnap Eureka answer them for you.
Ask PatSnap Eureka Your AI Chip QuestionMap the AI Chip Memory Bandwidth Patent Landscape — Before Your Competitors Do
Join 18,000+ innovators already using PatSnap Eureka to accelerate their R&D and IP strategy across semiconductor and AI chip domains.
References
- Breaking the Memory Wall for AI Chip with a New Dimension — IC League Inc., China, 2020
- In-Datacenter Performance Analysis of a Tensor Processing Unit — Google Inc., USA, 2017
- A compute-in-memory chip based on resistive random-access memory — Stanford University, USA, 2022
- Trends and challenges in the circuit and macro of RRAM-based computing-in-memory systems — Tsinghua University, China, 2022
- SIAM: Chiplet-based Scalable In-Memory Acceleration with Mesh for Deep Neural Networks — Arizona State University, USA, 2021
- Using Chiplet Encapsulation Technology to Achieve Processing-in-Memory Functions — Shanghai Sharetek Technology Co., Ltd., China, 2022
- DMA Bandwidth Determination Method, Device, and Medium Based on AI Chip (April 2024) — Shanghai Enflame Technology Co., Ltd. (Suiyuan Intelligent Technology), CN, 2024
- DMA Bandwidth Determination Method, Device, and Medium Based on AI Chip (May 2024) — Shanghai Enflame Technology Co., Ltd. (Suiyuan Intelligent Technology), CN, 2024
- Technologies for providing high efficiency compute architecture on cross point memory for artificial intelligence operations — Intel Corporation, EP, 2022
- Future Scaling of Memory Hierarchy for Tensor Cores and Eliminating Redundant Shared Memory Traffic Using Inter-Warp Multicasting — Seoul National University, Korea, 2022
- Accelerating Neural Network Inference With Processing-in-DRAM: From the Edge to the Cloud — ETH Zurich, Switzerland, 2022
- IMAC: In-Memory Multi-Bit Multiplication and ACcumulation in 6T SRAM Array — Purdue University, USA, 2020
- A Novel Ultra-Low Power 8T SRAM-Based Compute-in-Memory Design for Binary Neural Networks — Illinois Institute of Technology, USA, 2021
- CAP-RAM: A Charge-Domain In-Memory Computing 6T-SRAM for Accurate and Precision-Programmable CNN Inference — Rice University, USA, 2021
- Designing Efficient and High-Performance AI Accelerators With Customized STT-MRAM — Auburn University, USA, 2021
- In-Memory Computing Architecture for a Convolutional Neural Network Based on Spin Orbit Torque MRAM — National Taiwan University, Taiwan, 2022
- FPGA-Based Near-Memory Acceleration of Modern Data-Intensive Applications — Carnegie Mellon University, USA, 2021
- Accelerating bandwidth-bound deep learning inference with main-memory accelerators — University of Texas at Austin, USA, 2021
- Scale up your In-Memory Accelerator: Leveraging Wireless-on-Chip Communication for AIMC-based CNN Inference — University of Bologna, Italy, 2022
- eBrainII: a 3 kW Realtime Custom 3D DRAM Integrated ASIC Implementation of a Biologically Plausible Model of a Human Scale Cortex — KTH Royal Institute of Technology, Sweden, 2020
- Optically Connected Memory for Disaggregated Data Centers — Nokia, 2020
- Demystifying the characteristics of 3D-stacked memories: A case study for Hybrid Memory Cube — Georgia Institute of Technology, USA, 2017
- X-SRAM: Enabling In-Memory Boolean Computations in CMOS Static Random Access Memories — Purdue University, USA, 2018
- AI and ML Accelerator Survey and Trends — MIT Lincoln Laboratory, USA, 2022
- Accelerating ML Recommendation With Over 1,000 RISC-V/Tensor Processors on Esperanto's ET-SoC-1 Chip — Esperanto Technologies Inc., USA, 2022
- Deploy large-scale neural network by ROM-based computing-in-memory using residual branch on a chip — Tsinghua University, China, 2022
- A Fully Integrated System-on-Chip Design with Scalable Resistive Random-Access Memory Tile Design for Analog in-Memory Computing — Applied Materials Inc., USA, 2022
- Key Technologies of Photonic Artificial Intelligence Chip Structure and Algorithm — Beijing Jiaotong University, China, 2021
- IEEE — Institute of Electrical and Electronics Engineers (contextual reference for semiconductor IP standards)
- WIPO — World Intellectual Property Organization (contextual reference for global patent filing trends)
All data and statistics on this page are sourced from the references above and from PatSnap's proprietary innovation intelligence platform. This landscape is derived from a limited set of patent and literature records retrieved across targeted searches and represents a snapshot of innovation signals within this dataset only.
PatSnap Eureka searches patents and research to answer instantly.