Why stacked DRAM has a structural thermal resistance problem

Vertical die stacking in HBM creates a thermal resistance problem that is qualitatively different from planar 2D memory: each DRAM layer acts as a partial insulator for the layers below it, while simultaneously generating heat during read/write operations. The result is a progressive temperature gradient that is most severe at the bottom of the stack — which is also the highest-power-density location in the entire device.

As documented by Micron Technology (2025), the HBM interface die — positioned at the bottom of the stack — exhibits the highest power density in the entire device due to its high operating speed, yet is simultaneously the die furthest from conventional cooling media. In a 12-die HBM3 stack, the interface die must exhaust its heat through the entire column of overlying DRAM dies to reach the heat spreader at the top, creating additive inter-die thermal resistance that grows with stack height.

The junction temperature specification for HBM dies produced on DRAM processes is only 95°C — significantly below the 105°C permitted for peripheral CMOS logic, as noted in Huawei’s 2024 disclosure. This hotspot non-uniformity is not merely a steady-state issue; it creates timing margin degradation and increased bit-error rates at elevated temperature. Intel’s testing found that 60% of sampled parts exhibited bit errors when read from HBM stacks operating at elevated temperature.

The structural source of this thermal resistance is identified in filings from Beijing Superstring Memory Research Institute (2025): each inter-die gap in the stack is bridged by microbump arrays filled with underfill polymer, a material with inherently low thermal conductivity. The cumulative serial thermal resistance from the bottom die to the package lid is therefore dominated by these polymer-filled interfaces and the silicon bulk of each intervening die — not by the interconnects themselves. Conventional lid-plus-heatsink cooling, which addresses only the top of the stack, is therefore structurally mismatched to where the heat is generated.

The problem is further compounded by the proximity of HBM stacks to high-power host processors. According to ND-HI Technologies Lab (2024), GPUs currently consume up to 700 W and CPUs up to 400 W per chip, with projections above 1,000 W. Lateral thermal coupling from the processor combines with vertical heat accumulation within the stack, making passive single-sided cooling fundamentally inadequate for emerging stack heights — a conclusion reinforced by standards bodies including JEDEC, which governs the HBM specification.

Thermal TSVs, conductive layers, and integrated microfluidic cooling

The primary structural response to stack thermal resistance is to create dedicated thermal conduits that bypass the high-resistance underfill and silicon-die pathway. Multiple organisations have converged on thermally conductive structures running vertically through the die stack from the highest-power bottom region to a heat removal surface at or above the top die.

Micron Technology has the most detailed public disclosures on this approach. In its 2025 US patent, Micron describes an HBM device in which a thermally conductive layer is carried by a designated thermal region of the base interface die, and cooling TSVs extend from this layer upward through the entire memory die stack to an elevation at or above the top die surface. Critically, the thermal TSVs are electrically passive — they serve only as heat conduits — and are positioned within the same footprint as the signal routing region to avoid die area penalties.

Huawei Technologies addresses the specific problem of in-plane thermal hotspots (2024) by introducing a thermally conductive layer at the bottom-die level to improve lateral heat spreading before axial conduction. The key insight is that existing approaches — higher-density dummy solder balls, hybrid bonding for increased contact area — reduce inter-layer thermal resistance but do not address the in-plane spreading resistance that creates local hotspots. A planar conductive layer at the base of the stack converts point-source hotspots into a more uniform heat flux before routing to the cooling path.

“The cumulative serial thermal resistance from the bottom die to the package lid is dominated by polymer-filled inter-die interfaces and silicon bulk — not by the interconnects themselves. Conventional lid-plus-heatsink cooling addresses only the top of the stack.”

The most aggressive thermal coupling concept comes from Micron’s 2025 US and PCT filings on through-silicon trench cooling. Through-silicon trenches formed in two or more memory dies of the stack are fluidly coupled to a coolant supply, enabling direct liquid cooling of the die interior. These trench-based cooling systems are interconnected via connector channels in intermediate dies, creating a fully integrated microfluidic cooling loop within the die stack itself — eliminating dependence on external heat spreaders for mid-stack thermal management. This approach, if realised at volume, would represent the most fundamental departure from conventional semiconductor packaging thermal architecture, a direction also tracked by organisations such as IEEE in its advanced packaging roadmap work.

Packaging and system-level thermal management for HBM stacks

Die-level structural innovations alone cannot resolve HBM thermal resistance when host processors exceed 1,000 W. Coordinated packaging architecture and active control mechanisms are required, and the conventional single-sided cooling topology — in which the heat spreader contacts only the top of the DRAM stack — is now acknowledged as inadequate for deep stacks by multiple organisations.

ND-HI Technologies Lab (filing as 钰创科技) proposes a fundamentally different packaging topology in its 2025 Taiwan and 2024 China filings. Their approach integrates high-thermal-conductivity (HTC) interconnects placed between or alongside dies, a substrate with internal liquid flow cavities, and a cooling plate in direct thermal contact with the topmost die. A second internal cavity is hydraulically connected to the first, allowing liquid to circulate from the bottom to the top of the package. This achieves dual-sided heat removal: the bottom of the stack is cooled via the substrate liquid cavity, while the top is cooled via the plate — directly addressing the asymmetry inherent in top-only cooling.

Shanghai Xianfang Semiconductor addresses the limitations of lid-plus-heatsink cooling by developing a water-cooling system with three-dimensional honeycomb-shaped microchannels attached above the lid, combined with heat pipes that penetrate both the water-cooling system and the lid to reach the chip surface directly. This creates a three-dimensional heat flow network rather than the purely planar thermal conduction of standard solutions, improving cooling capacity for high-density 3D integrated structures.

Intel Corporation addresses the inverted stack thermal problem in its 2023 China filing, noting that in a traditional HBM arrangement the base die — where most power is generated — is the die farthest from the thermal interface material (TIM) and heat spreader. Intel’s inverted stack concept positions the highest-power logic die closest to the external cooling surface, reducing the thermal path length for the dominant heat source. In a conventional package where the logic die’s high-power-density peripheral region extends beyond the DRAM stack footprint, the primary heat path must traverse the entire DRAM stack to reach the lid, creating a 30°C Tmax increase; Intel’s stepped lid profile contacts the logic die peripheral region directly with a first thermal interface, bypassing the DRAM stack entirely for a substantial fraction of the logic die’s heat output.

Intel also discloses active thermal management at the package level (2021), implementing an intelligent crossbar that routes memory access commands based on per-die temperature information. By redistributing traffic away from thermally stressed dies, this approach prevents the accumulation of thermal energy that would otherwise require hard throttling — a software-hardware co-design approach that organisations such as JEDEC and Semiconductor Engineering have identified as increasingly important for AI accelerator memory subsystems.

Active thermal throttling: per-die and per-channel granularity

Active thermal control methods provide real-time management of temperature gradients that structural solutions cannot fully eliminate. The industry has moved from monolithic whole-stack throttling to fine-grained per-channel and per-cell-group control — a shift driven by the recognition that coarse throttling wastes performance unnecessarily.

TSMC’s multi-jurisdiction patent family on Differentiated Dynamic Voltage and Frequency Scaling (DDVFS), filed across Taiwan and South Korea (2023–2024), introduces an architecture in which each memory cell group in the HBM has its own sensing unit that generates environmental signals corresponding to local transistor conditions. A DDVFS device independently adjusts transistor temperature-affecting (TTA) parameters for each group based on its own environmental signals, rather than applying uniform throttling across all cells. This prevents the scenario — identified as both wasteful and performance-limiting — where a small fraction of the die is overtemperature, but the entire stack is throttled.

TSMC’s 2022 China filing explicitly diagnoses the failure mode of monolithic DVFS: because typically only a minority of a given core die exceeds the allowed temperature threshold, single-die-granularity throttling unnecessarily degrades the performance of the majority of cells and of all other dies in the stack. The per-group DDVFS approach surgically reduces performance only in the affected region.

Intel’s 2023 China patent on per-channel thermal management further refines granularity to the channel level, enabling the memory controller to throttle row commands and column commands independently within individual channels based on per-channel temperature telemetry. Row command throttling and column command throttling can proceed at different rates, and throttle signals can be interleaved across channels and pseudo-channels to maintain aggregate bandwidth while reducing thermal peaks. This is particularly relevant for HBM4-generation interfaces where each stack exposes 16 or more pseudo-channels.

Shanghai Biren Intelligent Technology’s 2023 patent implements hardware-level temperature protection using the HBM CATTRIP signal: when the HBM chip temperature exceeds a threshold and asserts CATTRIP, a hardware bus controller — rather than software — immediately severs the data interface between the host and the HBM die. The hardware response is faster than any software polling loop, preventing the thermal runaway scenario where software latency allows junction temperature to climb past the damage threshold.

Tsinghua University (2020) contributes a complementary approach: mapping data workloads to physical partitions based on each partition’s thermal characteristics, then optimising the DRAM refresh frequency per partition based on its operating temperature. Higher-temperature regions require more frequent refresh to counteract increased leakage; by detecting these regions explicitly, unnecessary refresh operations on cooler regions are eliminated, reducing both refresh power and the secondary heat generated by those refresh cycles. This workload-aware approach is increasingly relevant as AI training and inference workloads create highly non-uniform memory access patterns — a challenge also documented by WIPO in its annual Technology Trends reports on semiconductor innovation.

Key players and innovation trends across 40+ patents

The patent corpus reviewed spans more than 40 documents filed between 2014 and 2026, with jurisdiction across the US, China, South Korea, Taiwan, and PCT applications. The dominant assignees addressing thermal challenges in stacked DRAM are Micron Technology, Samsung Electronics, Intel Corporation, TSMC, Yangtze Memory Technologies, Huawei Technologies, and ND-HI Technologies Lab.

Micron Technology

Micron is the single most prolific filer on HBM thermal architecture, with multiple independent patent families covering cooling TSV networks, through-silicon trench cooling, active TSV-integrated cooling networks for the base die, and test access for thermally managed stacks. Their work spans both structural innovations at the die level and system-level integration, and represents the most comprehensive public disclosure of HBM4-generation thermal architecture in the corpus reviewed.

TSMC

TSMC has staked out a strong position in active thermal control, with its DDVFS family directly addressing the fine-grained thermal throttling gap identified in HBM3 and earlier. Their multi-jurisdiction filings (TW, KR) reflect the broad applicability of this approach to any foundry customer implementing HBM. TSMC’s 2022 China filing provides the clearest articulation of why monolithic throttling is insufficient: only a minority of cells typically exceed the temperature threshold, yet coarse throttling degrades the entire stack.

Intel Corporation

Intel addresses the problem at both the packaging architecture level — inverted stacks, stepped lid profiles, heat path optimisation — and the system level, with traffic-aware thermal management and per-channel throttling. Intel’s finding that 60% of sampled parts exhibited bit errors when read from HBM stacks operating at elevated temperature provides the most direct quantification of the reliability imperative in the corpus.

Yangtze Memory Technologies and Huawei Technologies

Both organisations represent significant investment in HBM thermal architecture from Chinese semiconductor companies, with filings that tackle the in-die thermal spreading problem and full-stack vertical heat conduit design. Yangtze Memory’s 2025 PCT and China filings are closely aligned with — and in some cases advance beyond — equivalent disclosures from established Western memory makers. Huawei’s 2024 filing provides the most detailed quantitative data on in-plane hotspot magnitude in the corpus: a 24°C differential on the bottom die of a four-layer HBM2 stack.

ND-HI Technologies Lab (钰创科技)

ND-HI Technologies Lab is notable for its multi-sided cooling topology, extending the thermal solution from the die stack itself to the package substrate, and proposing liquid-based cooling integrated directly into the substrate. This approach directly anticipates the thermal demands of future HBM stacks co-packaged with processors exceeding 1,000 W — a threshold that makes substrate-integrated liquid cooling a system necessity rather than an optional enhancement. The company’s IP intelligence footprint in this space is tracked on PatSnap’s platform.

Samsung Electronics

Samsung is the leading filer in bandwidth architecture and scale-out HBM systems, contributing indirectly to thermal management through bandwidth-efficiency approaches that reduce unnecessary toggle activity and associated heat generation. Samsung’s 2024 China filing on High Bandwidth Memory architecture addresses sparse data handling and bandwidth optimisation in ways that reduce the thermal load on the stack.

For teams tracking competitive IP in advanced memory packaging, PatSnap’s patent search tools provide full coverage of the filing activity described in this article across all jurisdictions.