Why stacked DRAM creates a structural thermal resistance crisis

Vertical die stacking creates a thermal resistance problem that is qualitatively different from planar 2D memory: each die layer acts as a partial thermal insulator for the layers below it while simultaneously generating its own heat during read/write operations. The result is a progressive temperature gradient most severe at the bottom of the stack — which is also the highest-power-density location. As documented by Micron Technology in its 2025 patent disclosures, the HBM interface die — positioned at the bottom of the stack — exhibits the highest power density in the entire device due to its high operating speed, yet is simultaneously the die farthest from conventional cooling media.

In a 12-die HBM3 stack, the interface die must exhaust its heat through the entire column of overlying DRAM dies to reach the heat spreader at the top. This creates an additive inter-die thermal resistance that grows directly with stack height. Quantitative data from Huawei Technologies’ 2024 semiconductor device patent reinforces the severity: the junction temperature specification for HBM dies produced on DRAM processes is only 95°C — significantly below the 105°C permitted for peripheral CMOS logic. In a four-layer HBM2 stack, the local temperature differential between the hottest and coolest points on the bottom die alone can reach 24°C.

This hotspot non-uniformity is not merely a steady-state concern. It creates timing margin degradation and increased bit-error rates at elevated temperature. The structural source of this thermal resistance is identified in a 2025 patent from Beijing Superstring Memory Research Institute: each inter-die gap is bridged by microbump arrays filled with underfill polymer — a material with inherently low thermal conductivity. The cumulative serial thermal resistance from the bottom die to the package lid is therefore dominated by these polymer-filled interfaces and the silicon bulk of each intervening die, not by the interconnects themselves. Conventional lid-plus-heatsink cooling addresses only the top of the stack.

The adjacency of the HBM stack to high-power host processors — GPUs currently consuming up to 700 W and CPUs up to 400 W per chip, with projections above 1,000 W — further aggravates the HBM thermal environment, as described in a 2024 ND-HI Technologies Lab filing. The mutual heating between adjacent DRAM dies in the vertical stack, combined with lateral thermal coupling from the processor, makes passive single-sided cooling fundamentally inadequate for emerging stack heights. Standards bodies including JEDEC continue to track these constraints as part of HBM specification development.

Thermal TSVs, conductive layers, and integrated microfluidic cooling

The primary structural response to stack thermal resistance is to create dedicated thermal conduits that bypass the high-resistance underfill and silicon-die pathway — vertical structures running from the highest-power bottom region to a heat removal surface at the top of the stack. Multiple organisations have converged on variants of the same concept, with Micron Technology, Yangtze Memory Technologies, and Huawei Technologies each filing independent patent families on this approach between 2024 and 2025.

Micron Technology’s 2025 US patent on thermal dissipation in stacked memory devices describes an HBM device in which a thermally conductive layer is carried by a designated thermal region of the base interface die, and cooling TSVs extend from this layer upward through the entire memory die stack to an elevation at or above the top die surface. Critically, these thermal TSVs are electrically passive — they serve only as heat conduits — and are positioned within the same footprint as the signal routing region to avoid die area penalties. This creates a low-resistance axial heat path operating in parallel with the electrical TSV network.

Yangtze Memory Technologies has filed a closely related architecture (WO and CN, 2025) that deposits a thermally conductive layer on the top surface of the uppermost die and routes vertical “first channels” through all stacked dies, with each channel contacting the base die at one end and the top thermal layer at the other. This allows heat generated at any layer — including the base die — to travel upward through the dedicated channel rather than diffusing laterally and slowly through underfill. The approach is consistent with, and in some cases advances beyond, equivalent disclosures from established Western memory makers, representing a significant Chinese investment in HBM thermal architecture.

Huawei Technologies addresses the specific problem of in-plane thermal hotspots in its 2024 patent by introducing a thermally conductive layer at the bottom-die level to improve lateral heat spreading before axial conduction. The key insight is that existing approaches — higher-density dummy solder balls, hybrid bonding for increased contact area — reduce inter-layer thermal resistance but do not address the in-plane spreading resistance that creates local hotspots. A planar conductive layer at the base of the stack converts point-source hotspots into a more uniform heat flux before routing to the cooling path. Research published by IEEE on 3D IC thermal management corroborates the importance of lateral spreading layers in multi-die stacks.

“Existing approaches reduce inter-layer thermal resistance but do not address the in-plane spreading resistance that creates local hotspots — a planar conductive layer at the base of the stack converts point-source hotspots into a more uniform heat flux.”

The most aggressive thermal coupling concept is explored by Micron in its 2025 through-silicon trench cooling patents (US and WO): through-silicon trenches formed in two or more memory dies are fluidly coupled to a coolant supply, enabling direct liquid cooling of the die interior. These trench-based cooling systems can be interconnected via connector channels in intermediate dies, creating a fully integrated microfluidic cooling loop within the die stack itself — eliminating dependence on external heat spreaders entirely for mid-stack thermal management. This represents the frontier approach in the current patent corpus and anticipates the thermal demands of HBM4 stacks with 12 or more dies.

Packaging and system-level thermal management for HBM4

Die-level structural innovations alone are insufficient when the HBM stack is co-packaged with processors consuming hundreds of watts. Coordinated packaging architecture and multi-directional cooling are required, and the conventional single-sided topology — in which the heat spreader contacts only the top of the DRAM stack — is inadequate for deep stacks. Multiple organisations are developing multi-directional alternatives that address the thermal asymmetry inherent in top-only cooling.

ND-HI Technologies Lab proposes a fundamentally different packaging topology in its 2025 Taiwan and 2024 China filings. Their approach integrates high-thermal-conductivity interconnects placed between or alongside dies, a substrate with internal liquid flow cavities, and a cooling plate in direct thermal contact with the topmost die. A second internal cavity is hydraulically connected to the first, allowing liquid to circulate from bottom to top of the package. This achieves dual-sided heat removal: the bottom of the stack is cooled via the substrate liquid cavity, while the top is cooled via the plate — directly addressing the asymmetry inherent in top-only cooling. The approach directly anticipates the thermal demands of future HBM stacks co-packaged with 1,000+ W processors.

Shanghai Xianfang Semiconductor addresses the limitations of lid-plus-heatsink cooling in its 2021/2024 patents by developing a water-cooling system with three-dimensional honeycomb-shaped microchannels attached above the lid, combined with heat pipes that penetrate both the water-cooling system and the lid to reach the chip surface directly. This creates a three-dimensional heat flow network rather than the purely planar thermal conduction of standard solutions, improving cooling capacity for high-density 3D integrated structures.

Micron’s 2014 semiconductor die assembly patent establishes an early framework for the problem: in a conventional HBM package where the logic die’s high-power-density peripheral region extends beyond the DRAM stack footprint, the primary heat path from the logic die must traverse the entire DRAM stack to reach the lid, creating a 30°C Tmax increase. Their solution uses a stepped lid profile that contacts the logic die peripheral region directly with a first thermal interface, bypassing the DRAM stack entirely for a substantial fraction of the logic die’s heat output. The progression from this 2014 baseline to the 2025 microfluidic trench approaches illustrates a decade of escalating thermal engineering investment. Technical standards from JEDEC and packaging research published by imec provide the broader context for these packaging innovations.

Active thermal throttling: per-channel and per-cell-group control

Active thermal control provides real-time management of temperature gradients that structural solutions cannot fully eliminate. The industry has moved decisively from monolithic (whole-stack) throttling to fine-grained per-channel and per-cell-group control — a shift driven by the recognition that only a minority of cells typically exceed temperature limits, making stack-wide throttling both wasteful and performance-limiting.

TSMC’s Differentiated Dynamic Voltage and Frequency Scaling (DDVFS) patent family — filed across Taiwan and South Korea in 2023 and 2024 — introduces an architecture in which each memory cell group in the HBM has its own sensing unit that generates environmental signals corresponding to local transistor conditions. A DDVFS device independently adjusts transistor temperature-affecting (TTA) parameters for each group based on its own environmental signals, rather than applying uniform throttling across all cells. TSMC’s 2022 CN patent explicitly diagnoses the failure mode of monolithic DVFS: because typically only a minority of a given core die exceeds the allowed temperature threshold, single-die-granularity throttling unnecessarily degrades the performance of the majority of cells and of all other dies in the stack.

Intel’s 2023 per-channel thermal management patent further refines granularity to the channel level, enabling the memory controller to throttle row commands and column commands independently within individual channels based on per-channel temperature telemetry. Row command throttling and column command throttling can proceed at different rates, and throttle signals can be interleaved across channels and pseudo-channels to maintain aggregate bandwidth while reducing thermal peaks. This is particularly relevant for HBM4-generation interfaces where each stack exposes 16 or more pseudo-channels. Intel’s testing found that 60% of sampled parts exhibited bit errors when read from HBM stacks operating at elevated temperature — underscoring the reliability imperative that drives this fine-grained control architecture.

“Intel’s testing found that 60% of sampled parts exhibited bit errors when read from HBM stacks operating at elevated temperature — underscoring why per-channel thermal control is a reliability requirement, not merely a performance optimisation.”

Shanghai Biren Intelligent Technology’s 2023 patent implements hardware-level temperature protection using the HBM CATTRIP signal: when the HBM chip temperature exceeds a threshold and asserts CATTRIP, a hardware bus controller — rather than software — immediately severs the data interface between the host and the HBM die. The hardware response is faster than any software polling loop, preventing the thermal runaway scenario where software latency allows junction temperature to climb past the damage threshold. This hardware-first approach to CATTRIP handling represents a necessary complement to the fine-grained DDVFS approaches from TSMC and Intel.

Tsinghua University’s 2020 patent contributes a complementary approach: mapping data workloads to physical partitions based on each partition’s thermal characteristics, then optimising the DRAM refresh frequency per partition based on its operating temperature. Higher-temperature regions require more frequent refresh to counteract increased leakage; by detecting these regions explicitly, unnecessary refresh operations on cooler regions are eliminated, reducing both refresh power and the secondary heat generated by those refresh cycles. Research on 3D memory thermal management published by Nature Electronics has independently identified refresh-induced self-heating as a meaningful contributor to stack thermal budgets.

Key assignees and the shape of the patent landscape

The 40+ patent filings reviewed span jurisdictions across the US, China, South Korea, Taiwan, and PCT applications, filed between 2014 and 2026. Six assignees account for the dominant technical contributions, each with a distinct area of focus that reflects their position in the HBM supply chain.

Micron Technology is the single most prolific filer on HBM thermal architecture, with multiple independent patent families covering cooling TSV networks, through-silicon trench cooling, active TSV-integrated cooling networks for the base die, and test access for thermally managed stacks. Their work spans both structural innovations at the die level and system-level integration, representing the most comprehensive public disclosure of HBM4-generation thermal architecture to date.

TSMC has staked out a strong position in active thermal control, with its DDVFS family directly addressing the fine-grained thermal throttling gap identified in HBM3 and earlier. Their multi-jurisdiction filings across Taiwan and South Korea reflect the broad applicability of this approach to any foundry customer implementing HBM. Intel Corporation addresses the problem at both the packaging architecture level — inverted stacks, heat path optimisation — and the system level through traffic-aware thermal management and per-channel throttling, contributing an applications-layer perspective relevant to CPU and GPU platform integrators.

Yangtze Memory Technologies and Huawei Technologies both represent significant Chinese investment in HBM thermal architecture, with filings that tackle the in-die thermal spreading problem and full-stack vertical heat conduit design. Their approaches are closely aligned with, and in some cases advance beyond, equivalent disclosures from established Western memory makers. ND-HI Technologies Lab is notable for its multi-sided cooling topology, extending the thermal solution from the die stack itself to the package substrate and proposing liquid-based cooling integrated directly into the substrate — an approach that directly anticipates the thermal demands of future HBM stacks co-packaged with 1,000+ W processors.

Samsung Electronics contributes indirectly to thermal management through bandwidth-efficiency approaches that reduce unnecessary toggle activity and associated heat generation, as seen in its 2024 CN patent on high bandwidth memory systems. The PatSnap patent analytics platform provides the full cross-jurisdictional view of this landscape, enabling R&D teams to map white spaces and freedom-to-operate risks across all six assignees simultaneously. For a broader view of semiconductor packaging innovation trends, the PatSnap Insights blog tracks emerging IP clusters across advanced packaging, chiplets, and 3D integration.