Does Shorter HBM Mean Less AI Memory Demand?

Rumors that future AI accelerators may use 8-high HBM instead of 12-high or 16-high stacks sound bearish for memory suppliers. Fewer DRAM dies in each stack should mean fewer HBM bits sold per accelerator.

It’s a possible conclusion but it can’t be based only on stack height.

A shorter stack can be a cost reduction, a response to poor packaging yield, a way to turn scarce dies into more complete stacks, or a trade of capacity for bandwidth.

What’s more important is how accelerator volume, memory capacity, bandwidth, manufacturing yield, and price move together.

The Two Competing Interpretations

The argument began with commentary in Irrational Analysis’s Hot Chips 2026: Irrational Recap. The post describes customers considering a move from 12-high HBM to 8-high or even 4-high products and interprets that as “de-specing.”

Its interpretation is bearish because:

1. AI accelerators created extraordinary HBM demand.
2. Limited supply gave memory vendors strong pricing power.
3. HBM became expensive enough for customers to redesign systems around using less of it.
4. Lower stack heights reduce the number of DRAM dies consumed per accelerator when stack count and die density are unchanged.
5. That efficiency eventually weakens bit scarcity and memory-vendor margins.

A second interpretation starts from an x.com user, the same rumored move but reaches a different conclusion. Perhaps HBM demand is not disappearing. Perhaps the constraint is shifting from the total number of DRAM bits toward the number of bits that qualify for high-speed operation.

Capacity and Bandwidth Are Different Products

A HBM stack supplies both storage capacity and memory bandwidth. Stack height mainly changes the capcacity.

Simple capacity equation:

capacity per stack = capacity per DRAM die × number of DRAM dies

An 8-high stack therefore contains half as many bits as a 16-high stack when both use the same die.

Bandwidth follows:

bandwidth per stack = (interface width × data rate per pin) ÷ 8

Adding DRAM layers increases capacity, but it does not automatically widen the external interface or double its transfer rate.

HBM4 for eg. Its interface grows from HBM3’s 1,024 bits to 2,048 bits. At 8 Gb/s per pin, giving about 2 TB/s per stack:

(2,048 bits × 8 Gb/s) ÷ 8 = 2,048 GB/s

An 8-high HBM4 stack can consequently deliver much more bandwidth than an older, taller stack while containing fewer bytes. A buyer optimizing a bandwidth-bound inference accelerator may prefer more bandwidth per die rather than maximum capacity per stack.

Why Shorter Stacks Improve Effective Supply

Assume a manufacturer has 1,600 known-good DRAM dies available. Ignoring base dies, spares, packaging losses, and product differences:

• 16-high products can populate 100 stacks.
• 12-high products can populate about 133 stacks.
• 8-high products can populate 200 stacks.

Moving from 16-high to 8-high doubles the number of potential stacks from the same die pool. If an accelerator keeps the same number of HBM sites, its DRAM-die consumption falls by half.

This is the strongest part of the bearish argument. Lower layers per accelerator release DRAM bits for other products.

But those extra stacks are only usseful if the system tolerates lower capacity. Offloading data to conventional DRAM or storage can reduce HBM capacity pressure, but it introduces bandwidth, latency, energy, and software costs. My long-context explainer covers why KV-cache demand grows with sequence length.

Yield Is More Complicated Than Multiplying Die Yield

Give each die a 98% chance of working and calculate:

0.98⁸ ≈ 85%, while 0.98¹⁶ ≈ 72%

It’s is not a super realistic model of finished-stack yield. HBM manufacturers test and select known-good dies before committing them to an expensive stack. A bad memory cell discovered at wafer test does not normally get chosen at random and buried in a 16-layer package.

Still, assembly risk is compounded. Each additional layer requires a bunch of consecutive processes, where a defect during a later step can strand dies and work already invested in the stack.

SemiAnalysis gives a more relevant simplified illustration in Scaling the Memory Wall: The Rise and Roadmap of HBM: if every layer-attachment step yields 99%, an 8-layer stack yields roughly 92%, while a 12-layer stack yields roughly 87%. The publication also stresses that real defects are not independent and that warpage or non-coplanarity can worsen as layers accumulate.

So taller stacks expose more scarce silicon to cumulative process and packaging risk.

Pin Speed Does Not Consume More Dies

The phrase that wafer consumption is moving from stack height to pin speed is not quite precise enough.

Doubling signaling speed does not require twice as many DRAM dies but higher speed can reduce qualified output through:

• tighter timing and electrical specifications
• more restrictive speed binning
• harder power-delivery and thermal limits
• more complex base-die and PHY design
• tougher package and interposer signal integrity
• yield loss when a nominally functional part misses the required performance bin.

A move to 8-high stacks can reduce aggregate DRAM-bit consumption while preserving—or increasing—the value of the fastest qualified dies and packages. Bit supply becomes easier at the same time that top-bin product supply remains constrained.

A Toy Generational Comparison

Consider two hypothetical accelerators with eight HBM stacks each:

Metric	Generation A	Generation B
Stack height	12-high	8-high
Capacity per die	3 GB	4 GB
Total HBM capacity	288 GB	256 GB
Per-pin data rate	8 Gb/s	16 Gb/s

Assuming the same interface width, Generation B has 11% less total capacity but twice the peak bandwidth. It also consumes 64 DRAM dies per accelerator instead of 96—a 33% reduction.

This product has changed the mix from maximum capacity toward bandwidth per die. Memory suppliers lose some bit content per accelerator, but may still sell a more demanding and expensive performance bin.

The reverse is possible. If capacity per accelerator falls sharply, stack count does not rise, accelerator shipments disappoint, and price per GB normalizes, then it will be likely bear scenario.

The Metrics That Matter

A rough measure of the HBM revenue pool is:

accelerator shipments × HBM GB per accelerator × average selling price per GB

Investors and system designers should also separately track:

• HBM stacks per accelerator
• DRAM dies and total GB per accelerator
• bandwidth per stack and per accelerator
• accelerator shipment volume
• front-end DRAM and TSV capacity
• known-good-die, speed-bin, and finished-stack yield
• advanced-packaging capacity and yield
• price and manufacturing cost per bit
• premiums for the fastest qualified products

A shorter stack is bearish for bits per package, all else equal. All else rarely remains equal across accelerator generations.

The Best Reading of the Rumor

The first interpretation correctly identifies customer response to high prices. Hardware companies do not passively accept an expensive bottleneck; they quantize models, shrink KV caches, disaggregate inference, offload colder data, and redesign packages. Sustained HBM margins create their own incentive for substitution and efficiency.

The second interpretation correctly separates capacity from bandwidth and highlights packaging yield. Fewer layers can turn a fixed pool of scarce dies into more usable stacks, reduce the value at risk in each assembly, and concentrate demand on higher-speed qualified output.

So overall:

• Bearish for bit intensity: fewer layers with unchanged stack count and die density consume fewer DRAM bits per accelerator.
• Not automatically bearish for HBM revenue: higher accelerator volume, denser dies, more stacks, faster performance bins, or stronger pricing can offset that reduction.
• Potentially bearish for specific packaging equipment: a durable move away from very tall stacks could delay technologies whose main value is enabling additional layers.
• Still evidence of customer pushback: even when bandwidth demand remains strong, redesigning around fewer layers shows that capacity is being rationed by price, supply, yield, or all three.

The rumor is evidence that the industry is optimizing a different objective function: useful bandwidth and capacity per scarce die, per package, and per dollar.

Sources and Further Reading

• Irrational Analysis: Hot Chips 2026: Irrational Recap
• Irrational Analysis: HBM: High-Bandwidth Mistake
• SemiAnalysis: Scaling the Memory Wall: The Rise and Roadmap of HBM
• EDN: JEDEC Finalizes HBM4 Standard