Chips on chips
An AI accelerator is a package of chips: a compute die, stacks of memory dies beside it, and a slab of silicon wiring under both. This page takes one apart, down to the copper that runs through a die.
Layer by layer.
Slide the package apart. On top sit the compute die and the HBM stacks. Microbumps join them to the silicon interposer, which wires them to each other. Bumps under the interposer join it to the package substrate, and solder balls join the substrate to the circuit board.
Amkor, which assembles such packages, names the same three layers: the dies on microbumps, the interposer with vias through it, and the substrate. Ibiden, which makes substrates, gives the substrate's job: wiring measured in nanometres on a chip does not connect directly to wiring measured in micrometres on a circuit board.
64 wires, or 2,048.
An AI chip works only as fast as its data arrives from memory. Micron, at Hot Chips 2026, called memory a key factor limiting AI system performance. Think of memory as a motorway. A memory module on a circuit board has 64 lanes. One HBM stack has 2,048. The cars drive at about the same speed. The extra lanes are why the stack moves 40 times as much data. But a circuit board cannot carry 2,048 lanes to a chip: its wires are too wide and too far apart. So the memory moves into the chip's own package, onto silicon wiring at least 10 times as dense, less than a tenth of a millimetre from the processor. That is HBM: memory stacked as close to the processor as it can go, on as many lanes as fit, so the chip waits less for its data.
A DDR5 module on a circuit board carries 64 data bits, as two independent sub-channels of 32. At 6,400 million transfers a second that is 51.2 GB/s. One HBM4 stack carries 2,048 data bits at up to 8 Gb/s each: 2,048 GB/s, 40 times the module.
Amkor writes that HBM2's roughly 4,000 connections need a very high connection density; it joins the stack and the compute die directly to a silicon interposer, under 100 µm apart. On one research chip from ETH Zurich and the University of Bologna, every memory wire is under 4.9 mm long.
The makers state faster stacks than the standard's 8 Gb/s: Micron more than 2.0 TB/s a stack, Samsung 11.7 Gb/s a pin and up to 3.3 TB/s a stack, SK hynix over 10 Gb/s a pin.
Up to 16 dies in 775 µm.
An HBM stack is DRAM dies on a base die, wired straight down through the silicon. JEDEC's HBM4 standard allows 4, 8, 12 or 16 dies of 24 or 32 Gb each, up to 64 GB a stack. Micron has sampled and Samsung ships 12-high HBM4 of up to 36 GB; Samsung plans 16-high at 48 GB, on a base die made in a 4 nm logic process. TSMC offers base dies in its N12 and N3 processes.
The whole stack has to fit in 775 µm, up from 720 µm through HBM3E. SK hynix gives the reason, as Tom's Hardware reports: the logic wafer beside it is 775 µm thick, and a taller stack would stand proud of the processor. A 300 mm wafer is sold 775 µm thick; an ID-1 card is 760 µm.
So each die is thin: 30 to 50 µm, Brewer Science told Semiconductor Engineering, about a third to three-fifths of a hair's width. SK hynix fills the gaps between its dies with a liquid it then hardens, a process it calls MR-MUF. In its 12-high HBM3E, Samsung lays a non-conductive film between the dies and holds them 7 µm apart.
Copper through the chip.
To stack dies, each needs wires straight through its silicon: through-silicon vias. Nvidia wrote in 2016 that one HBM2 die holds over 5,000 of them. The via drawn here is imec's reference, 5 µm across and 50 µm deep, to scale: 10 times deeper than it is wide.
A finished wafer: circuits and wiring on top, 775 µm of silicon below.
Etch: a deep silicon etch, usually the Bosch process, cuts the hole.
Line: an insulator, silicon oxide or nitride, then a barrier, which stops copper atoms diffusing out, and a copper seed coat the walls.
Fill: copper is plated into the hole, and polishing removes what is left on top. The via still ends inside the silicon.
Thin: bonded face down to a support, the wafer is ground from the back and its stress relieved, until the copper shows. Drawn here, the die is now 50 µm thick, the via's depth.
Bump: a copper pillar with a tin-silver cap is plated on the copper, to join this die to the next.
DISCO, which makes the grinders, has tested silicon thinned to 10 µm or less.
Solder, or copper on copper.
The dies in HBM4 are joined by microbumps; Lam Research's plating tool makes them as copper pillars with a cap of tin-silver solder. Their pitch, centre to centre, has been around 40 µm, and imec expects bonding with solder to stall at a pitch of 10 to 5 µm.
Hybrid bonding drops the solder. Two polished faces are placed together and heated; copper pads set under 2.5 nm below the surface swell into contact and join. JEDEC's higher limit left room for microbumps in HBM4, and SK hynix does not expect hybrid bonding in its HBM before HBM5.
40 µm: microbumps as the industry has long made them, per Semiconductor Engineering and imec.
30 µm: where imec says production microbumps settled (2021), and SK hynix's HBM bumps today, from its Hot Chips 2026 slides as Tom's Hardware reports.
10 µm: HBM4's pad pitch, per Semiconductor Engineering, and where imec expects solder to stall.
9 µm: TSMC's SoIC hybrid bonding in 2023, per Tom's Hardware, and Intel's first Foveros Direct, per Intel.
6 µm: TSMC's SoIC in 2025, per Tom's Hardware.
4.5 µm: TSMC's SoIC, planned for 2029, per Tom's Hardware.
imec has shown hybrid bonding at 2 µm from die to wafer, and at 200 nm from wafer to wafer.
Bigger than one exposure.
A lithography scanner prints at most one field, 26 by 33 mm: 858 mm². An interposer larger than that is stitched from several fields, each within the limit. TSMC calls its version CoWoS, in volume since 2012, and counts its size in reticles.
In 2016 Nvidia's P100 stood 4 HBM2 stacks beside one GPU in a 55 mm package. TSMC's CoWoS-S reaches 3.3 reticles, about 2,700 mm². TSMC said in April 2025 that it planned 9.5 reticles with 12 stacks or more for 2027. In April 2026 it said it was making 5.5 reticles and planned 14, with about 10 compute dies and 20 stacks, for 2028. The ID-1 card beside it covers 4,621 mm², about a 5.5-reticle interposer.
Above 3.3 reticles TSMC recommends CoWoS-L or CoWoS-R; CoWoS-L sets small silicon bridges in a wiring layer in place of one silicon plate. Intel's EMIB joins large chiplets without a silicon interposer. In October 2025 TSMC's chairman said the company was working hard to narrow the gap between CoWoS demand and supply.
Who makes the layers.
Two accelerators, as their makers describe them, in alphabetical order. AMD's MI355X is 8 vertically stacked compute dies and 2 I/O dies, 185 billion transistors, with 8 stacks of 12-high HBM3E: 288 GB and up to 8 TB/s, peak theoretical. Nvidia's Blackwell GPUs are 2 reticle-limited dies with 208 billion transistors, made in a TSMC 4NP process.
Makers of what this page shows, with pages on this record. The three whose HBM4 this page cites: Micron, Samsung Electronics and SK hynix. The two chipmakers whose packaging it cites: Intel and TSMC. Every company the record files under assembly, packaging and test: Amkor, ASE, Chipbond, ChipMOS, Hana Micron, JCET, King Yuan, Powertech, TongFu and Unisem. Every maker of package substrates it lists: AT&S, DNP, Ibiden, Kyocera, Nan Ya PCB, Shinko Electric, Toppan and Unimicron. Every equipment maker whose record description names packaging: BESI, Camtek, Disco, Hanmi Semiconductor, Kulicke & Soffa and Onto Innovation. Research: imec.
- Writing with lightwhy one field is 26 by 33 mm03
- Wrapping the gatethe transistors inside the compute die06
- Sculpting in atomsetch and deposition, from the chip up05
- Anatomy of a fabthe building the wafers go through02