Checkpoint 2 · Chips & LLMs

Active Recall: Lessons 12–18 + Thesis Falsifiers

No notes. The new phase — memory, training, alignment, Blackwell, ASML, AMD, EDA — interleaved with the foundations, then the durable-judgment test: state every name's falsifier from memory.

Type: Retention checkpoint (Phase 2) Covers: L12–L18, interleaved with L1–L11 Capstone: all 7 thesis falsifiers

Same rules as Checkpoint 1. Produce the answer out loud or in writing before you reveal. The failed-retrieval struggle is the mechanism; reading first gives you fluency, not retention. New this time: Part C is a falsifier drill — for each of your seven names, state the specific signal that would change your mind. If you can produce the falsifiers cold, your THESIS.md has become real judgment rather than notes on a page.

Part A — Free Recall (L12–L18, interleaved)

Ten prompts, out of order. Answer before revealing; note every gap.

L17 · ASML

1. State the one equation that governs printing resolution, define each term, and name the three levers it gives you. Which lever did EUV pull, and by how much?

Model answer CD = k₁ · (λ / NA) — CD = smallest printable feature; λ = wavelength; NA = numerical aperture; k₁ = process factor (~0.25 floor). Three levers: shorter λ, bigger NA, lower k₁. EUV pulled wavelength: 193 nm → 13.5 nm, a ~14× reduction. (High-NA later pulls the NA lever: 0.33 → 0.55.)

L13 · TrainingL12 · HBM

2. Why does mixed-precision AdamW training cost ~16 bytes/param, and what does that imply for how many GPUs a 70B model needs just to hold training state?

Model answer 2 (BF16 weight) + 2 (BF16 grad) + 4 (FP32 master) + 4 (Adam m₁) + 4 (Adam m₂) = 16 bytes/param (before activations). 70B × 16 = ~1.1 TB → ~18–20 H100s (80 GB) before any activations. FP32 copies exist because tiny gradients vanish in BF16.

L16 · AMDL15 · Blackwell

3. AMD has matched NVIDIA on silicon and leads on HBM capacity. Name the two parts of the moat that live outside the die, and say which is "physics-rooted for training" vs. "eroding fastest at inference."

Model answer

Rack-scale interconnect (NVSwitch / NVLink). Physics-rooted, holds for training — TP all-reduces every layer at ~900 GB/s+; NVSwitch scales TP to 72 GPUs, AMD Infinity Fabric caps at ~8/node.
Software (CUDA vs ROCm). Erodes fastest at inference — most inference runs through PyTorch/Triton/vLLM abstractions, so ROCm can be "good enough" without matching CUDA kernel-for-kernel.

(Third layer — TSMC CoWoS/HBM — is a shared bottleneck, not a moat vs AMD.)

L12 · HBM

4. How does HBM physically achieve multiple TB/s when DDR5 can't, and what production factor (not design) gates supply?

Model answer Bandwidth = bus width × clock. HBM stacks DRAM dies vertically, linked by TSVs, giving a 1,024-bit bus per stack (vs 64-bit DDR), co-packaged next to the GPU on a CoWoS interposer (short wires). HBM4 → 2,048-bit. Gate = yield: stacking 8–12 known-good dies + TSVs compounds defects, and CoWoS interposer capacity at TSMC is itself constrained.

L18 · EDA

5. List the chip-design pipeline from RTL to fab in order (6 stages), and name the single deepest source of EDA switching costs.

Model answer RTL design → logic synthesis → verification → place & route → signoff (STA/power/DRC/LVS) → tape-out (GDSII to fab). Deepest switching cost: per-node PDK co-certification — each foundry node ships with incumbent tool support, so a challenger must co-engineer with every foundry for every node (chicken-and-egg). (Plus accumulated flows/IP/trained engineers.)

L15 · BlackwellL8 · Inference

6. What is "disaggregated prefill/decode," why does it exist, and what hardware feature makes KV-cache migration between the two pools practical?

Model answer Prefill (compute-bound, ~300 FLOPs/byte) and decode (memory-bandwidth-bound, ~2 FLOPs/byte) have opposite hardware appetites, so Blackwell serves them in separate GPU pools optimized for each. After prefill, the KV cache migrates to the decode pool. Practical because NVLink within the NVL72 rack at 1.8 TB/s moves a ~10 GB KV cache in ~5 ms; over InfiniBand (~50 GB/s) it'd take ~200 ms — impractical.

L16 · AMDL13 · Networking

7. What are UALink and Ultra Ethernet, who backs them, and why do they matter to the NVIDIA durability question?

Model answer Open, multi-vendor standards backed by hyperscalers (AMD, Broadcom, Google, Intel, MSFT, Meta): UALink = scale-up (GPU-to-GPU, rack-scale) vs NVLink/NVSwitch; Ultra Ethernet = scale-out (cluster) vs InfiniBand. If they mature and get adopted, NVIDIA's interconnect advantage becomes a buyable commodity, collapsing the contest back to silicon + software — the structural basis of the NVDA-durability bear case.

L14 · Alignment

8. RLHF → DPO → GRPO: models held in memory for each, what each eliminated, and why "GRPO is 10× cheaper" barely moves total GPU demand.

Model answer RLHF+PPO = 4 (policy, reference, reward model, critic); DPO = 2 (derives reward from policy log-probs, drops reward model + critic); GRPO = 2–3 (replaces critic with group statistics; with verifiable rewards drops the reward model too). Total impact is small because alignment is only ~1–5% of total training compute — a 10× cut there saves <1% overall. Real signal: capability access + more inference tokens from reasoning models.

L17 · ASMLL3 · China

9. Why is one EUV scanner the most precise instrument of chip-export policy that exists — and what does denying it actually do to China's leading edge?

Model answer ASML is the sole gate to leading-edge fabrication, so controlling it controls who can advance. EUV has never been sold to China (and advanced DUV immersion is now restricted too). Denial freezes China at the 193 nm multi-patterning wall: SMIC can reach ~7 nm but at poor yield/economics, and it does not scale to 3/2 nm — the single most effective non-tariff control in the stack.

L15 · BlackwellL4 · TSMC

10. The B200 is dual-die. Give the reticle-limit reason, the yield argument, and the constraint that actually gates shipment volume.

Model answer A single die with B200's compute (~1,600 mm²) exceeds TSMC's reticle limit (~850 mm²); two ~800 mm² dies (joined by NVLink-C2C at 10 TB/s, appearing as one GPU) have far better combined yield since defects scale with area. The shipment gate isn't die yield or HBM — it's CoWoS-L interposer capacity (~2,200 mm², 2 dies + 8 HBM stacks), made only by TSMC.

Part B — Interleaved Quiz (auto-graded)

Six questions, mixed topics

Choose the best answer.

1. AMD's HBM-capacity lead (192/288 GB) most directly helps which workload?

Thousand-GPU pretraining runs

Serving a large/long-context model in fewer GPUs

Tensor-parallel all-reduce latency

Porting CUDA kernels to ROCm

2. EUV uses mirrors, not lenses, because:

Mirrors are cheaper than lenses

13.5 nm light is absorbed by glass and air

Mirrors let the wafer stage move faster

Lenses can't be made precise enough for any litho

3. Which parallelism strategy is the physics basis for the NVLink/NVSwitch moat?

Data parallelism

Pipeline parallelism

Tensor parallelism (all-reduce every layer)

Expert parallelism

4. The custom-ASIC boom is, for EDA (Cadence/Synopsys):

A threat that bypasses their tools

A tailwind — more design teams, more seats and IP

Irrelevant; ASICs skip place-and-route

A margin-collapse risk

5. "GRPO is far cheaper than RLHF" barely affects total GPU demand because:

Alignment is only ~1–5% of total training compute

GRPO needs more GPUs than PPO

Inference demand is fixed regardless of models

Pretraining compute scales down with GRPO

6. Which item is a shared bottleneck for both NVIDIA and AMD (not a moat for either)?

NVSwitch rack-scale topology

The CUDA software ecosystem

TSMC CoWoS packaging and HBM supply

ROCm kernel tuning

Part C — The Falsifier Drill (durable judgment)

For each name, state from memory the specific, observable signal that would change your mind. This is the test that separates a thesis from a belief.

All 7 names — name the falsifier

Cover the right column. Produce each falsifier before revealing.

Model answers (from THESIS.md)

Name	Falsifier — the signal that flips the view
TSMC	Samsung/Intel Foundry wins a flagship leading-edge customer at volume with competitive yield; OR CoWoS stops being the binding packaging constraint.
NVIDIA	A qualified, rack-scale NVSwitch competitor adopted by a major hyperscaler; OR ROCm reaches CUDA parity (<5% gap) in production; OR UALink/UEC adoption + ASIC mix takes material share.
Broadcom	Major ASIC customers in-source design (cut Broadcom out); OR GPUs stay cheaper/more flexible than custom silicon for the marginal workload; OR Ethernet scale-out share stalls.
SK Hynix	Samsung achieves full HBM3e/HBM4 NVIDIA qualification at volume (3-way price competition); OR HBM supply outruns demand and ASPs roll over.
ASML	A credible second EUV source reaches leading-edge volume (Canon nanoimprint / SMEE EUV); OR scaling stalls and High-NA is deferred; OR a sustained down-cycle in net EUV bookings.
AMD	Confirm: qualified rack-scale Helios/UALink adopted at volume + ROCm near-parity. Break: fails to gain inference share despite competitive specs (moat holds).
Cadence/Synopsys	A credible third platform or open-source flow wins production tape-outs at an advanced node; OR a major customer cohort in-sources tooling; OR design starts structurally decline.

Synthesis — one dollar of hyperscaler AI capex (updated)

Now that ASML, AMD, and EDA are grounded, re-run the trace: where the dollar lands and who taxes it regardless of the socket winner.

Where the dollar flows

The accelerator dominates the bill. Largest slice to the GPU/system vendor — NVIDIA (margin), with AMD the funded second source. Within the GPU's BOM, the two biggest costs flow to TSMC (wafer + CoWoS) and SK Hynix/Micron (HBM).

Networking. Switches, NICs, optics for scale-out. Broadcom, Arista, NVIDIA (Mellanox) — and UALink/UEC tilt this toward merchant silicon over time.

Power, cooling, facility. A growing slice at 120 kW/rack. Vertiv, Eaton, utilities, data-center REITs.

Invisible upstream tax — paid no matter who wins the socket: every chip in that dollar was designed on Cadence/Synopsys tools and printed on ASML EUV. NVIDIA vs AMD vs ASIC doesn't change their cut.

Key insight: durability is highest where the cut is socket-agnostic — ASML, TSMC, and the EDA duo sit at the top of that spectrum; the accelerator winner can still change beneath them.

Part D — Score Yourself & What to Review

If you stumbled on…	Re-read	The one idea to re-anchor
Cards 1, 9 (ASML, China)	L17, L3	Resolution = wavelength; EUV denial freezes China at the 193 nm wall
Cards 2, 4 (training memory, HBM)	L13, L12	16 bytes/param; HBM bandwidth = wide TSV bus, gated by stacking yield
Cards 3, 7 (AMD moat, UALink)	L16	The moat lives outside the die; open standards attack the interconnect
Cards 5 (EDA)	L18	RTL→GDSII; PDK co-certification is the deepest switching cost
Cards 6, 10 (disagg, dual-die)	L15, L8	Prefill vs decode appetites; CoWoS-L is the shipment gate
Any falsifier in Part C	THESIS.md	A thesis you can't falsify is a belief — re-read that name's entry

Where you are. All 7 names grounded, two checkpoints done, both synthesis traces internalized. The remaining roadmap is the half of the picture that isn't chips-or-models: datacenter power & energy economics, transistor physics (the device-physics floor under TSMC), MoE & state-space models, and the inference-economics → valuation → portfolio capstone that turns all of this into position sizing.

From your instructor: Tell me which cards or falsifiers you missed and I'll build a short, spaced re-test of only those a few days out. Otherwise, pick the next thread — power economics is the highest-leverage gap (it's where the AI-capex bottleneck is actually moving), and transistor physics is the one piece of manufacturing first-principles still missing.