No notes. The newest phase — transistor physics, power economics, inference $/token, the prefill/decode engine, and MoE/SSM architectures — interleaved with the foundations, then an integrative judgment drill: turn an architecture announcement into a hardware-demand call.
Same rules as Checkpoint 1 & Checkpoint 2. Produce each answer out loud or in writing before you reveal — the failed-retrieval struggle is the mechanism; reading first buys fluency, not storage. New this time: Part C is an integrative drill — you'll connect the newest lessons (power, inference, architecture) back into the thesis, the form the capstone will demand. Note every gap; Part D tells you exactly what to re-read.
Part A — Free Recall (L19–L23, interleaved)
Ten prompts, out of order. Answer before revealing; note every gap.
L19 · Physicswhole course
1. State the dynamic-power equation, explain why shrinking transistors used to be a free lunch (Dennard), why it ended ~2005, and the single downstream consequence that created the entire AI-hardware boom.
Model answerP ≈ α·C·V²·f. Dennard: shrink dims by k and scale V down → ~k² more transistors, faster, at constant power density (the V² term crushed per-transistor power). It ended ~2005 when V couldn't scale lower without leakage (subthreshold + gate tunneling) exploding → the power wall, clocks stuck ~3–4 GHz. Consequence: performance had to come from parallelism + specialization → GPUs, then ASICs/TPUs + advanced packaging. The end of Dennard scaling is the root cause of the whole course.
L22 · InferenceL12 · HBM
2. Inference has two phases. Name them, say which is compute-bound vs memory-bandwidth-bound and why, and use that to explain why output tokens are priced ~5× input.
Model answerPrefill (reads prompt): one parallel pass over all input tokens → compute-bound, high arithmetic intensity. Decode (writes answer): one token at a time, each re-streaming all weights from HBM → memory-bandwidth-bound. Input tokens = cheap parallel prefill; output tokens = expensive sequential decode → the ~5:1 price ratio is physics, not markup. (Decode speed = HBM bandwidth, not FLOPS → why L12 is the binding lever.)
L20 · PowerL19 · Physics
3. Define PUE. Then explain why power — not silicon — is now the binding constraint on AI growth, and the one diligence question that replaces "did they get GPU allocation?"
Model answerPUE = total facility power ÷ IT (compute) power (1.0 ideal, ~1.1–1.2 best, ~1.5 avg; cooling is the main overhead). The L19 density cascade → ~120 kW racks → 300–500 MW clusters hits a grid where a new connection takes 7–10 years (PJM application→operation >8 yrs). So a contracted, connected, shovel-ready megawatt is the scarce asset. New diligence question: "Where is the power, is it contracted, and when does it energize?"
L23 · MoEL13 · Networking
4. State the MoE inversion (what scales with active vs total params). Then name the two hardware bottlenecks MoE shifts toward, and the thesis names that benefit.
Model answerCompute/token ∝ ACTIVE params; HBM capacity ∝ TOTAL params (DeepSeek-V3: 671B total, ~37B active). MoE makes compute cheap but all experts still sit in HBM, and expert parallelism triggers all-to-all communication. So it shifts the bottleneck toward (1) HBM capacity (SK Hynix, AMD wedge) and (2) interconnect (NVLink, Broadcom). It's a Jevons deflation engine: cheaper tokens → more tokens.
L21 · $/MtokL10 · ASIC
5. Write the $/Mtok identity. Name its three levers and who owns each. Then state the build-vs-buy rule and which thesis name it is the quantitative form of.
Model answer$/Mtok = (cost-per-GPU-hour ÷ tokens-per-second ÷ 3600) × 1e6. Levers: throughput ↑ (NVDA software/model lab — copyable), HBM bw/capacity ↑ (SK Hynix; AMD wedge), $/GPU-hr ↓ (AVGO ASIC; utilities/power, L20). Build-vs-buy: own/ASIC wins at high, stable utilization; rent wins for spiky/experimental (that markup = NVDA pricing power). It's the quantitative form of the Broadcom ASIC-TAM case.
L19 · PhysicsL4 · TSMC
6. "nm is marketing." What does a node name not mean, what should you judge a node on instead, and how does the planar→FinFET→GAA progression connect device physics to the TSMC/ASML moat?
Model answer
No feature in a "3 nm" node is 3 nm — names are labels since ~22 nm. Judge on transistor density + PPA (power/performance/area), and never assume same-named nodes from different foundries are equal. Planar→FinFET→GAA→CFET each wraps the gate around more sides of the channel to restore electrostatic control lost to shrinking. That escalating difficulty (+ EUV) is why only ~3 firms remain — the device physics is the foundation of the TSMC/ASML moats.
L22 · KV cache
7. What is the KV cache and why does it exist? Give the size formula in words, the two things it grows with, why GQA exists, and what it ultimately caps.
Model answer
It caches the keys/values of all prior tokens so decode doesn't re-process the whole prompt every step (avoids O(n²) recompute). KV bytes/token = 2 × layers × (kv_heads × head_dim) × precision_bytes. Grows linearly with context length and batch (concurrent users). GQA shares KV heads to shrink it 4–8×. It caps how many users fit on a GPU → caps throughput → sets $/Mtok. (At 128k context it can need tens of GB per user.)
L23 · SSML12 · HBM
8. What does a state space model (Mamba) replace, and with what? Give the complexity win, the weakness, the hybrid fix (with the Jamba numbers), and why broad adoption is a falsifier — for which name?
Model answer
Replaces attention's growing KV cache with a fixed-size recurrent state. Win: O(n) compute, ~constant context memory (vs attention's O(n²)/growing KV). Weakness: fixed state forgets → weaker exact recall. Fix: hybrids (Jamba = Transformer-Mamba-MoE, ~1:7 attn:Mamba, 256K context at ~9 GB KV, ~10× lower than pure transformer). Broad SSM/hybrid adoption cuts HBM-per-accelerator → a concrete SK Hynix falsifier (the "memory-architecture shift" trigger). Counter-force: MoE raises total-param HBM → watch the mix.
L20 · Powerthesis
9. Name the new investable layer that power opens up (4 categories + an example each), and explain the key relationship: how does power "gate" the core seven rather than replace them?
Model answer
(1) Power gen / utilities-IPPs (Constellation, Vistra, Talen); (2) grid & electrical equipment (Eaton, Schneider, GE Vernova); (3) cooling (Vertiv, nVent); (4) nuclear / SMR (Constellation restarts; Cameco upstream). Gating: if megawatts can't be delivered, ordered GPUs can't be energized → caps realized vs nameplate demand. So interconnection approvals + PPA volume are a leading indicator for NVDA/AVGO — power data leads the silicon thesis.
L21 · $/MtokL22 · triangle
10. Explain the latency–throughput–cost triangle: what dials along it, why a tight latency SLA is literally a cost, and the one question to ask any vendor quoting a cheap $/Mtok.
Model answerBatch size dials along it. Bigger batches amortize the memory-bound weight-read across more users → higher throughput + lower $/token, but each user waits longer (higher latency). A tight latency SLA forces smaller batches → fewer tokens per GPU-hour → higher $/Mtok, so a latency guarantee is a cost. The question: "cheap $/Mtok at what latency and what batch size?" — most cost claims quietly pick the favorable corner.
Part B — Interleaved Quiz (auto-graded)
Six questions, mixed topics
Choose the best answer.
1. The first-principles cause of the entire AI-hardware boom is:
The invention of the transformer in 2017
The end of Dennard scaling forcing parallelism + specialization
Cheaper electricity making large datacenters viable
EUV lithography arriving at high volume around 2019
2. MoE shifts the binding inference constraint away from FLOPS toward:
CPU orchestration and host memory bandwidth
Disk throughput for streaming expert weights
HBM capacity plus all-to-all interconnect
Display bandwidth and PCIe lane count
3. A tight latency SLA raises serving cost because it forces operators to run:
Larger batches, which overload the interconnect
Smaller batches, so fewer tokens per GPU-hour
More copies of the model in system RAM
Higher clock frequencies on the host CPU
4. "Power is the new CoWoS" because:
Power and packaging are made by the same suppliers
Grid connection lead times now gate realized GPU deployment
Electricity has become the largest line item in a GPU's BOM
Cooling and lithography share the same physics limit
5. Broad adoption of SSM/hybrid models would most directly act as a falsifier for:
ASML — by removing the need for EUV at advanced nodes
Cadence/Synopsys — by eliminating place-and-route
SK Hynix — by cutting HBM content per accelerator
TSMC — by making leading-edge logic unnecessary
6. When does owning an ASIC most clearly beat renting GPUs?
Spiky, experimental traffic that changes models weekly
High, stable volume that fully utilizes the fixed capex
Any workload, since ASICs are always cheaper per token
Latency-critical chat with very small batch sizes
Part C — Integrative Drills (the capstone form)
These ask you to combine lessons into a call — the exact skill valuation & portfolio will demand. Produce each before revealing.
Drill 1 — Architecture → hardware demand (turn news into a call)
For each architectural move, state which hardware/thesis names it favors and which it pressures. Cover the answer first.
Pressures: SK Hynix (less HBM/long-context — the falsifier). Favors: whoever serves long context cheaply; relieves the memory wall.
Long-context + agentic/reasoning grows
Favors: decode + KV → HBM bandwidth and capacity (SK Hynix, AMD). More output tokens = more decode = more silicon + power.
Inference price deflation continues
Favors all hardware via volume (the demand engine), but compresses MaaS margin → value migrates to whoever owns the scarcest lever.
Drill 2 — Follow one dollar of capex (power + architecture aware)
Re-run the dollar trace, now including the power layer and naming the socket-agnostic toll booths. Reveal after you've said it.
Where the dollar flows (~$750B in 2026, up from ~$450B)
Accelerator + system. Largest slice → NVIDIA (margin), AMD the funded #2. Inside the BOM: TSMC (wafer + CoWoS) and SK Hynix/Micron (HBM — and HBM content rises with MoE + long context).
Networking. Scale-up + scale-out, now load-bearing for MoE all-to-all. Broadcom, NVIDIA (NVLink/Mellanox), Arista.
Power, cooling, facility (the NEW big branch, L20). At ~$45–55B/GW, the shell, substation, transformers, switchgear, liquid cooling, and the PPA absorb the dollar before silicon. Utilities/IPPs, Eaton/Schneider/GE Vernova, Vertiv, nuclear/SMR.
Invisible upstream toll — paid regardless of socket OR architecture: every chip was designed on Cadence/Synopsys and printed on ASML EUV.
Key insight: durability is highest where the cut is socket- and architecture-agnostic (ASML, TSMC, EDA), and the newest scarce hop is power — which gates whether any of the rest gets switched on.
Drill 3 — Write the two formulas from memory
The capstone is numbers. Reproduce both before revealing.
V² lever died ~2005 → parallelism/specialization; judge nodes on density + PPA
Cards 3, 9 (PUE, power layer)
L20
Power gates realized demand; a connected megawatt is the scarce asset
Cards 2, 7, 10 (prefill/decode, KV, triangle)
L22
Decode is memory-bound; KV caps users; a latency SLA is a cost
Card 5 + Drill 3 ($/Mtok, levers)
L21
Cost = $/GPU-hr ÷ tokens/hr; own vs rent = the ASIC case
Cards 4, 8 + Drill 1 (MoE, SSM)
L23
MoE → memory+network; SSM → SK Hynix falsifier; track the mix
Drill 2 (dollar trace)
L20 + THESIS.md
Socket- & architecture-agnostic tolls (ASML, TSMC, EDA) are most durable; power is the newest gate
Where you are. Three checkpoints done; the technical spine is complete — manufacturing physics (L19), the power layer (L20), and the full inference stack from $/token down to the prefill/decode engine and the architectures reshaping it (L21–L23). What remains is purely the investor half: valuation frameworks (turn all of this into TAM, margins, and per-name multiples) and the portfolio capstone (size by conviction × falsifier independence). You now have every input those need.
From your instructor: Tell me which cards or drills you missed and I'll build a short, spaced re-test of only those a few days out (true spacing). If Part C felt smooth, you're ready for the capstone — that drill (turning a technical fact into a named, falsifiable call) is the investing skill the whole course was building toward. Next: Valuation Frameworks.