Lesson 06 · Chips & LLMs

NVIDIA and the CUDA Moat

What NVIDIA actually sells, why 18 years of software compounding beats hardware specs, and where the moat actually breaks

⏱ ~55 min 📍 GPU architecture · CUDA ecosystem · Competitive moat · Investment ← Lesson 05 🔗 Glossary

The core thesis of this lesson

Most people think NVIDIA's moat is "the best GPU." That is wrong. The best GPU without the software stack is just silicon. NVIDIA's moat is 18 years of accumulated software — cuDNN, cuBLAS, NCCL, TensorRT — optimized for each new GPU generation before it ships, by engineers who have early hardware access no competitor has. The GPU is the hardware expression of that software moat, not the other way around. This lesson explains why, and what would actually have to happen for the moat to erode.

Part 1 — What NVIDIA Actually Sells

Not a chip. A platform that happens to include a chip.

In 2006, NVIDIA launched CUDA — Compute Unified Device Architecture — alongside the GeForce 8800 GTX. The nominal purpose was general-purpose GPU computing for scientists. The actual effect, 18 years later: CUDA is the de-facto programming environment for every major AI framework, every AI research institution, and every AI startup on earth.

When a user writes PyTorch code, they write something like tensor.cuda(). This is not just syntax — it is a dependency declaration. That code will only run on NVIDIA hardware without rewriting. The 10 million developers who've written this line have made a choice that is expensive to reverse. The ecosystem of models, fine-tuning recipes, profiling tools, and debugging workflows that has grown around CUDA since 2006 is the real product. The H100/B200 GPU is the hardware that runs it.

The full NVIDIA software stack — what you actually depend on

User Application (PyTorch / JAX / TensorFlow model code)

What the user writes

PyTorch / TensorFlow / JAX

Hugging Face Transformers

LangChain / vLLM / TGI

ML frameworks — all call into CUDA libraries below

cuDNN
Conv, Attention, LSTM kernels

cuBLAS
Matrix multiply (GEMM)

NCCL
Multi-GPU communication

TensorRT
Inference optimization

← The real moat. 18 years of per-GPU hand-tuning. Each new GPU ships with these already optimized. AMD's equivalents lag 6–12 months.

CUDA Runtime + CUDA Driver (kernel launches, memory allocation, stream management)

CUDA C/C++ / PTX (assembly)

GPU Hardware (H100 / B200 / RTX 5090 — SM, Tensor Core, HBM, NVLink)

What AMD / Intel / Google can match on silicon

AMD can match or exceed NVIDIA on raw silicon specs. They cannot match the middle layers — cuDNN, cuBLAS, NCCL. These are not open-source libraries you can fork; they are continuously maintained, closed-source, GPU-generation-specific kernel databases that NVIDIA engineers have been building since 2007.

Part 2 — GPU Architecture: What's Actually Inside

The H100: 132 SMs, 528 Tensor Cores, and why the numbers don't explain the performance

Understanding GPU architecture at the SM level is what separates developers who can actually optimize AI training from those who just run benchmarks.

The die layout — 132 Streaming Multiprocessors

H100 SXM5 die — simplified SM tile layout (not to scale)

HBM3 Stack #1 · 16GB

NVLink 4.0 Port — 900 GB/s bidirectional to other GPUs

HBM3 Stack #2 · 16GB

SM 1

SM 2

SM 3

SM 4

SM 5

SM 6

SM 7

SM 8

SM 9

SM 10

SM 11

SM 12

SM 13

SM 14

SM 15

SM 16

SM 17

SM 18

SM 19

SM 20

SM 21

SM 22

SM 23
← one SM
zoom below

SM 24

SM 25

SM 26

SM 27

SM 28

SM 29

SM 30

SM 31

SM 32

SM 33

· · · 132 SMs total · · ·

L2 Cache — 50MB shared across all SMs · ~10 TB/s internal bandwidth

HBM3 Stack #3 · 16GB

HBM3 Stack #4 · 16GB

HBM3 Stack #5 · 16GB

H100 SXM5: 132 SMs, 80GB HBM3, 3.35 TB/s memory bandwidth, 900 GB/s NVLink. The SM is the fundamental compute unit — a mini-processor with its own scheduler, register file, shared memory, and Tensor Cores. All 132 SMs execute independently in parallel.

Inside one SM — the unit of execution

H100 Streaming Multiprocessor (SM) — internal structure

The critical takeaway: Tensor Cores produce 59× more BF16 throughput than CUDA cores on the same die. Deep learning is almost entirely matrix multiplications — the operation Tensor Cores execute natively. This is why H100 delivers 3,958 TFLOPS BF16 on AI workloads while only 66.9 TFLOPS FP32 for general compute. The ratio is the same for B200: 10× more Tensor Core throughput than FP32.

Part 3 — The CUDA Execution Model

Threads → Warps → Blocks → Grids: the four-level execution hierarchy

CUDA's execution model is what every developer must understand to write fast GPU code — and what every investor should understand to know why switching GPUs isn't just recompiling.

●

Thread

The atomic unit of execution. One CUDA thread = one scalar operation instance. Has its own program counter and registers. A single CUDA kernel might launch billions of threads.

▬▬▬▬

Warp (32 threads)

32 threads that execute the same instruction simultaneously — SIMT (Single Instruction Multiple Thread). The warp is the smallest schedulable unit. If threads in a warp diverge (different if/else branch), both paths execute serially — this is warp divergence and kills performance.

⊞

Thread Block (1–1024 threads)

Developer-defined group of warps (must be divisible by 32). Warps in a block share L1/shared memory and can synchronize via __syncthreads(). An entire block runs on one SM — it can't be split across SMs. Block size is the first tuning knob in CUDA performance optimization.

⊞⊞⊞⊞

Grid (many blocks)

Collection of thread blocks distributed across all available SMs. CUDA runtime schedules blocks onto SMs as they become free. A 1M-element vector add might launch 4,000 blocks of 256 threads each — the scheduler fills all 132 SMs automatically.

Memory coalescing — the optimization that matters most

When a warp of 32 threads accesses memory, the GPU tries to coalesce these into a single 128-byte memory transaction. If thread 0 accesses address 0, thread 1 accesses address 4, thread 2 accesses address 8... (contiguous) — the 32 accesses merge into one HBM transaction. If threads access random addresses — one transaction per thread = 32× the memory traffic. Non-coalesced memory access is the most common reason a GPU kernel runs at 10% of theoretical bandwidth. This is the kind of micro-knowledge that takes years to internalize — and why the ML engineers who know CUDA deeply are rare and expensive.

H100 memory hierarchy — bandwidth and latency

~20 TB/s per SM — fastest possible

~20 TB/s

~1 cycle

Shared Mem / L1

~20 TB/s — programmable by developer (key optimization tool)

~20 TB/s

~20 cycles

L2 Cache (50MB)

~10 TB/s — shared across all SMs

~10 TB/s

~100 cycles

HBM3 (80GB)

3.35 TB/s — the DRAM ceiling

3.35 TB/s

~700 cycles

NVLink (peer GPU)

900 GB/s

~2μs

PCIe (host CPU)

128 GB/s

~5μs

The 6,000× bandwidth drop from register file to PCIe is why ML engineers spend careers managing data movement. Writing a matrix multiply kernel that achieves >80% of HBM bandwidth requires explicit management of shared memory (staging data from HBM into L1 in tiles), warp divergence minimization, and coalesced access patterns. cuBLAS does all of this automatically, tuned for each GPU generation. This is why cuBLAS outperforms naive implementations by 30–100× — and why AMD's first version of any new GPU is always slower than NVIDIA's equivalent on real workloads, regardless of raw TFLOPS.

Part 4 — The Library Ecosystem: Where the Moat Actually Lives

cuDNN, cuBLAS, NCCL — the three pillars of 18-year lock-in

Every serious AI framework call eventually hits one of these libraries. They are not convenience wrappers — they are optimized implementations that took years of GPU-generation-specific tuning to build, maintained by engineers who get early access to pre-production hardware.

cuDNN — Deep Neural Network Library

Launched 2014. Implements the foundational operations of deep learning: convolutions (for CNNs), attention (for Transformers), batch normalization, activation functions, pooling. Each operation has hand-tuned kernel implementations for every NVIDIA GPU architecture — Volta, Turing, Ampere, Hopper, Blackwell all have different optimal implementations because their Tensor Core wiring, register file size, and shared memory layout differ.

When you call torch.nn.functional.scaled_dot_product_attention() in PyTorch, you call FlashAttention, which calls cuDNN. AMD's equivalent (MIOpen) has caught up on convolutions but consistently lags on the attention kernels that dominate Transformer training — because attention is newer and NVIDIA's engineers optimize for each new GPU before it ships. The H100's FlashAttention3 implementation took NVIDIA researchers 6 months of GPU-specific tuning to achieve 75% of HBM bandwidth utilization. MIOpen's equivalent for MI300X achieves ~60% for comparable kernels.

cuBLAS — Linear Algebra (the foundation of deep learning)

Every forward pass and backward pass in a neural network decomposes to matrix multiplications (GEMM — General Matrix Multiply). A BERT forward pass is ~300 GEMMs. A GPT-4 class forward pass is thousands. cuBLAS implements GEMM for every combination of matrix size, data type (FP32/BF16/FP16/INT8/FP8), and transpose pattern, using optimal tiling strategies for each GPU's Tensor Core configuration.

A naive GEMM implementation achieves ~10–20% of peak Tensor Core throughput due to poor tiling and memory layout choices. cuBLAS achieves 85–95%. That is a 5–9× gap between "same GPU, different software" — meaning an AMD GPU with equivalent raw TFLOPS but a worse BLAS library delivers materially worse training throughput. This is the gap AMD is closing, but NVIDIA rebuilds it with each new GPU generation.

NCCL — Multi-GPU Communication (the hardest part to replicate)

Training a 70B parameter model requires distributing work across 512+ GPUs simultaneously. At every gradient step, each GPU must share its gradient updates with all others — an operation called AllReduce. NCCL (NVIDIA Collective Communications Library) implements AllReduce, AllGather, ReduceScatter, and Broadcast across arbitrarily large GPU clusters, optimized for NVLink topology within a node and InfiniBand between nodes.

AMD's RCCL is a fork of NCCL. It works but consistently underperforms on multi-node jobs where NVLink + InfiniBand topology optimization is critical. Training GPT-4 class models requires near-perfect AllReduce efficiency — a 5% NCCL underperformance across 10,000 GPUs translates to 500 GPU-equivalents of wasted capacity. At $3/GPU-hour, that is $1.3M/year of wasted compute per training run. [Megatron-LM: multi-GPU training paper]

TensorRT — Inference Optimization

After a model is trained, deploying it for inference requires different optimizations: layer fusion (merge consecutive ops into one kernel), quantization (FP16/INT8/FP8 to fit more in memory), kernel auto-tuning (try many implementations, pick the fastest for this specific model shape). TensorRT automates all of this. A naive PyTorch model served with vLLM on an H100 achieves ~30% of TensorRT's throughput. Closing the gap requires TensorRT. AMD's equivalent (ROCm + MIGRAPHX) is functional but less battle-tested. Cloud inference providers (AWS, Azure, GCP) base their LLM serving on TensorRT — meaning every token you generate from a cloud LLM API is a TensorRT kernel call.

Part 5 — NVLink and NVSwitch: The AI Supercomputer Interconnect

Why the GPU interconnect is as important as the GPU itself

Training a frontier model (Llama 3 405B, GPT-4 class, Claude 3 Opus class) requires thousands of GPUs working simultaneously. The bottleneck is not compute — it is the speed at which gradients propagate between GPUs. This is where NVIDIA's NVLink creates a second moat that AMD cannot replicate with software.

AllReduce communication math — why interconnect bandwidth limits training

Model: 70B params × 4 bytes/param (FP32 gradients) = 280 GB per gradient step
Cluster: 512 H100s (64 nodes × 8 GPUs)

AllReduce traffic per GPU (Ring-AllReduce): 2 × (N-1)/N × data = ~2 × 280 GB = 560 GB

With PCIe Gen5 (128 GB/s inter-GPU): 560 GB / 128 GB/s = 4.4 seconds per step
With NVLink 4.0 (900 GB/s intra-node) + IB (400 Gbps inter-node):
intra-node (8 GPUs): 560 GB / 900 GB/s = 0.62s · inter-node: ~1s on IB
NVLink reduces intra-node communication from 4.4s → 0.62s = 7× throughput gain

At 70B model scale, communication time with PCIe exceeds compute time — the GPUs sit idle waiting for gradients. NVLink keeps compute/communication ratio high enough that GPUs stay busy. This is why training efficiency (MFU — Model FLOP Utilization) drops from 50%+ on NVLink clusters to 20–30% on PCIe-only multi-GPU setups.

GB200 NVL72: NVIDIA's DGX SuperPOD-level integration

The GB200 NVL72 system connects 72 Blackwell GPUs (36 Grace-Blackwell SMs) via NVSwitch 4.0 — a dedicated switching chip with 57.6 TB/s of all-to-all bandwidth. At this scale, all 72 GPUs communicate at full NVLink speed as if they were one logical GPU with 13.5 TB of HBM and 1.4 ExaFLOPS of BF16 compute. AMD has no equivalent at this integration level. Intel's Gaudi 3 cluster topology (using Ethernet-based RoCE instead of NVLink) achieves lower bandwidth and higher latency at scale. The NVSwitch fabric is the direct technical reason why hyperscalers building AI infrastructure default to NVIDIA — the alternative requires solving the distributed training software problem on a less mature interconnect.

Part 6 — Competitive Landscape: What Threatens the Moat

Four challengers — and an honest assessment of each

AMD + ROCm (MI300X / MI350)

Hardware: MI300X: 192GB HBM3 (2.4× H100), competitive BF16 throughput, excellent for inference of large models that don't fit in 80GB. MI350: targeting performance parity with H100.

Software: ROCm 6.x is meaningfully better than ROCm 4.x. HIP (Heterogeneous-computing Interface for Portability) lets most CUDA code be ported with minimal changes. MIOpen and RCCL have closed the gap, though NCCL at multi-node scale remains NVIDIA's advantage.

Where AMD wins today: LLM inference (large models need 192GB), cost-sensitive hyperscaler deployments where per-GPU cost matters, Microsoft Azure has made public commitments to MI300X.

Where AMD still loses: Training at scale (NCCL vs RCCL gap), fine-tuning with custom CUDA kernels, any customer whose ML team is CUDA-native (which is almost all of them).

Threat to NVIDIA moat: Medium — real but bounded. AMD gaining inference share; not training.

Google TPU v5 (internal) + JAX

Architecture: Not a GPU — a systolic array matrix multiply accelerator (MXU). TPU v5p: 459 TFLOPS BF16 per chip, but 4,096 chips run as a single pod with 4,800 GB/s memory bandwidth per chip. XLA compiler handles kernel generation — no hand-written kernels needed.

JAX framework: Google's ML framework (NumPy-like, functional) runs natively on TPU and is increasingly preferred for research at Google Brain / DeepMind. It is hardware-agnostic (runs on GPU/TPU/CPU) via XLA.

The key fact: Google does NOT sell TPUs to external AI labs. Cloud TPU access exists but is limited. Google's TPUs reduce NVIDIA's TAM by removing one of the world's largest AI compute buyers from the market — but they don't win share from other NVIDIA customers.

Investment implication: TPUs are a Google cost advantage, not a NVIDIA revenue threat in the traditional sense. Google's internal AI spend ($15–20B/yr) that doesn't flow to NVIDIA is the real impact.

Threat to NVIDIA moat: Low externally — very high to NVIDIA's Google revenue.

Amazon Trainium 2 / Inferentia 3

Architecture: Custom ASIC trained on AWS's specific ML workload distribution. Trainium 2 announced for 2024: 4× better performance/watt vs. Trainium 1, clusters up to 100,000 chips connected via AWS UltraCluster.

Neuron SDK: PyTorch-compatible (via torch_neuronx). Models must be compiled for Trainium, not just relinked. Compilation handles kernel generation automatically — closer to TPU's XLA approach than AMD's HIP approach.

Current state: AWS runs internal workloads (Alexa, Amazon Ads, AWS foundational models) on Trainium to reduce NVIDIA dependency. External customers (startups) still default to GPU instances (P4/P5). Anthropic has access to Trainium through AWS partnership but Claude training is still primarily NVIDIA.

Long-term: AWS's stated goal is 70%+ of internal AI on Trainium by 2027. This directly reduces AWS's NVIDIA GPU orders — and AWS is one of NVIDIA's largest customers.

Threat to NVIDIA moat: Medium for AWS revenue specifically — low for general market.

Huawei Ascend 910B/910C (China)

Hardware: Ascend 910B: 256 TFLOPS FP16 (vs H100's 2,000 TFLOPS) — roughly an 8× performance gap. SMIC 7nm-class process, yield constraints. 910C (2025): estimated 600 TFLOPS FP16 — closes to ~3× gap.

Software: CANN (Compute Architecture for Neural Networks) + MindSpore framework. Ecosystem is immature vs. CUDA but rapidly improving under export control pressure. Chinese AI labs (Baidu, Alibaba, ByteDance) are being forced to optimize for Ascend.

Key constraint: Each cluster of Ascend chips requires 3–4× more chips to match H100 training throughput — meaning China's AI training capacity is severely constrained by chip availability (SMIC yield issues), not just raw performance gap.

Investment implication: Ascend is not a threat to NVIDIA outside China. Inside China, it is the only legal alternative — and the export controls that created this situation are NVIDIA's biggest regulatory risk (limiting its China market) and competitive protection (limiting Ascend's volume).

Threat globally: Low. Threat to NVIDIA's China-specific revenue: already realized (export controls).

The moat structure — where NVIDIA dominates vs. where it's contested

NVIDIA dominant AMD competitive Google/custom ASIC only Contested

Training — foundation models

NVIDIA (NCCL + NVLink makes scale work)

AMD

TPU

Inference — large models

NVIDIA (TensorRT, broad deployment)

AMD MI300X (192GB fits more)

Custom ASICs (AWS, Google)

Multi-GPU scale-out

NVIDIA (NVLink/NVSwitch, NCCL — no peer)

AMD (RoCE-based)

TPU

Developer ecosystem

NVIDIA (10M+ CUDA developers, all tooling)

AMD (HIP compat)

Edge / consumer inference

NVIDIA (RTX, Jetson)

AMD

Apple Silicon / Qualcomm NPU (mobile)

The moat erosion signal to watch: compiler abstraction layers

The biggest structural threat to CUDA lock-in is not AMD hardware — it's compiler abstraction. Triton (OpenAI's GPU kernel language, hardware-agnostic), MLIR (Google/LLVM multi-level IR), and the JAX/XLA stack all target "write once, compile to any GPU." If Triton-based kernels achieve 90%+ of hand-tuned CUDA performance on both NVIDIA and AMD hardware, the "CUDA developer knows NVIDIA-specific APIs" argument weakens. PyTorch 2.0's torch.compile uses Triton internally. This is the slow-burning threat — not a competitor's chip. [OpenAI Triton]

Part 7 — Financial Analysis: What You're Actually Buying

Revenue, concentration, and the question the P/E doesn't answer

~$130B

FY2026 Revenue

+114% YoY. Data center = 87% of total.

~76%

Gross Margin

Record high. Pure software moat expressed in margin.

~35–45×

Forward P/E (FY2027E)

Priced for sustained 30–40% growth 3+ yrs.

~$3.4T

Market Cap

Most valuable company by mkt cap at times in 2025.

Revenue concentration — who is actually paying NVIDIA

Microsoft (Azure)

Maia + H100/B200

~17%

The hyperscaler in-house silicon risk — the right framing

Microsoft, Google, Amazon, and Meta are all building in-house AI silicon. Microsoft has Maia 100. Google has TPU v5. Amazon has Trainium 2. Meta has MTIA. The common investor reaction: "this will destroy NVIDIA." The correct framing is more nuanced:

In-house silicon reduces growth rate, not current revenue. If Microsoft in 2027 runs 30% of Azure AI workloads on Maia instead of H100, that is 30% × 17% of NVIDIA's data center revenue = ~5% revenue headwind. Meaningful — but not existential if NVIDIA's remaining revenue is still growing 20–30%.
In-house silicon targets commodity inference, not frontier training. You build custom silicon to run known, stable workloads cheaply (inference of deployed models). You still buy NVIDIA for the bleeding edge (training next-generation models) where optimization is ongoing and software maturity matters. The hyperscalers are not stopping NVIDIA H100 purchases — they are adding Maia/TPU alongside them.
Meta is the risk outlier. Meta has no cloud revenue to protect — they run AI for their own products (Facebook, Instagram, WhatsApp, Llama). Meta is the one hyperscaler that could genuinely shift a large fraction of its AI compute to MTIA without customer-facing risk. Watch Meta's capex allocation between NVIDIA and MTIA as the leading indicator.

Valuation — what the P/E is pricing in

The forward P/E is not the right frame for NVIDIA at this stage

At ~35–45× forward P/E (FY2027E earnings), NVIDIA is priced for sustained ~30–40% earnings growth for at least 3–5 more years. The question is not whether NVIDIA is "expensive on P/E" — at those growth rates, it is. The question is: what is the probability that the growth sustains?

The growth sustains if: (1) AI training demand grows as hyperscalers scale foundation models, (2) TSMC delivers B200/B300 CoWoS-L supply without disruption, (3) no credible competitor closes the software library gap within the holding period, (4) no regulation or export control severely limits the addressable market.

The growth disappoints if: hyperscaler AI capex pulls back (too much capex, not enough ROI from AI products), China export controls expand to more countries, or TSMC CoWoS-L hits a yield wall that limits B200 supply while AMD MI350X ships on time.

A TSMC long-term holder (Lesson 4) and a NVIDIA holder both need the same thing: AI training demand to be structural, not cyclical. Their risks are therefore correlated. Holding both is not as diversified as it appears.

Practice Project — NVIDIA Moat Stress Test

Five competitive scenarios. Rate each as a HIGH, MEDIUM, or LOW threat to NVIDIA's moat. Then see the assessment and reasoning. There is one clearly correct answer for each — the reasoning is what matters.

Scenario 1: AMD releases MI400 with 2× H100's BF16 TFLOPS and 256GB HBM4.

AMD closes the hardware spec gap entirely. Benchmark: AMD wins on raw TFLOPS and memory capacity.

Scenario 2: PyTorch 3.0 adopts Triton as its default kernel compiler, achieving 95% of cuDNN performance on both NVIDIA and AMD hardware automatically.

The compiler abstraction removes the cuDNN-specific advantage.

Scenario 3: Meta announces it will run 80% of AI training on its in-house MTIA chip by 2027, eliminating its NVIDIA dependency.

Meta is ~15% of NVIDIA data center revenue and the largest non-cloud buyer without its own public cloud business.

Scenario 4: US government imposes strict export controls extending to all non-allied countries, reducing NVIDIA's addressable data center market by 25%.

This would include countries like India, Brazil, Saudi Arabia — current major GPU buyers.

Scenario 5: Hyperscalers (Microsoft, Google, Amazon) collectively announce a 30% reduction in AI infrastructure capex, citing insufficient ROI from current AI products.

This is the "AI capex bubble" scenario. Top 4 hyperscalers = ~55% of NVIDIA data center revenue.

Technical quiz

1. An H100 GPU has 3,958 TFLOPS of BF16 Tensor Core throughput but only 67 TFLOPS of FP32 CUDA core throughput. What is the primary reason for this 59× difference?

Tensor Cores run at a higher clock speed than CUDA cores

Tensor Cores execute a 4×4 matrix multiply-accumulate (MMA) as a single operation, doing 64+ floating-point ops per clock that CUDA cores would need 64 cycles to complete one-by-one

BF16 numbers are 2× smaller than FP32 numbers so twice as many fit in one register

H100 has 128 CUDA cores but 4,096 Tensor Cores per SM

2. A developer writes a CUDA kernel where each thread in a warp accesses a random, non-consecutive memory address. What performance problem is this?

Warp divergence — threads executing different instructions

Non-coalesced memory access — 32 separate HBM transactions instead of one 128-byte burst, destroying effective bandwidth

Bank conflict in shared memory — concurrent threads hitting same bank

3. Why does training a 70B-parameter model on 512 GPUs connected only via PCIe Gen5 result in severely degraded GPU utilization (MFU ~20%)?

PCIe cannot carry the CUDA kernel binaries large enough for 70B models

70B model weights don't fit in 80GB HBM and PCIe swapping kills performance

AllReduce gradient communication at 128 GB/s PCIe takes 4+ seconds per step, exceeding compute time — GPUs sit idle waiting for gradients from peer GPUs

PCIe doesn't support NCCL's ring-AllReduce algorithm, forcing tree-AllReduce at lower bandwidth

4. The most meaningful near-term threat to NVIDIA's software moat comes from:

AMD releasing ROCm 7.0 with full cuDNN API compatibility

Compiler abstraction layers (Triton, torch.compile, XLA) that generate near-optimal kernels for any GPU hardware, removing the need for GPU-specific hand-tuned libraries

Intel acquiring AMD and combining Gaudi with ROCm's ecosystem

Huawei Ascend achieving H100-level performance via Tao's Law architectural compression

5. From an investor perspective, why does a TSMC long-term holder and an NVIDIA long-term holder have correlated (not diversified) risk?

TSMC manufactures NVIDIA's chips, so TSMC supply disruption directly limits NVIDIA's revenue

NVIDIA is a major TSMC shareholder, creating cross-ownership risk

Both positions require the same underlying belief: that AI training demand is structural and sustained. A hyperscaler capex pullback or AI demand disappointment hurts both — NVIDIA's revenue directly and TSMC's HPC node utilization and CoWoS orders

Both companies face the same Taiwan geopolitical risk since NVIDIA's HQ is in California but all production is in Taiwan

Primary sources

Acquired — NVIDIA (2023, 5 hours) — The definitive business narrative. Covers Jensen Huang, CUDA's origin, the missed mobile opportunity, the AI bet that worked. Essential listening.
NVIDIA H100 Architecture Whitepaper — Full SM architecture, Tensor Core specs, NVLink details. The authoritative technical reference.
OpenAI Triton tutorials — Write and understand GPU kernels yourself. The best way to build intuition for what cuDNN does that takes years to replicate.
NCCL documentation — Understand AllReduce, ring topology, bandwidth calculations. The best reference for why multi-GPU training is NVIDIA-specific.
SemiAnalysis — "NVIDIA's Software Moat" — Dylan Patel on what exactly makes the library ecosystem hard to replicate.

What comes next

Lesson 7: The Transformer architecture from first principles — what self-attention actually computes, why it is memory bandwidth-bound (connecting back to this lesson), and why that determines which hardware wins at inference scale

Ask me anything. Good follow-ups for a tech worker: "Walk me through what happens when I call torch.nn.Linear down to the Tensor Core instruction level" · "How does FlashAttention reduce HBM memory traffic for the attention operation?" · "Write a simple CUDA kernel and explain why the naive version is slow" · For an investor: "How do I model the revenue impact if Meta shifts 50% of training to MTIA?" · "What does NVIDIA need to do to sustain 30% revenue growth for 5 more years?" · "How should I think about NVIDIA + TSMC as a portfolio position — correlated or complementary?"