Why hyperscalers pay $1B to design their own chips, how Broadcom became the invisible engine of AI infrastructure, and what the XPU TAM means for a long-term holder
Broadcom (AVGO) has two distinct AI businesses that most investors conflate.
Business 1: Custom ASIC design (XPUs) for Google, Meta, and a third unnamed hyperscaler —
chips designed specifically for AI training and inference at hyperscaler scale.
Broadcom's CEO Hock Tan has guided each XPU customer to represent a $60–90B TAM by FY2027.
Business 2: AI networking — Ethernet switch chips (Tomahawk, Trident) and
SerDes (serializer/deserializer) that connect every GPU and ASIC in every AI cluster,
regardless of who wins the GPU war. Broadcom has ~80%+ share of high-speed datacenter Ethernet.
Understanding both — and why they're structurally different risk profiles — is the lesson.
Part 1 — The Programmability Spectrum: From CPU to ASIC
Every point on the spectrum trades flexibility for efficiency
CPU
General purpose
Any program
~1 TFLOPS
GPU
Massively parallel
Any SIMT workload
~4,000 TFLOPS BF16
DSP
Signal processing
Fixed pipelines
Domain specific
FPGA
Reconfigurable
Post-fab flexible
Re-program any time
ASIC
Fixed hardware
One workload forever
3–10× more efficient
An ASIC (Application-Specific Integrated Circuit) is a chip designed to do exactly one thing —
and to do it more efficiently than any general-purpose alternative. Every unnecessary circuit
is eliminated. The logic is hardwired to match the specific computation, minimizing wire
length, power, and area. The result: for the workload it was designed for,
an ASIC achieves 3–10× better performance-per-watt than a GPU.
The cost: an ASIC cannot be reprogrammed. If the workload changes — new model architecture,
new data format, new precision requirement — the chip becomes less useful or useless.
The other cost: tapeout. Designing and manufacturing a leading-edge ASIC on TSMC 3nm
costs $500M–$2B+ in NRE (Non-Recurring Engineering) before you produce a single chip.
This is why only hyperscalers — organizations running 10,000–100,000+ chips of the same
workload continuously — can justify custom ASICs.
Part 2 — What's Inside an AI ASIC: The Design Space
Three core choices: compute engine, memory architecture, dataflow
Generic AI ASIC architecture — the building blocks every hyperscaler custom chip shares
Every major AI ASIC follows this basic blueprint. The differences are in the details:
systolic array vs. vector engine for the matrix multiply unit;
HBM2e vs. HBM3 for memory; ICI vs. Ethernet for inter-chip communication.
What is consistent: compiler-managed scheduling (AOT, no runtime warp dispatcher),
dedicated SRAM for weight tiling, HBM for model storage.
Broadcom's involvement spans the full chip (XPU co-design for Google/Meta)
and the infrastructure around it (switch chips connecting all ASICs in the cluster).
The three dataflow patterns — the most important architectural choice
Weight-Stationary
Model weights loaded into on-chip SRAM or systolic array accumulators once. Input activations stream through the array. Weights don't move while a layer is being computed.
Best for: inference of a single model served repeatedly
Minimizes weight re-loading overhead when the same layer runs for thousands of requests. Google TPU v4 uses this for inference workloads. Groq LPU is the extreme case: weights are stored in a massive on-chip SRAM (not HBM) — zero weight re-loading latency ever.
Output-Stationary
Each PE accumulates its partial sum (one output element) across all input activations. Partial sums stay in PE registers. Inputs and weights both stream through.
Best for: large-batch training (multiple outputs computed per weight load)
Minimizes memory writes (partial sums stay in registers until complete). Amazon Trainium 2 uses a variant of output-stationary dataflow optimized for training large batches.
Input-Stationary
Input activations are broadcast to all PEs simultaneously. Each PE applies its own set of weights. Weights stream in from SRAM.
Best for: small batch inference (few inputs, many weight variations)
Minimizes input re-loading. Meta's MTIA reportedly uses a flexible dataflow that can switch between input-stationary and output-stationary depending on layer type — important for serving diverse model architectures.
Part 3 — The Custom ASIC Economics: When Does $1B of NRE Pay Off?
The breakeven calculation that every hyperscaler's CFO runs
Custom ASIC business case — Google TPU as the reference case
Assumption: hyperscaler runs 100,000 AI chips continuously for 3 years (training + inference) GPU alternative: 100,000 × H100 @ $3.50/hr = $350,000/hr = $9.2B/year × 3 = $27.6B over 3 years
Custom ASIC option: NRE (design, tapeout, bring-up): ~$1–2B one-time Wafer cost (same TSMC node): same as GPU silicon cost ← no licensing premium Efficiency gain: 3–5× performance/watt vs. GPU for the specific workload TCO reduction: ~40–60% less power + cooling + GPU premium → ~$11–16B savings over 3 years
ROI: $11–16B savings / $1–2B NRE = 6–15× return over 3 years
The math only works at hyperscaler scale. A company running 500 GPUs has NRE of $2B and savings of $138M — deeply negative. The crossover is roughly 10,000+ chips of sustained, stable workload.
The key phrase is "stable workload." An ASIC is designed for one specific computation.
Google's search ranking neural network has run the same basic architecture (with incremental updates)
for years. Inference of a deployed model is stable. Training frontier models is less stable
— architectures change with each generation. This is why hyperscaler ASICs are more economically
justified for inference than for frontier training, though the largest labs (with 2–3 year
architecture cycles) now design training ASICs too (Google TPU v5p for Gemini training).
The ASIC risk for NVIDIA: it reduces TAM, not existing revenue
When Google deploys TPU chips for Gemini inference instead of H100s, that's
NVIDIA revenue that never existed — not revenue lost. Hyperscaler ASIC programs are
15–18 month design cycles, and existing GPU infrastructure continues running in parallel.
Meta runs MTIA alongside H100s — MTIA handles stable workloads (ranking, recommendation)
while H100s handle novel models and research. The correct frame: hyperscaler ASICs
cap NVIDIA's growth rate within that hyperscaler, they don't immediately reduce NVIDIA's
revenue. For the TSMC/Broadcom investor, hyperscaler ASICs are additive demand
(more chip design work, more wafers), not substitutive.
Part 4 — The Hyperscaler ASIC Landscape
Every major hyperscaler now has custom silicon. The status of each.
Broadcom confirmed as Google TPU design partner. Revenue relationship spans v3–v5. Google's $75B 2025 capex flows partly through Broadcom NRE and wafer costs through TSMC.
MTIA v2
Meta
Broadcom (confirmed 2024)
Inference
Custom matrix engine, TSMC 5nm, optimized for ranking/recommendation/ads inference. RoCE Ethernet interconnect.
Meta confirmed Broadcom as MTIA design partner in 2024. Meta's stated goal: 70%+ of ad serving inference on MTIA by 2026. Meta's ~$60B AI capex includes Broadcom XPU revenue.
Primary design partner is Marvell Technologies (MRVL) — Broadcom's main competitor here. AWS targets 70%+ of internal AI on Trainium by 2027. Marvell is the Broadcom analog for the Amazon relationship.
Maia 100
Microsoft
Unknown / internal (some Broadcom IP)
Inference
Optimized for Azure OpenAI workloads (GPT-4 inference). First deployed on Azure in 2023. TSMC 5nm.
Microsoft has been less public about Maia's performance. External customers (OpenAI, enterprise) still use H100. Maia reduces Microsoft's own inference cost without displacing NVIDIA for external GPT-4 API workloads.
MTIA / "Project Athena"
ByteDance
Broadcom (rumored/confirmed)
Train + Infer
Details under NDA. Optimized for TikTok recommendation model serving. TSMC node unknown.
ByteDance confirmed as a Broadcom XPU customer (unnamed "third customer" in Hock Tan's guidance). Significant revenue contributor to Broadcom's AI segment. Export control risk: ByteDance is Chinese-owned, subject to US chip export regulations.
Apple Neural Engine
Apple
Apple internal + Broadcom SerDes IP
On-device
Built into M/A-series SoCs. 38 TOPS (A17 Pro). On-device inference for AI features.
Apple recently mentioned as potential fourth Broadcom XPU customer (unconfirmed — may refer to SerDes/networking IP rather than full chip design). Apple has historically designed its own chips.
Part 5 — Broadcom's XPU Business: The Co-Design Model
Broadcom doesn't manufacture chips. It designs them and takes a royalty.
Broadcom's XPU model is a co-design partnership: the hyperscaler brings the AI architecture
knowledge (what the chip needs to compute) and the software stack (compiler, framework).
Broadcom brings the physical design expertise — RTL implementation, timing closure,
place-and-route, DFT (design for test), packaging integration, bring-up engineering.
TSMC manufactures the final chip. Broadcom receives an NRE fee plus a per-chip royalty.
The XPU co-design flow — who does what
Broadcom's role is the high-skill, high-margin middle layer: physical implementation expertise
that a software company like Google cannot do in-house without years of recruiting specialized
chip design engineers. Broadcom's 20,000+ chip engineers have done thousands of tapeouts on
every TSMC node — they deliver designs that hit timing, yield well, and package correctly
the first time. A missed tapeout on TSMC 5nm costs 6 months and $100M+ in re-spin costs.
Broadcom's experience is worth the royalty.
Hock Tan's $60–90B per-customer TAM statement — what it means
Assumption: each hyperscaler XPU customer reaches 1M chip deployment by FY2027 Broadcom revenue per chip (NRE amortized + royalty): ~$60–90 per chip 1M chips × $60–90/chip = $60–90B per customer
With 3 confirmed customers (Google, Meta, ByteDance): TAM = 3 × $60–90B = $180–270B addressable market
⚠ Addressable ≠ Broadcom revenue. Broadcom captures a royalty on chip value, not the chip price. Broadcom's revenue share: estimated 10–15% of chip value = $18–40B from XPU alone This is on top of networking revenue (~$8B+ AI networking run rate for FY2025)
FY2027 AI revenue target: $25–35B+ (XPU + networking combined) Key risk: ByteDance (unnamed 3rd customer) subject to US export controls. If ByteDance is restricted from receiving these chips, Broadcom loses ~1/3 of XPU TAM. This is the most important near-term risk for AVGO.
Part 6 — AI Networking: The Business That Wins Regardless of GPU/ASIC Wars
Every GPU and every ASIC needs to talk to others. Broadcom sells the pipes.
A single H100 in isolation is not a useful AI system. Training a 70B model requires
512+ GPUs. Inference serving requires dozens. They all communicate — AllReduce for
training, request routing for inference, checkpoint storage. Every communication packet
passes through network switch chips. Broadcom makes ~80% of the high-speed Ethernet
switch chips used in datacenter AI clusters worldwide.
🖥
Host NIC (Network Interface Card)
400G/800G Ethernet adapters on each server. Connects GPU/CPU to the network fabric. Broadcom Stingray / Inline NICs. Also: NVIDIA ConnectX-7 (InfiniBand + Ethernet).
Broadcom + NVIDIA compete
🔀
Top-of-Rack (ToR) Switch
One switch per rack (32–64 servers). Broadcom Trident 4 (12.8 Tbps): 32×400G or 16×800G ports. Aggregates rack-level traffic.
~80% Broadcom share
⚡
Spine/Fabric Switch (AI training)
Broadcom Tomahawk 5 (51.2 Tbps): 64×800G ports. The spine switch for AI fat-tree networks connecting 100,000+ GPUs. Each cluster needs dozens of Tomahawk 5 chips.
Dominant — Broadcom
🌐
Wide-Area / DCI Routing
Broadcom Jericho 3 (10.8 Tbps) router chips for inter-datacenter AI traffic. Google, Meta use Jericho for global AI inference routing.
~70% Broadcom share
⚔
InfiniBand (NVIDIA Mellanox)
NVIDIA's Quantum-X800 InfiniBand switch: 3,200 Gbps throughput, purpose-built for HPC/AI. Used by most H100 GPU clusters today. RDMA, lossless, very low latency.
NVIDIA Mellanox dominates GPU clusters
The InfiniBand vs. Ethernet battle — the most important networking investment question
Ultra Ethernet (UEC) — led by Broadcom
Ultra Ethernet Consortium founded 2023 — Broadcom, AMD, Meta, Cisco, Intel, HPE, Microsoft
Goal: match InfiniBand's RDMA performance with standard Ethernet economics
Tomahawk 5 already supports 800G RoCE (RDMA over Converged Ethernet) — functionally equivalent to InfiniBand for many workloads
Ethernet switch cost: 3–5× cheaper than equivalent InfiniBand
Meta runs entire AI cluster on RoCE Ethernet (Tomahawk/Trident) — no InfiniBand
Google uses ICI (custom) internally, but buys Broadcom switches for GCP GPU instances
Purpose-built for HPC/AI cluster communication since the 1990s
Native RDMA (Remote Direct Memory Access) — GPU reads peer GPU's memory directly without CPU
Lossless by design — credit-based flow control means zero packet drops
Sub-microsecond latency — critical for synchronous AllReduce in training
NVIDIA's Quantum-X800: 400G per port, 2× ports vs. Ethernet equivalent — denser
Bundled with NVLink ecosystem — if you buy H100s, NVIDIA pushes InfiniBand hard
50%+ revenue from GPU cluster deployments — tied to NVIDIA GPU growth
Why Ethernet winning would be very good for Broadcom
If Ultra Ethernet achieves InfiniBand parity for AI workloads (credible — Meta proved it works at scale),
Broadcom replaces NVIDIA Mellanox as the AI networking supplier of choice.
Broadcom already makes 80% of Ethernet switch ASICs. NVIDIA's InfiniBand business (Mellanox)
contributed ~$10B of revenue in FY2024 and carries high margins.
Displacing InfiniBand with Ethernet is not just a Broadcom revenue opportunity —
it directly attacks one of NVIDIA's two major revenue streams (GPU + networking).
This is the highest-upside scenario for AVGO's networking business.
The signal to watch: hyperscaler procurement decisions in 2025–2026 for their next-generation
AI clusters — whether they spec InfiniBand or RoCE Ethernet.
Part 7 — Broadcom Investment Thesis
Revenue breakdown, AI segment trajectory, and the honest bear case
$51.6B
FY2024 Revenue
~$30B semiconductor + ~$21B software (VMware). First full year including VMware.
$12.2B
FY2024 AI Revenue
XPU design + AI networking combined. Up from $2.3B in FY2023 — 5× in one year.
$17–20B+
FY2025 AI Revenue Guidance
Hock Tan guided to this at FY2024 earnings. Would be ~35–40% of total revenue.
~76%
Semiconductor Gross Margin
~91% for software (VMware). Blended ~79%. Software acquisition was margin-accretive.
~30–35×
Forward P/E (FY2026E)
Cheaper than NVIDIA (~40×) on comparable forward earnings, reflecting lower AI revenue visibility.
Broadcom AI revenue composition — the two businesses
XPU design (Google)
TPU v4/v5 design fees + royalties
~$5B est.
XPU design (Meta)
MTIA v2 — ramping in 2025
~$3B est.
XPU design (3rd cust.)
ByteDance — export control risk
~$2B est.
AI networking (switch)
Tomahawk/Trident in AI clusters
~$2.5B est.
AI networking (SerDes)
In every AI chip
~$0.6B est.
Estimates based on analyst work-ups from public guidance. Broadcom does not break out by customer. Google is the dominant XPU revenue contributor; Meta is ramping rapidly.
The key investment questions for a long-term Broadcom holder
ByteDance export control risk
ByteDance (TikTok's parent, China-headquartered) is Broadcom's third XPU customer. US export controls already restrict H100 sales to China. If controls expand to cover custom-designed ASICs (not just off-the-shelf GPUs), Broadcom could be forced to terminate the ByteDance relationship. Estimated impact: ~$2B annual revenue at risk. Probability: non-trivial given current US-China tech tensions. This is the single most important near-term risk for AVGO.
HIGH severity, MEDIUM probability
Hyperscaler XPU in-sourcing
Google, Meta, or a future customer decides to build their own chip design team and stop using Broadcom. Google has ~5,000 custom silicon engineers already — but the physical implementation expertise Broadcom brings (timing closure, DFT, packaging co-design) would take 5+ years to build in-house. More likely for new greenfield designs, not imminent for existing programs.
MEDIUM severity, LOW probability near-term
XPU program delays or cancellations
Custom chip programs are 18–24 month cycles. If a hyperscaler's AI strategy pivots (e.g., they find a new architecture works better on GPU, or an external AI provider wins their internal use cases), they may delay or cancel the next XPU generation. NRE already spent is sunk cost; royalty revenue disappears. Meta's MTIA was delayed 1+ year — this is not hypothetical.
MEDIUM severity, MEDIUM probability
Marvell gains 2nd + 3rd XPU customers
Marvell Technologies (MRVL) is directly competing for XPU design wins. Amazon Trainium uses Marvell. If Marvell wins Microsoft Maia and a future customer, Broadcom's XPU market share could stall at Google + Meta. Marvell trades at a lower valuation and is the pure-play alternative for investors who want ASIC design exposure without VMware software.
MEDIUM severity, MEDIUM probability
InfiniBand wins the AI networking battle
If GPU clusters continue using InfiniBand and the Ultra Ethernet standard stalls, Broadcom's networking share of AI clusters stays below its potential. Currently Broadcom has high share of Ethernet but InfiniBand dominates GPU-heavy clusters (NVIDIA's home territory). The resolution depends on whether customers adopting ASICs prefer Ethernet (more likely) or whether GPU-first customers stick with InfiniBand.
LOW severity (Ethernet growing regardless), LOW probability of complete InfiniBand dominance
VMware integration execution
Broadcom paid $69B for VMware (closed Nov 2023). VMware transition to subscription pricing is causing enterprise customer churn. If VMware software revenue disappoints, the financial leverage Broadcom used to buy it (elevated debt) becomes a problem. AI semiconductor growth is masking VMware risk in 2024–2025. Watch VMware ARR trajectory as a separate risk factor.
MEDIUM severity, MEDIUM probability near-term
The bull case in one paragraph
Broadcom is the only public company that is simultaneously:
(1) a direct financial beneficiary of Google, Meta, and ByteDance's AI hardware buildout through XPU design royalties;
(2) the dominant supplier of Ethernet switch chips connecting every AI cluster globally, regardless of GPU or ASIC vendor;
(3) positioned to replace NVIDIA InfiniBand networking if the Ultra Ethernet Consortium succeeds.
At ~30–35× forward P/E vs. NVIDIA's ~40×, Broadcom offers broader AI infrastructure exposure
at a lower multiple, with the VMware software business as a second profit engine providing
earnings stability. The bull case requires: ByteDance relationship surviving export controls,
XPU programs executing on schedule, and Ethernet gaining share vs. InfiniBand.
If all three materialize, FY2027 AI revenue of $25–35B+ at Broadcom's margins
supports a significantly higher stock price from any current valuation baseline.
Practice Project — ASIC Business Case Calculator
Enter the parameters of a hyperscaler's AI compute deployment. Calculate whether a custom ASIC is financially justified vs. renting GPU compute.
Technical quiz
1. A hyperscaler designs a custom ASIC for LLM inference, choosing a weight-stationary dataflow. What specific operational pattern makes weight-stationary the right choice for inference serving?
Inference requests are stateless, so weights don't need to persist between requests — weight-stationary minimizes unnecessary weight retention in SRAM
During inference, the same model weights are applied to thousands of different input requests. Weight-stationary loads each weight once and streams many activations through it, amortizing the weight load cost across many requests and minimizing HBM reads
Weight-stationary allows the same weight to be quantized to different precisions for different requests simultaneously, enabling mixed-precision serving
Inference computations are output-bounded, so stationary weights allow outputs to accumulate across time steps without reloading
2. Broadcom's XPU co-design model earns NRE fees plus a per-chip royalty. Why can't a hyperscaler like Google simply hire chip design engineers and eliminate Broadcom entirely?
Broadcom holds patents on the systolic array design that TSMC requires be licensed before manufacturing
Physical implementation expertise (timing closure, DFT, place-and-route on leading-edge nodes, packaging co-design) requires 10,000+ person-years of accumulated institutional knowledge — a missed tapeout costs $100M+ and 6+ months. Google can specify architecture; it cannot yet replace Broadcom's implementation depth cost-effectively
TSMC's PDK (Process Design Kit) is only licensed to foundry partners like Broadcom, not to fabless customers directly
US export regulations require that AI chip designs be reviewed by an approved domestic design house before TSMC is permitted to manufacture them
3. Meta's AI cluster uses RoCE (RDMA over Converged Ethernet) with Broadcom Tomahawk switches instead of NVIDIA InfiniBand. From Broadcom's perspective, what does this prove about the Ultra Ethernet Consortium's commercial viability?
It proves that Ethernet is faster than InfiniBand for AI training, since Meta trains Llama models faster than any InfiniBand-based competitor
It provides a large-scale, production proof point that Ethernet-based RDMA can support frontier AI training and inference at 100,000+ GPU scale — removing the technical risk argument that InfiniBand is necessary, and making the cost argument (Ethernet is 3–5× cheaper) the primary decision factor for new clusters
It proves that Broadcom has already won the AI networking battle since the largest social media AI spender chose Ethernet
It proves only that MTIA chips work better with Ethernet than with InfiniBand, since MTIA uses RoCE natively — a hardware compatibility argument, not a general networking preference
4. Broadcom's "third XPU customer" (widely believed to be ByteDance) generates an estimated $2B of annual XPU revenue. US export controls currently restrict sale of H100/B200 GPUs to China. What regulatory distinction currently allows Broadcom to serve ByteDance?
Broadcom's chips are designed in the US and manufactured in Taiwan, placing them outside the scope of US export control jurisdiction
Current US export controls (ECCN restrictions) specifically target high-performance AI accelerator chips defined by performance thresholds (FLOPS, memory bandwidth). Custom-designed ASICs that fall below these thresholds, or that are explicitly licensed for a specific customer, currently occupy regulatory gray areas that off-the-shelf H100/B200 chips do not
ByteDance's HQ is incorporated in the Cayman Islands, not China, placing it outside the geographic scope of China-specific export controls
TikTok's pending US ownership divestiture granted ByteDance temporary exemption from AI chip export restrictions pending the final ownership decision
5. As a long-term Broadcom investor, how should you think about Marvell Technologies (MRVL) — as a competitive threat or a corroborating signal?
Marvell is a pure competitive threat — every dollar of XPU revenue Marvell wins from a new hyperscaler is a dollar Broadcom failed to capture
Marvell is irrelevant — Broadcom's relationships with Google and Meta are exclusive and contractually locked, preventing Marvell from competing for the same customers
Primarily a corroborating signal: Marvell winning Amazon Trainium confirms that the custom ASIC design market exists, is large, and is growing — validating the category. Broadcom and Marvell together serve a market large enough for both to grow rapidly. The risk is if one company takes disproportionate share in new customer wins (e.g., if Marvell wins Microsoft Maia). Track new XPU customer announcements for both companies as the leading indicator
Marvell's lower P/E makes it the better investment vehicle for the same ASIC design thesis — Broadcom is overpriced relative to Marvell for equivalent exposure
Primary sources
Broadcom FY2024 Earnings Call Transcript — Hock Tan's $60–90B per-customer XPU TAM statement, AI revenue breakdown, and guidance. The most important primary source for Broadcom investors. Re-read every quarter.
Meta Engineering Blog — MTIA v1 — Meta's public description of their custom AI chip. Architecture, use cases, and Broadcom co-design relationship (implied).
Ultra Ethernet Consortium — The standards body Broadcom leads. Technical specifications for Ethernet RDMA for AI clusters. The clearest statement of Broadcom's networking strategy.
What comes next
Lesson 11: Scaling laws — the Chinchilla result, compute-optimal training, and why model size, data, and compute form a mathematically predictable relationship that drives every hyperscaler's chip purchasing decision
Ask me anything. Good follow-ups as a Broadcom investor:
"Walk me through Broadcom's AI revenue model — is it more recurring (royalty) or lumpy (NRE)?" ·
"What happens to Broadcom's networking business if NVIDIA's NVLink becomes the standard for all AI clusters instead of Ethernet?" ·
"How should I think about Broadcom vs. Marvell — different customer exposure, different risk profiles?" ·
"Model the impact of losing ByteDance revenue on Broadcom's FY2026 EPS" ·
"What signals in quarterly earnings tell me whether the XPU program is executing on Hock Tan's guidance?"