Nvidia, or the Repricing of a Watt
Tokens per watt, bottleneck inversion, and the Hopper–Blackwell arithmetic.
TL;DR: in seven lines
- Asked how he sees the AI industry, Huang draws a stack of five layers, with one specific twist: he separates chips and systems into distinct layers, when most analysts tend to merge the two into a single "compute" bucket.
- Huang calls Nvidia a software company whose product is manufactured by others.
- Nvidia's 75% gross margin is a historical peak, not a trend: the 10-year median is 62%, with a low of 57%.
- Data centres are queuing for grid capacity, while the energy infrastructure that feeds them generally takes longer to plan and build than the data centres themselves.
- The useful metric is the one attached to the resource that is limiting the next unit of deployment.
- Nvidia increasingly measures the product at rack and factory level because a growing share of realised performance depends on how the components interact.
- Huang's wager is several layers lower: even if individual components improve or commoditise, the work of making them operate efficiently together does not disappear.
When Dwarkesh Patel asks Jensen Huang to explain Nvidia in one sentence, in his recent podcast interview, Huang gives him this formula:
Input: electrons. Output: tokens. In the middle: Nvidia.
The formula sounds like a marketing line at first. Read carefully, it is also Huang’s bet on where AI value will accumulate over the next decade, and how Nvidia plans to keep capturing it.
The AI stack, and where Nvidia sits
Asked how he sees the AI industry, Huang draws a stack of five layers, with one specific twist: he separates chips and systems into distinct layers, when most analysts tend to merge the two into a single “compute” bucket.
Two things stand out from the diagram.
First, the stack is a relay race. Inside the chips and systems layers sit a sequence of handoffs that no single company controls. Nvidia1 designs the logic and writes the software, then sends a digital file to TSMC2 which prints the dies in Taiwan. SK Hynix3, Micron4 and Samsung make the HBM (high-bandwidth memory stacks bonded directly onto the package). An ODM in Taiwan5, typically Foxconn, Wistron, Quanta or Inventec, assembles the racks. A hyperscaler (AWS, Azure, Google Cloud, OCI) or a neocloud (CoreWeave, Nscale, Nebius) plugs them in. The model on top, GPT, Claude, Gemini, Llama, finally emits tokens.
Second, the choice to separate chips from systems is itself a strategic claim. Chips are (almost) commoditisable. AMD has chips. Broadcom designs ASICs6. SMIC7 fabricates them. Systems are not: they are the racks, the networking, the cooling, the software-hardware co-design that takes a chip and turns it into a thing that produces tokens at scale. By naming systems as a distinct layer, Huang is staking out the part of “compute” he thinks remains uncommoditisable, and where Nvidia’s investments in NVLink, NVSwitch, Mellanox and Spectrum-X are concentrated. Reading this stack tells you where Huang has decided to fight.
Nvidia, a software company (really?)
What Nvidia sells across the chain is the difficulty of turning a given watt into the largest possible number of valuable tokens. The silicon is incidental. Huang calls Nvidia a software company whose product is manufactured by others.
That framing should be read with caution. It is a very intentional financial reframing. Plenty of people on the sell side already argue Nvidia is overpriced; the more interesting move is to notice what the formula is doing to the analyst conversation. By describing himself as a software company, Huang is asking investors to ignore the capital intensity of his actual business and to anchor on software-company multiples.
And looking at the numbers, you could conclude he is right. Nvidia is a software company. The capex/revenue ratio is around 3%, against roughly 40% at TSMC and 25 to 35% at the hyperscalers. The balance sheet looks fabless. The margins look fabless. Gross margin sits at 75% in Q4 FY2026, against an industry median of 29%, an AMD at 45% and an Intel at 42%. By every standard ratio used in financial analysis, Nvidia behaves like a software company that happens to ship physical products.
But the framing is more fragile than it looks. Nvidia’s 75% gross margin is a historical peak, not a trend: the 10-year median is 62%, with a low of 57%. More importantly, the fabless model does not mean Nvidia is economically detached from industrial capacity. By July 2026, the company disclosed $279 billion of future supply and capacity commitments, primarily for memory and manufacturing capacity, up from $119 billion one quarter earlier8. These commitments are not capex and some can be adjusted or cancelled, but they show how much physical capacity Nvidia now has to coordinate without owning the factories itself. We return to that in the final entry of the series.
For now, accept the framing with the caution it deserves. The insight behind the formula stays: Nvidia’s core value proposition is transforming electrons into tokens. The question is whether that proposition will resist the same commoditisation pressure that is currently eroding the rest of the AI stack.
The thesis: bottleneck inversion
If AI is commoditising software, why would it spare the company that ships the silicon underneath? Huang’s reply:
The journey from watt to token has to be redone every year at a better cost.
Behind that sentence sits a thesis about where the binding constraint of AI is moving. For most of the last decade, compute availability dominated the conversation, and flops per dollar was the natural benchmark. Power was rarely the first constraint a software company had to think about. That is changing. Data centres are queuing for grid capacity, while the energy infrastructure that feeds them generally takes longer to plan and build than the data centres themselves9.
I would now use “bottleneck inversion” less literally than I did when I first wrote this piece. The chip has not stopped being scarce, and there is no single bottleneck shared by every project. HBM, advanced packaging, networking, cooling, power and grid connections can each become binding at different moments. The more useful idea is that the bottleneck moves.
When it moves toward power, tokens per watt becomes economically important for a simple reason: a fixed electrical envelope can support more useful computation if the whole system extracts more output from every megawatt.
From which three things follow.
1. The relevant benchmark depends on the constraint. Flops per dollar measures how cheaply you can perform arithmetic. Tokens per watt asks how much model output fits inside a power envelope. Neither supersedes the other. The useful metric is the one attached to the resource that is limiting the next unit of deployment.
The distinction matters because a system can be compute-constrained at one point, memory-constrained at another, and power-constrained when it reaches the data centre. The scarce input determines where another unit of optimisation has the highest value.
2. The advantage is not buyable off the shelf. A tokens-per-watt advantage does not live in any single component. It lives in how silicon, networking, libraries, compilers and model-level optimisations are co-designed. That co-design has to be redone, end to end, every generation. Money helps. Money does not skip the work, because the work is institutional knowledge accumulated across thousands of engineers over twenty years of CUDA development. We come back to this point in the next entry of the series, because it deserves much more than a sentence.
3. The gap widens with cycles, not narrows. Whether the moat is real depends on a single ratio: how much of the Hopper-to-Blackwell efficiency jump came from physics versus from system design. If system design dominates, the gap between Nvidia and the next contender compounds rather than erodes, and every additional product cycle deepens it.
Doing the arithmetic
Between Hopper and Blackwell10, Nvidia claimed very large improvements in inference efficiency on selected workloads. When I first wrote this piece, I tried to split that gain between lithography11 and “everything above the transistor”: architecture, memory, networking, kernels and software. I used a rough Moore’s-law estimate for the process contribution12 and assigned the remainder to system design.
That arithmetic was too neat.
Generation-to-generation efficiency does not decompose cleanly that way. Precision formats change. Memory bandwidth and capacity change. Model architectures and software change. Some gains only appear because several layers are redesigned together. Nvidia also chooses the workloads and comparison points in its own headline benchmarks.
The useful observation does not need the 17–30x estimate. Nvidia increasingly measures the product at rack and factory level because a growing share of realised performance depends on how the components interact. With Vera Rubin, the company now reports system-level comparisons such as agent throughput per megawatt and explicitly designs its DSX architecture across chips, networking, software, power, cooling and facility operations1314.
The narrower claim is enough: economically relevant performance is increasingly determined above the transistor as well as inside it.
If that narrower claim holds, three things follow:
1. Lithography is only one layer of the capability gap. US export controls can make frontier AI substantially harder and more expensive by restricting access to advanced manufacturing and accelerators. System-level optimisation does not erase that disadvantage. Worse silicon imposes costs in density, power and engineering that compound at scale.
What it does mean is that the final capability gap cannot be inferred from process nodes alone. China can still improve compilers, networking, cluster design, model efficiency and utilisation around the hardware it can obtain or manufacture. Export controls constrain the system; they do not determine every other layer of it. The policy question is therefore empirical: how much of the hardware disadvantage can be offset elsewhere, at what cost, and for which workloads?
2. Hyperscaler ASICs need more than good silicon to compete. The standard argument for why Google’s TPU or Amazon’s Trainium will eventually displace Nvidia is that they are tailored to specific workloads and built without Nvidia’s margin. The Hopper-to-Blackwell transition is a reminder that good silicon is only one part of the problem. The hard part is the software that turns the silicon into actual tokens per watt: kernels15, scheduling16, networking17, all the layers above transistor count. Nvidia engineers, when they get inside a partner’s stack, have publicly delivered 2x or more on a meaningful share of kernels and up to 4x throughput in joint optimisation efforts18. That gap is the dollar value of CUDA. An ASIC team that wants to match it has to rebuild twenty years of accumulated kernel-level optimisation expertise.
3. Vera Rubin and Feynman remain useful tests, but not of a clean “system design versus physics” ratio. Rubin is now in production19, and Nvidia’s early system-level results are directionally consistent with the thesis: the company is claiming large gains at rack and megawatt level, not simply from a faster standalone GPU13. The evidence is still largely Nvidia’s own and the workloads are changing, so it is too early to treat those comparisons as proof.
The better test is whether Nvidia can keep improving the economics of useful computation at system level as individual components become easier to substitute. Feynman, hyperscaler ASICs and independent operating data from Rubin deployments should make that easier to judge.
The standard commoditisation argument about AI tends to focus on models and applications, where capability diffuses quickly and prices can fall fast. Huang’s wager is several layers lower: even if individual components improve or commoditise, the work of making them operate efficiently together does not disappear.
If anything, denser racks, faster product cycles, more specialised accelerators and harder power constraints create more interfaces to optimise.
That is why the next paper moves away from the chip itself. If part of Nvidia’s advantage lives in coordination, what exactly has accumulated inside the organisation that lets it coordinate the system better than a well-funded competitor can?
“The journey from watt to token has to be redone every year at a better cost.”
Jensen Huang, on what resists commoditisation in AI
Footnotes
-
[Context] Nvidia designs the GPU architecture, the NVLink networking fabric, and the CUDA software stack. It does not own fabs. Founded 1993. Market cap roughly $4T at the time of the interview. What Nvidia sends to TSMC is a GDS2 file, the industry-standard digital description of a chip’s physical layout. ↩
-
[Context] Taiwan Semiconductor Manufacturing Company. Founded 1987. The world’s largest contract chip manufacturer, producing for Nvidia, Apple, AMD, Qualcomm and Broadcom. Runs the leading-edge nodes (N3, N2). Nvidia has been TSMC’s largest customer by revenue since 2024. ↩
-
[Context] SK Hynix. Korean memory maker. Dominant supplier of HBM3 and HBM3E to Nvidia. HBM (high-bandwidth memory) is stacked DRAM bonded directly to the GPU package, sitting next to the logic die rather than on a separate board. ↩
-
[Context] Micron. US memory maker. Third-largest supplier of HBM after SK Hynix and Samsung. Nvidia qualified Micron’s HBM3E in 2024 as part of a multi-sourcing strategy. ↩
-
[Context] ODM = Original Design Manufacturer. In the Nvidia supply chain, the four main ODMs (Foxconn, Wistron, Quanta, Inventec) are all based in Taiwan. They assemble GPUs, HBM, networking cards, power delivery and cooling into the finished rack, which then ships to hyperscalers or neoclouds. ↩
-
[Context] ASIC = Application-Specific Integrated Circuit. A chip designed for one workload (rather than a general-purpose GPU which can run many). In AI, the most prominent ASIC programmes are Google’s TPU, Amazon’s Trainium, Meta’s MTIA and Microsoft’s Maia. ASICs typically achieve better performance per watt on their target workload, in exchange for far less flexibility. ↩
-
[Context] SMIC = Semiconductor Manufacturing International Corporation. Shanghai-based, founded 2000, China’s largest contract chip foundry. Plays in China roughly the role TSMC plays globally, at one or two technology generations behind. Currently fabricates at 7nm using older DUV (deep ultraviolet) lithography with multi-patterning, since US export controls have blocked its access to ASML’s advanced EUV machines. Huawei’s most advanced AI chips, including the Ascend 910C, are made at SMIC. ↩
-
[Source] NVIDIA, Form 10-Q for the quarter ended July 26, 2026, Note 10, “Commitments and Contingencies.” Nvidia disclosed $279 billion of future supply and capacity commitments, primarily relating to memory and manufacturing capacity. The filing notes that some agreements may be cancellable, reschedulable or adjustable before firm orders are placed. SEC filing. ↩
-
[Source] International Energy Agency, Energy and AI, 2025. The IEA notes that a data centre can be operational in two to three years while the broader energy system generally requires longer planning and construction lead times. Its base case projects global data-centre electricity consumption at around 945 TWh in 2030. IEA. ↩
-
[Context] Hopper (H100, H200) is Nvidia’s GPU architecture released in 2022, named after Grace Hopper. It is the chip that trained most of the frontier models of 2023-2024 (GPT-4, Claude 3, Gemini). Blackwell (B100, B200, GB200), named after mathematician David Blackwell, is its 2024-2025 successor and the current top of Nvidia’s product line. The transition from one generation to the next is the basic unit on which Nvidia measures its own progress. ↩
-
[Context] Modern advanced chips are made in two coupled steps. First, ASML (Dutch company, near-monopoly) builds the lithography machines that print circuits onto silicon wafers using extreme ultraviolet light (EUV). Each machine costs $200-410 million. Second, TSMC (Taiwanese, around 90% of advanced foundry market) operates these machines inside its fabs and turns wafers into finished chips. Sub-7nm chips, including everything Nvidia ships today, require EUV. SMIC in China does not have access to ASML EUV machines (US export controls) and is therefore stuck on 7nm via older DUV multi-patterning techniques. ↩
-
[Context] A wafer is the round disc of silicon (typically 300mm in diameter today) onto which transistors are printed. A finished wafer is then cut into individual chips. Wafer cost, defect rates and processing time per layer are the three structural levers of chip economics. For more on the upstream physics of wafers and the constraints they impose on AI compute, see our deep dive: The Three Bottlenecks of AI Compute. ↩
-
[Source] NVIDIA, “Vera Rubin Ramps Into Full Production to Power Agentic AI Factories Worldwide,” May 31, 2026. Nvidia reports system-level agent-throughput gains for Vera Rubin relative to Grace Blackwell. Treat vendor benchmarks as directional until independent production data is available. NVIDIA. ↩ ↩2
-
[Source] NVIDIA, “NVIDIA DSX Gives Infrastructure Builders the Playbook for AI Factories,” May 31, 2026. DSX explicitly covers chips, systems, software, facilities and partner technologies, with MaxLPS software designed around token performance per megawatt. NVIDIA. ↩
-
[Context] A kernel is a small piece of code that runs on the GPU and performs one specific computation, such as a matrix multiplication or an attention operation. Models are built by chaining thousands of these kernels. The same model running on the same hardware can be two to ten times faster or slower depending on how the kernels are written: how memory is accessed, how parallelism is exploited, how data flows between the GPU and the rest of the system. ↩
-
[Context] Scheduling is the layer that decides when and where each kernel runs on the GPU, how memory is shared between concurrent tasks, and how compute is overlapped with data transfers. A modern GPU has thousands of cores running in parallel. A bad scheduler leaves them idle, and a single training run can be 30% to 50% slower than necessary. ↩
-
[Context] Networking here means how dozens, hundreds or thousands of GPUs communicate when training or serving a model that does not fit on a single GPU. Inside a rack, GPUs talk over NVLink (Nvidia’s proprietary high-bandwidth bus). Between racks, they use InfiniBand or Spectrum-X (Nvidia’s Ethernet-based AI fabric). A 1000-GPU cluster can run at 30% or 90% of theoretical efficiency depending on networking quality. ↩
-
[Caveat] Huang’s “2x” in the interview is a confident average. Independent benchmarks suggest the real picture is more nuanced. A recent Cursor / Nvidia collaboration on Blackwell kernel optimisation (October 2025) reported a 38% geometric mean speedup across 235 production kernels, with 2x or more achieved on 19% of them. A separate vLLM / Nvidia collaboration delivered up to 4x throughput improvement at similar latency between Hopper and Blackwell. The 2x figure is therefore a real upper bound on a meaningful share of workloads, more than a guaranteed average. ↩
-
[Context] Vera Rubin (named after the astronomer who discovered evidence of dark matter) and Feynman (named after the physicist Richard Feynman) are the two Nvidia generations that follow Blackwell. Vera Rubin entered full production in 2026; Feynman is expected in 2028. Each one represents a roadmap commitment Nvidia has already made publicly to its supply chain and customers. ↩
