--- title: "Inference: what it means · The AI Compute Race · DRYBURGH" source: https://dryburgh.com/vol1/glossary/inference description: "Inference: Running a trained model for the people who use it. Training builds a model, and inference runs it. Chapter 1." --- # Inference What does “Inference” mean? InferenceRunning a trained model for the people who use it. Training builds a model, and inference runs it. *Chapter 1.* From *The AI Compute Race*. [Read the book](https://dryburgh.com/vol1#buy) ## Where the book uses it - [Chapter 1 — The Stakes](https://dryburgh.com/vol1/notes/chapter-1)What the race is for: the belief that a lead in AI can compound into a lasting advantage, how long such leads have actually lasted, and what a trillion dollars a year is being bet against. ## Where the record uses it - Person [Dally, Bill](https://dryburgh.com/vol1/people#dally-bill) ## Sources that use it - Groq, “Groq and Nvidia Enter Non-Exclusive Inference Technology Licensing Agreement to Accelerate AI Inference at Global Scale,” 24 December 2025 (“Jonathan Ross, Groq’s Founder, Sunny Madra, Groq’s President, and other members of the Groq team will join Nvidia”; “GroqCloud will continue to operate without interruption”; no price disclosed); the $20 billion: CNBC, 24 December 2025 (“for $20 billion in cash, according to Alex Davis, CEO of Disruptive, which led the startup’s latest financing round in September” — the investor’s figure, not NVIDIA’s; Jensen Huang: “we are not acquiring Groq as a company”; Groq founder and CEO Jonathan Ross along with Sunny Madra, the company’s president, and other senior leaders “will join Nvidia to help advance and scale the licensed technology”). [In the Sources](https://dryburgh.com/vol1/sources/company#groq-nvidia) - Groq, “Groq Closes $350 million Series A, Building the World’s Leading AI Inference Cloud,” 17 August 2026 (a $350 million Series A at a $3.5 billion valuation, led by Disruptive “with planned participation from NVIDIA”; “This latest round, together with $650 million raised in June 2026, brings recent funding in the company to $1 billion”; “Groq is an NVIDIA Cloud Partner (NCP), certified to design, deploy, and operate NVIDIA accelerated computing”; “Groq today operates 13 data centers across North America, Europe, the Middle East, and Asia Pacific”). [In the Sources](https://dryburgh.com/vol1/sources/company#groq-series-a-aug26) - Groq, “Groq Raises $650M to Scale Its AI Inference Cloud Business,” 22 June 2026 (“$650 million in new growth capital to accelerate the expansion of its AI inference cloud”; “The round was led by Disruptive and Infinitum, with participation from investors who elected to reinvest in the company”). [In the Sources](https://dryburgh.com/vol1/sources/company#groq-650m-jun26) - Moonshot AI, “Kimi K3 Tech Blog: Open Frontier Intelligence,” Kimi blog, July 2026 (“Pricing is $0.30/MTok for cache-hit input, $3.00/MTok for cache-miss input, and $15.00/MTok for output”, that is, $3 and $15 per million tokens of input and output; the post prints no date of its own, gives its benchmark scores as “current as of July 16, 2026” and promises the full weights “by July 27, 2026”; Moonshot’s API price card, “Model Inference Pricing Explanation,” Kimi API Platform, listed the same $0.30, $3.00 and $15.00 for kimi-k3 on 20 September 2026, with cache-write prices of $3.00 and $6.00 that the post does not give; read live 20 September 2026). [In the Sources](https://dryburgh.com/vol1/sources/company#kimi-k3-pricing) - Andrew Feldman, co-founder and chief executive of Cerebras, interviewed by Matt Turck, “Cerebras CEO: Why GPUs Can’t Do Fast Inference,” *The MAD Podcast with Matt Turck*, FirstMark, YouTube, 23 July 2026 (a seller of the wafer-scale alternative to the GPU, speaking of his own product and of his competitors’ supply, with no independent benchmark in the interview; of the memory made by three companies, “they’re sold out. That’s the number one problem. We don’t use it” 19:41–19:46; of TSMC’s CoWoS packaging, “That process is sold out, is highly constrained. We don’t use it” 20:38–20:45; “we don’t use the 3-nanometer technology” 21:00, his chips being at 5 nanometers; of the push for more CPUs, “is being driven by AI on machines like ours and GPUs doing agentic work” 24:14–24:22; of datacenter availability, “is a limiting factor for everybody” 55:58; of the deal with OpenAI, “because data center capacity is a limiting constraint” 56:16–56:20, measured in megawatts; “It takes you eight keystrokes to move from a GPU to us in the cloud” 1:02:20–1:02:26, after his sentence that inference has no moat; of TSMC at 1:04:04, close collaboration, an investor in the company and an exceptional partner, paraphrased; of diversifying away from the foundry at 1:08:26, very hard, since part of designing a chip is designing for the foundry’s rules, paraphrased; the quotations read against two transcriptions of the audio held by the set, which carry three words the automatic caption drops, the word moat taken on the term and the speaker’s use of it elsewhere in the interview where the caption and both transcriptions render it otherwise, the 3-nanometer clause held on the caption and the sentences either side of it, the foundry’s name at 1:04:04 the host’s, TSMC on both transcriptions of 3 October 2026, and at 1:08:26 not heard cleanly and so not quoted). [In the Sources](https://dryburgh.com/vol1/sources/interview#feldman-mad-jul26) And 8 more: [search the Sources](https://dryburgh.com/vol1/sources). ## Related terms - [Training](https://dryburgh.com/vol1/glossary/training) - [Gross margin](https://dryburgh.com/vol1/glossary/gross-margin) - [Reasoning model](https://dryburgh.com/vol1/glossary/reasoning-model) - [Algorithmic efficiency](https://dryburgh.com/vol1/glossary/algorithmic-efficiency) - [Application layer](https://dryburgh.com/vol1/glossary/application-layer) - [Arsenal behind every arsenal](https://dryburgh.com/vol1/glossary/arsenal-behind-every-arsenal) - [Capital expenditure](https://dryburgh.com/vol1/glossary/capital-expenditure) - [Clocks of diffusion](https://dryburgh.com/vol1/glossary/clocks-of-diffusion) - [Cluster](https://dryburgh.com/vol1/glossary/cluster) - [Epoch AI](https://dryburgh.com/vol1/glossary/epoch-ai) - [Exclusive generation](https://dryburgh.com/vol1/glossary/exclusive-generation) - [Exclusivity of the model](https://dryburgh.com/vol1/glossary/exclusivity-of-the-model) From the glossary of The AI Compute Race. - [Campuses](https://dryburgh.com/vol1/ledger) - [Tests](https://dryburgh.com/vol1/tests) - [Scorecard](https://dryburgh.com/vol1/scorecard) - [Data](https://dryburgh.com/vol1/data/releases) - [Chapter by chapter](https://dryburgh.com/vol1/companion) - [The record at a glance](https://dryburgh.com/vol1/observatory) - [The machine](https://dryburgh.com/vol1/machine) - [One campus, four futures](https://dryburgh.com/vol1/hyperion) - [Glossary](https://dryburgh.com/vol1/glossary) - [People](https://dryburgh.com/vol1/people) - [Sources](https://dryburgh.com/vol1/sources) - [Notes](https://dryburgh.com/vol1/notes) - [Where the book was wrong](https://dryburgh.com/vol1/corrections) - [Where its sources were wrong](https://dryburgh.com/corrections?book=vol1) - [The Paper](https://dryburgh.com/vol1/paper) - [Consent](https://dryburgh.com/vol1/consent) - [Who builds and pays](https://dryburgh.com/vol1/parties) *The AI Compute Race: Five Races, One Machine* is a book by Lee S. Dryburgh, the first volume of THE COMPUTE ECONOMY, published by Dryburgh, an imprint of Contestra OÜ, Tallinn, Estonia; its record closes on 8 November 2026. [About the book](https://dryburgh.com/vol1)[Tell me when it is out](https://dryburgh.com/vol1#notify)