---
title: "Training: what it means · The AI Compute Race · DRYBURGH"
source: https://dryburgh.com/vol1/glossary/training
description: "Training: Building a model from its data, as against inference, which runs it. A training run is one such computation, and a training cluster is built for it."
---

# Training

What does “Training” mean?

TrainingBuilding a model from its data, as against inference, which runs it. A training run is one such computation, and a training cluster is built for it. *Chapter 1.*

From *The AI Compute Race*. [Read the book](https://dryburgh.com/vol1#buy)

## Where the book uses it

- [Chapter 1 — The Stakes](https://dryburgh.com/vol1/notes/chapter-1)What the race is for: the belief that a lead in AI can compound into a lasting advantage, how long such leads have actually lasted, and what a trillion dollars a year is being bet against.

## Where the record uses it

- Campus [Colossus 2](https://dryburgh.com/vol1/ledger/colossus-2)
- Campus [Colossus 1](https://dryburgh.com/vol1/ledger/colossus-1)
- Scorecard indicator [Largest evidenced operating site](https://dryburgh.com/vol1/scorecard#row-11)
- Dated test [Chapter 2, the ladder of cost, due 31 December 2027](https://dryburgh.com/vol1/tests#2027-11)
- Dated test [Chapters 1 and 12, the first route, due 31 August 2028](https://dryburgh.com/vol1/tests#2028-1)

## Sources that use it

- OpenAI, “How AI Training Scales,” Sam McCandlish, Jared Kaplan and Dario Amodei, 14 December 2018 (gradient-noise scale as a predictor of useful batch size and parallelism; treats bulk compute as proportional to dollar cost; the public methodological precursor to the January 2020 scaling-laws paper, thirteen months before it). [In the Sources](https://dryburgh.com/vol1/sources/company#openai-training-scales)
- xAI, “Colossus: The World’s Largest AI Supercomputer,” accessed 29 August 2026 and, by Wayback capture, 9 September 2026 (the page gives 122 days, “Then we doubled it in 92 days to 200k GPUs” and 276 days from groundbreak, and prints 200,000 over “H100 GPUs in a single interconnected cluster”; its counter further down is an animation that the page’s script, as served on 30 September 2026, runs from 180 to 200 once half its tile is in view, so the 180,000 read in the unscrolled markup at the close was its first frame, and the page scrolled into view on 12 October 2026 showed 200K; the page prints no 100,000 and no Megapack count; the script counts its days from groundbreak from an origin it writes as 1 May 2024, the first of a month like each of its seven milestones, and of those milestones, drawn only when a reader presses a point on the timeline, the one of 1 January 2025 reads “Training finished with 80K+ GPUs. 122 days from groundbreak to training.” and the one titled “Training begins”, of 1 September 2024, reads “42K-GPU job with elastic training, running across 4 data halls.”; the timeline’s own data dates the doubling 1 February 2025, its axis ending Feb 2025, and 1 May 2024 to 1 February 2025 is the page’s 276 days, though the page’s script prints that date in the reader’s own time zone, so a browser west of UTC shows the milestone as “January 2025”; re-read through Browserbase on 27 September 2026, a plain fetch refused by Cloudflare: the same 122, 92 and 276 days, the axis still ending Feb 2025; the milestone card showed the earlier month on a live read of 12 September 2026 and on that Browserbase read, and the later month on the Wayback capture of 20 August 2026; the page’s timeline script read whole on 30 September 2026; the page’s lead: “Built in 122 days — outpacing every estimate — Colossus was the most powerful AI training system yet”). [In the Sources](https://dryburgh.com/vol1/sources/company#xai-colossus)
- Musk v. Altman, No. 4:24-cv-04722-YGR (N.D. Cal.): Microsoft Corporation’s Proposed Findings of Fact as to Liability, 27 April 2026, Dkt. 504 (para. 112: “He made the original call to Microsoft’s CEO to ask if Microsoft would donate Azure credits to OpenAI. Microsoft answered that call and provided OpenAI with compute.”; para. 3: “the company could not raise sufficient funds from donations alone”; para. 27: “Training LLMs at the scale that OpenAI envisioned would require custom supercomputers” and “massive amounts of capital to finance that build”; para. 5: “in July 2019, Microsoft agreed to make an initial $1 billion investment and build the custom supercomputers that OpenAI needed”). [In the Sources](https://dryburgh.com/vol1/sources/filing#msft-findings-apr26)
- Amin Vahdat (Google’s vice-president and general manager of AI and infrastructure) on Sequoia Capital’s Training Data podcast, “Google’s AI Infrastructure Chief on the Physics & Economics of Frontier AI,” published 6 October 2026 (at 52:24 to 53:01, asked about his 2025 remark on the old TPUs: “I didn’t mean for it to be a meaningful statement”; “our older TPUs are seeing significant utilization. We do replace them”; “depreciation lifetime is approximately 6 years”; “once they’re fully depreciated and given the power efficiency of newer generations etc it does make sense to replace them and upgrade them”; at 44:03, on siting: “our preferred model at Google is always to be utility connected”; the words read against the tape at two decoders on 6 October 2026; the host’s figures for Google’s capex and serving capacity are the host’s, not his; “but it was a meaningful statement, apparently”; on goodput, asked whether it is a Google term or an industry term, “It’s a Google term” at 10:27, continuing “but I think more and more people across the industry are starting to pick it up” on the smaller of the two decoders, the larger omitting “the” before industry, so the one word is unsettled between them). [In the Sources](https://dryburgh.com/vol1/sources/interview#sequoia-vahdat-oct26)
- Action News 5 (Memphis), “xAI Training Begins in Memphis,” 22 July 2024 (Musk: first training run at 4:20 a.m.; on Musk’s post embedded in the article: “the most powerful AI training cluster in the world” (his post on X of 22 July 2024, set below the article’s opening lines)). [In the Sources](https://dryburgh.com/vol1/sources/press#actionnews-training-jul24)

And 11 more: [search the Sources](https://dryburgh.com/vol1/sources).

## Related terms

- [Inference](https://dryburgh.com/vol1/glossary/inference)
- [Cluster](https://dryburgh.com/vol1/glossary/cluster)
- [122 days](https://dryburgh.com/vol1/glossary/122-days)
- [Brake, the](https://dryburgh.com/vol1/glossary/brake-the)
- [Checkpoint](https://dryburgh.com/vol1/glossary/checkpoint)
- [Firm power](https://dryburgh.com/vol1/glossary/firm-power)
- [Parameter](https://dryburgh.com/vol1/glossary/parameter)
- [Pre-training](https://dryburgh.com/vol1/glossary/pre-training)
- [Reinforcement learning](https://dryburgh.com/vol1/glossary/reinforcement-learning)
- [Scaling laws](https://dryburgh.com/vol1/glossary/scaling-laws)
- [Algorithmic efficiency](https://dryburgh.com/vol1/glossary/algorithmic-efficiency)
- [Application layer](https://dryburgh.com/vol1/glossary/application-layer)

From the glossary of The AI Compute Race.

- [Campuses](https://dryburgh.com/vol1/ledger)
- [Tests](https://dryburgh.com/vol1/tests)
- [Scorecard](https://dryburgh.com/vol1/scorecard)
- [Data](https://dryburgh.com/vol1/data/releases)
- [Chapter by chapter](https://dryburgh.com/vol1/companion)
- [The record at a glance](https://dryburgh.com/vol1/observatory)
- [The machine](https://dryburgh.com/vol1/machine)
- [One campus, four futures](https://dryburgh.com/vol1/hyperion)
- [Glossary](https://dryburgh.com/vol1/glossary)
- [People](https://dryburgh.com/vol1/people)
- [Sources](https://dryburgh.com/vol1/sources)
- [Notes](https://dryburgh.com/vol1/notes)
- [Where the book was wrong](https://dryburgh.com/vol1/corrections)
- [Where its sources were wrong](https://dryburgh.com/corrections?book=vol1)
- [The Paper](https://dryburgh.com/vol1/paper)
- [Consent](https://dryburgh.com/vol1/consent)
- [Who builds and pays](https://dryburgh.com/vol1/parties)

*The AI Compute Race: Five Races, One Machine* is a book by Lee S. Dryburgh, the first volume of THE COMPUTE ECONOMY, published by Dryburgh, an imprint of Contestra OÜ, Tallinn, Estonia; its record closes on 8 November 2026.

[About the book](https://dryburgh.com/vol1)[Tell me when it is out](https://dryburgh.com/vol1#notify)