The field’s word for the training run itself, in which a model learns from its data, such as human text. Post-training is the work done on the model after it, such as reinforcement learning. Chapter 2.
From The AI Compute Race. Read the book
Where the book uses it
- Chapter 2 — The Discovery That Changed Everything
How a formula about compute and capability became a bill: the scaling laws, the first supercomputers built to test them, and why the money came before the paper.
Sources that use it
- Jan Kocoń and 98 co-authors of the PLLuM consortium (Wrocław University of Science and Technology, NASK, OPI, IPI PAN, IS PAN and the University of Łódź), “PLLuM: A Family of Polish Large Language Models,” arXiv 2511.03823, version 1 of 5 November 2025 (“The project received funding from the Ministry of Digital Affairs in 2023, and full-scale development began in early 2024”; “Pre-training experiments were conducted on the WCSS LEM cluster, which features NVIDIA 304 H100 GPUs, as well as the Cyfronet Helios cluster, which features 440 NVIDIA GH200 GPUs”; Table 19, every publicly released model “Based On” Llama 3.1, Mistral-Nemo-Base-2407 or Mixtral-8x7B-v0.1). In the Sources
Related terms
From the glossary of The AI Compute Race.
How to cite this page
Dryburgh, Lee S. “Pre-training.” In The AI Compute Race. DRYBURGH. https://dryburgh.com/vol1/glossary/pre-training (accessed 2026-11-01).
BibTeX
@misc{dryburgh-pre-training-2026,
author = {Dryburgh, Lee S.},
title = {{Pre-training}},
howpublished = {In \emph{The AI Compute Race}, DRYBURGH},
year = {2026},
url = {https://dryburgh.com/vol1/glossary/pre-training},
note = {Accessed 2026-11-01}
}