One of the adjustable numbers inside a network that training sets, and the unit a model’s size is counted in. The weights are their trained values. Chapter 1.
From The AI Compute Race. Read the book
Where the book uses it
- Chapter 1 — The Stakes
What the race is for: the belief that a lead in AI can compound into a lasting advantage, how long such leads have actually lasted, and what a trillion dollars a year is being bet against.
Sources that use it
- Aakanksha Chowdhery et al., “PaLM: Scaling Language Modeling with Pathways,” arXiv:2204.02311, v1 submitted 5 April 2022, with Google Research’s announcement post of 4 April 2022 (the abstract: “a 540-billion parameter, densely activated, Transformer language model”, trained “on 6144 TPU v4 chips using Pathways”; the announcement post is titled “Pathways Language Model (PaLM): Scaling to 540 Billion Parameters for Breakthrough Performance”; v1’s reference list cites Hoffmann et al., “Training compute-optimal large language models”, arXiv:2203.15556, 2022; arXiv lists five versions, v1 of 5 April 2022 through v5 of 5 October 2022; Hoffmann appears 12 times in v1 and Chinchilla 17, against 29 in the version arXiv now serves, and the bibliography sets the title in sentence case, which a title-cased search misses). In the Sources
Related terms
From the glossary of The AI Compute Race.
How to cite this page
Dryburgh, Lee S. “Parameter.” In The AI Compute Race. DRYBURGH. https://dryburgh.com/vol1/glossary/parameter (accessed 2026-11-01).
BibTeX
@misc{dryburgh-parameter-2026,
author = {Dryburgh, Lee S.},
title = {{Parameter}},
howpublished = {In \emph{The AI Compute Race}, DRYBURGH},
year = {2026},
url = {https://dryburgh.com/vol1/glossary/parameter},
note = {Accessed 2026-11-01}
}