---
title: "KV cache: what it means · The AI Compute Race · DRYBURGH"
source: https://dryburgh.com/vol1/glossary/kv-cache
description: "KV cache: A served model's short-term memory of the conversation, filled by reading the whole prompt at once."
---

# KV cache

What does “KV cache” mean?

KV cacheA served model’s short-term memory of the conversation, filled by reading the whole prompt at once. Filling it accounts for the pause before the first word appears, the time to first token. *Chapter 14.*

From *The AI Compute Race*. [Read the book](https://dryburgh.com/vol1#buy)

## Where the book uses it

- [Chapter 14 — The Convergence](https://dryburgh.com/vol1/notes/chapter-14)One campus, Meta’s Hyperion, run through four possible futures to show how science, power, money and permission meet in the price of an accelerator-hour.

## Sources that use it

- DeepSeek-AI, “DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression,” technical report, posted to the model’s Hugging Face repository 10 September 2026 (the laboratory’s own report on its own model, and the four-point per-token cache series that circulates with it, 389,120 to 48,068 to 3,514 to 890, is read off Figure 1(b) and appears nowhere in the paper’s text; the report carries no date on its face, the repository metadata giving createdAt 2026-09-10 02:17 UTC and the PDF’s own file metadata 2026-09-10 05:49 UTC; Abstract: these designs “reduce its global KV cache footprint (always in HBM) to 890 bytes per token, roughly 1/4 of the corresponding footprint of DeepSeek-V4-Flash”; Figure 1(b) caption: “approximately 4-fold and 437-fold reductions in per-token global KV cache size relative to DeepSeek-V4-Flash and DeepSeek-V1, respectively”; the report prints no price and no dollar sign, and makes no hardware, export-control or national claim; its architecture overview: “Overall, DeepSeek-V4.1-Flash has 552B backbone parameters and 196B Engram parameters, activating 8B parameters per token during prefill and 16B during decode”; section 2.4.2, of Engram: “During inference, deterministic addressing enables embeddings to be prefetched from host memory via background RDMA transfers, with prefetching for the first module overlapping computation in the first Transformer block”). [In the Sources](https://dryburgh.com/vol1/sources/research#deepseek-v41-flash-paper)

## Related terms

- [Time to first token](https://dryburgh.com/vol1/glossary/time-to-first-token)
- [Accelerator-hour](https://dryburgh.com/vol1/glossary/accelerator-hour)
- [B200](https://dryburgh.com/vol1/glossary/b200)
- [Break fee](https://dryburgh.com/vol1/glossary/break-fee)
- [Break-point](https://dryburgh.com/vol1/glossary/break-point)
- [Bridge power](https://dryburgh.com/vol1/glossary/bridge-power)
- [Capital recovery](https://dryburgh.com/vol1/glossary/capital-recovery)
- [Clearing house](https://dryburgh.com/vol1/glossary/clearing-house)
- [Commodity infrastructure](https://dryburgh.com/vol1/glossary/commodity-infrastructure)
- [Compute futures](https://dryburgh.com/vol1/glossary/compute-futures)
- [Deemed export](https://dryburgh.com/vol1/glossary/deemed-export)
- [Distressed asset](https://dryburgh.com/vol1/glossary/distressed-asset)

From the glossary of The AI Compute Race.

- [Campuses](https://dryburgh.com/vol1/ledger)
- [Tests](https://dryburgh.com/vol1/tests)
- [Scorecard](https://dryburgh.com/vol1/scorecard)
- [Data](https://dryburgh.com/vol1/data/releases)
- [Chapter by chapter](https://dryburgh.com/vol1/companion)
- [The record at a glance](https://dryburgh.com/vol1/observatory)
- [The machine](https://dryburgh.com/vol1/machine)
- [One campus, four futures](https://dryburgh.com/vol1/hyperion)
- [Glossary](https://dryburgh.com/vol1/glossary)
- [People](https://dryburgh.com/vol1/people)
- [Sources](https://dryburgh.com/vol1/sources)
- [Notes](https://dryburgh.com/vol1/notes)
- [Where the book was wrong](https://dryburgh.com/vol1/corrections)
- [Where its sources were wrong](https://dryburgh.com/corrections?book=vol1)
- [The Paper](https://dryburgh.com/vol1/paper)
- [Consent](https://dryburgh.com/vol1/consent)
- [Who builds and pays](https://dryburgh.com/vol1/parties)

*The AI Compute Race: Five Races, One Machine* is a book by Lee S. Dryburgh, the first volume of THE COMPUTE ECONOMY, published by Dryburgh, an imprint of Contestra OÜ, Tallinn, Estonia; its record closes on 8 November 2026.

[About the book](https://dryburgh.com/vol1)[Tell me when it is out](https://dryburgh.com/vol1#notify)