What It Costs to Compose, Rebuild, and Correct Precomputed Memory

Language models can answer from precomputed memory, a model's saved reading of a body of material, reused across requests instead of read again at each.

Language models can answer from precomputed memory, a model's saved reading of a body of material, reused across requests instead of read again at each. This paper maps where that practice preserves correctness and the conditions under which it fails. Across experiments on…