ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression

KV cache compression is essential for efficient long-context inference.

KV cache compression is essential for efficient long-context inference. Existing eviction methods permanently discard unselected tokens and consequently remove their aggregate contribution to attention. Merging-based alternatives preserve more information but can perturb…