PagedAttention: Shared Prefix Mapping
Medium · Solved in Python · Machine learning coding practice
Problem
In PagedAttention(Kwon et al, 2023), the KV cache is divided into fixed-size physical blocks. When multiple requests share a common "prefix" (like claude code/codex can have very long system prompt), we can optimize memory by mapping their logical blocks to the same physical blocks.
Topics: inference, vllm, attention, memory-management
The full statement, worked examples, hints and the test suite are available once you sign in. You can then solve PagedAttention: Shared Prefix Mapping in the browser and run it against the tests.