PagedAttention: Paged Attention Score
Medium · Solved in PyTorch · Machine learning coding practice
Problem
Implement the core computation from PagedAttention (Kwon et al, 2023), the technique used in inference systems like vLLM.
In PagedAttention, the Key and Value caches are not stored as contiguous sequences. Instead, they are fragmented across fixed-size physical blocks to enable efficient memory management.
Topics: transformer, memory-management, vllm, inference
The full statement, worked examples, hints and the test suite are available once you sign in. You can then solve PagedAttention: Paged Attention Score in the browser and run it against the tests.