PagedAttention

The KV blocks and block tables lesson left the scheduler holding one ordered row of block numbers per request, its entries free to name blocks anywhere in the pool and in any order. A row of numbers is bookkeeping; attention is computed over memory. This lesson is about the place where the one becomes the other.

In this lesson

• The mapping layer
• Two fields on one carrier
• Reading from the pool
• Writing into the pool
• One design, several kernels

Continue learning

Pro unlocks the complete lesson, video, practice, and code where available.