Attention work at a generation step
One cell is one allowed query–key score pair. Compare the work at a step with t visible tokens.
+ One visible token
← Back
KV cache: OFF
computed now
earlier query rows skipped
not yet