Generating token by token
Each row = a token doing attention. Each cell = one attention computation.
+ Generate token
← Back
KV cache: OFF
computed now
reused from cache
not yet