principles.fyi · the brain · concept

decoder

A transformer that reads left-to-right and writes the next word — the model from Book 01 that you chat with.

decoder(tokens so far) -> next-token probabilities

A decoder is a stack of transformer blocks built for generation: it reads the text so far, left-to-right, and predicts the next token, then appends it and repeats (autoregression). Its defining feature is the causal mask — each position can attend only to itself and earlier tokens, never the future — which is exactly what lets it be trained on, and used for, next-word prediction. This is the architecture of Book 01 and of essentially every model you chat with today (GPT, Llama, Claude). 'Decoder-only' just means the whole model is this single stack, with no separate reading half.