principles.fyi · the brain · concept

generative AI

AI that produces new content — text, images, audio — rather than just labeling or scoring existing input.

prompt -> newly generated tokens

Generative AI refers to models whose output is freshly created content rather than a category or a number. For language, that's the decoder's job: sampling token after token to write text that never existed before. It contrasts with the discriminative side (an encoder classifying a review as positive or negative). The whole reason a decoder can generate is that it was trained to predict the next token, so running that prediction in a loop produces original sequences. This book is about generative text models specifically — the chat LLMs.