[CLS] token

A special slot added at the front whose final vector summarizes the whole input.

[CLS] x1 x2 ... xn -> use [CLS]'s final vector as the sentence summary

The [CLS] token (short for 'classification') is a special token prepended to every input; because attention lets it gather from all the other tokens, its top-layer vector becomes a single summary of the entire sequence. That summary is exactly what you feed to a classifier head for whole-sentence decisions like sentiment or topic — instead of pooling all the token vectors yourself, you just read [CLS]. It's the encoder's built-in 'sum it all up here' handle.

Appears in

Two sentences at once Masked Language Models · pt 4
Put it to work Masked Language Models · pt 5

Nearby in the brain

natural language inference BERT dot product next sentence prediction positional encoding segment embedding [SEP] token BIO tagging