principles.fyi · the brain · concept

sequence classification

Reading a whole piece of text and tagging it with one label.

label = softmax(W · [CLS]-vector)

Sequence classification is assigning a single label to an entire input — positive/negative for a review, spam/not-spam for an email, a topic for an article. With an encoder you do it by taking the [CLS] token's summary vector and running it through a small classifier head that outputs logits over the labels, then softmax (both straight from Book 01). It's the simplest and most common way encoders get put to work.