principles.fyi · the brain · concept

cosine similarity

How much two vectors point the same way, ignoring their size.

cos_sim(a, b) = (a . b) / (|a| * |b|)

Cosine similarity is the dot product of two vectors after dividing out both their lengths, so all that's left is the angle between them: +1 means same direction, 0 means at right angles, -1 means opposite. It works because meaning here lives mostly in direction, not size — two word vectors can point the same way whether they're long or short. (Transformer attention is a cousin but keeps the lengths, so size still matters there.)