principles.fyi · the brain · concept

word sense

One of the distinct meanings a single word can have.

"bank" = {river bank, money bank, ...}

A word sense is a specific meaning of a word — 'bank' has a river-edge sense and a money-place sense, and these are genuinely different concepts that happen to share a spelling. Plain Book-01 embeddings give a word just one vector, which has to blur all its senses together; contextual embeddings fix this by letting the surrounding words pin down which sense is active. Distinguishing senses is one of the clearest things an encoder buys you over a static lookup table.