principles.fyi · the brain · concept

word-sense disambiguation

Figuring out which meaning of a word is intended from its context.

(word, sentence) -> which sense

Word-sense disambiguation (WSD) is the task of picking the correct sense of an ambiguous word given its sentence — deciding that 'bass' means the fish, not the instrument, in 'I caught a bass'. Contextual embeddings make this almost natural: because attention pulls in the surrounding words, the encoder's vector for the word already leans toward the intended sense, so a small classifier on top can read it off. It's a direct, concrete payoff of reading both directions.