principles.fyi · the brain · concept

cloze task

A fill-in-the-blank test, like 'The ___ chased the cat.'

"The ___ barked." -> fill the blank

A cloze task is the classic fill-in-the-blank: delete a word from a sentence and have someone (or a model) supply what belongs there. Masked language modeling is essentially a cloze task run at massive scale — the name comes from psychology, where cloze tests measured reading comprehension. It works as a training signal because guessing the missing word well requires actually understanding the sentence around the gap.