How a word becomes numbers

A model never sees letters. It sees page numbers in a giant dictionary.

text

pick a word above

pieces → ids → vectors

id = a row in a ~50,000-row table → that row IS the vector

The model reads dictionary page-numbers, not letters — which is exactly why it miscounts the r’s in “strawberry.”

Real tokenizers (BPE / SentencePiece) learn ~50k pieces. This is a faithful fake.