principles.fyi · the brain · concept

matrix multiply

Multiply two grids of numbers by dot-producting each row of the first against each column of the second.

C[i,j] = sum_k A[i,k] * B[k,j]

A matrix is just a grid of numbers, and to multiply two of them you slide each row of the first across each column of the second, multiply the lined-up pairs, and add them up — that single sum-of-products is one number in the answer grid. It matters because that dot product roughly captures how related two lists of numbers are: when they point the same way and aren't tiny, the score is big (it reflects both direction and size — measuring direction alone is cosine similarity, the same thing divided by the two lengths). So one multiply scores thousands of "how related are these?" questions at once. In a transformer this is nearly everything: the projections that build the query, key, and value vectors, letting tokens compare and attend to each other, and reshaping those vectors layer after layer in the feed-forward blocks — all the same row-times-column move, repeated at scale. (A few steps aren't matmuls — softmax, layer-norm, and turning a token into its first vector, which is really a lookup in an embedding table.)