MMLU is a multiple-choice test. The model scores each option as a next-token probability — the highest bar (amber) is its answer. We check it against the key. Cycle questions to see: accuracy is a proxy, not a guarantee.
Question
Toggle data contamination: if this exact question leaked into the model's training data, it can memorise the answer instead of reasoning. The bars spike toward the correct letter and the score climbs — but the model didn't get smarter, the test got easier. This is why a high MMLU number isn't always honest.