principles.fyi · The Brain

The Brain

Every concept in the library, wired by the articles that share them. It's dark when you arrive — reading lights it up. Click any node.

…a line = two concepts in the same article · node size = how many articles use it

Every concept, A–Z

anisotropy attention autoregression backpropagation base model Bayes' rule BERT bidirectional BIO tagging Bradley-Terry model causal mask chain of thought compute-optimal scaling cloze task [CLS] token Common Crawl conditional generation context window contextual embedding cosine similarity cross-entropy loss data contamination decoder denoising derivative dot product DPO e embedding encoder encoder-decoder entropy expected value exponential few-shot feed-forward network fine-tuning foundation model GeLU generative AI grouped-query attention gradient descent greedy decoding hallucination hidden state in-context learning induction head instruction tuning KL divergence KV cache layer norm learning rate logarithm log-odds logit lens logits LoRA [MASK] token masked language modeling matrix multiply mean min-p sampling mixture of experts MMLU model alignment multi-head attention named entity recognition natural language inference next sentence prediction nonlinearity odds parameters repetition / presence / frequency penalty perplexity policy polysemy positional encoding post-training power law PPO preference alignment preference data pretraining prior probability prompt prompt engineering quantization query, key, value RL for reasoning reference policy ReLU representational harm residual stream reward reward hacking reward model RLHF RMSNorm sampling scaling laws segment embedding [SEP] token sequence classification sigmoid softmax sparse autoencoder standard deviation superposition gated FFN sycophancy system prompt temperature test-time compute The Pile token top-k sampling top-p transfer learning unembedding vector vocabulary weight tying weights word sense word-sense disambiguation zero-shot