principles.fyi sources

principles.fyi · Post-training

Sources

Every factual claim in this topic traces back to one of these 27 sources — the textbook spine plus the primary papers.

  1. Jurafsky, D., & Martin, J. H. (2026). Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition with Language Models (3rd ed., draft of January 6, 2026). Stanford University. Ch. 9 (Post-training: Instruction Tuning, Alignment, and Test-Time Compute). pt. 1, 2, 3, 4, 5, 6, 7
  2. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., & Lowe, R. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35 (NeurIPS 2022). arXiv:2203.02155. pt. 1, 2, 3, 5
  3. Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. (2022). Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. arXiv:2204.05862. pt. 3, 5
  4. Bradley, R. A., & Terry, M. E. (1952). Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons. Biometrika, 39(3/4), 324-345. pt. 3, 4
  5. Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., & Finn, C. (2023). Direct Preference Optimization: Your Language Model is Secretly a Reward Model. Advances in Neural Information Processing Systems 36 (NeurIPS 2023). arXiv:2305.18290. pt. 6
  6. Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., & Irving, G. (2019). Fine-Tuning Language Models from Human Preferences. arXiv:1909.08593. pt. 4, 5
  7. Stiennon, N., Ouyang, L., Wu, J., Ziegler, D. M., Lowe, R., Voss, C., Radford, A., Amodei, D., & Christiano, P. (2020). Learning to Summarize from Human Feedback. Advances in Neural Information Processing Systems 33 (NeurIPS 2020). arXiv:2009.01325. pt. 4, 5
  8. Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: An Introduction. MIT Press. pt. 5
  9. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Advances in Neural Information Processing Systems 35 (NeurIPS 2022). arXiv:2201.11903. pt. 7
  10. Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., & Schulman, J. (2021). Training Verifiers to Solve Math Word Problems. arXiv:2110.14168. pt. 7
  11. Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q. V., Chi, E. H., Zhou, D., & Wei, J. (2023). Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them. Findings of ACL 2023. arXiv:2210.09261. pt. 7
  12. Wang, Y., Mishra, S., Alipoormolabashi, P., Kordi, Y., Mirzaei, A., Naik, A., Ashok, A., et al. (2022). Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks. EMNLP 2022. arXiv:2204.07705. pt. 2
  13. Longpre, S., Hou, L., Vu, T., Webson, A., Chung, H. W., Tay, Y., Zhou, D., Le, Q. V., Zoph, B., Wei, J., & Roberts, A. (2023). The Flan Collection: Designing Data and Methods for Effective Instruction Tuning. ICML 2023. arXiv:2301.13688. pt. 2
  14. Iyer, S., Lin, X. V., Pasunuru, R., Mihaylov, T., Simig, D., Yu, P., Shuster, K., Wang, T., Liu, Q., Koura, P. S., Li, X., et al. (2022). OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization. arXiv:2212.12017. pt. 2
  15. Mishra, S., Khashabi, D., Baral, C., & Hajishirzi, H. (2022). Cross-Task Generalization via Natural Language Crowdsourcing Instructions. ACL 2022. arXiv:2104.08773. pt. 2
  16. Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016). SQuAD: 100,000+ Questions for Machine Comprehension of Text. EMNLP 2016. arXiv:1606.05250. pt. 2
  17. Singh, S., Vargus, F., D'Souza, D., Karlsson, B. F., Mahendiran, A., Ko, W.-Y., Shandilya, H., Patel, D., Mataciunas, J., O'Mahony, L., Zhang, M., et al. (2024). Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning. arXiv:2402.06619. pt. 2
  18. Cui, G., Yuan, L., Ding, N., Yao, G., He, B., Zhu, W., Ni, Y., Xie, G., Xie, R., Lin, Y., Liu, Z., & Sun, M. (2024). UltraFeedback: Boosting Language Models with Scaled AI Feedback. ICML 2024. arXiv:2310.01377. pt. 3, 4
  19. Cao, Y., Kang, Y., Wang, C., & Sun, L. (2024). Instruction Mining: Instruction Data Selection for Tuning Large Language Models. First Conference on Language Modeling (COLM 2024). arXiv:2307.06290. pt. 4
  20. Ethayarajh, K., Choi, Y., & Swayamdipta, S. (2022). Stanford Human Preferences Dataset (SHP). pt. 3
  21. Lambert, N., Tunstall, L., Rajani, N., & Thrush, T. (2023). HuggingFace H4 Stack Exchange Preference Dataset. pt. 3
  22. Wang, Z., Dong, Y., Zeng, J., Adams, V., Sreedhar, M. N., Egert, D., Delalleau, O., Scowcroft, J., Kant, N., Swope, A., & Kuchaiev, O. (2024). HelpSteer: Multi-Attribute Helpfulness Dataset for SteerLM. NAACL HLT 2024. arXiv:2311.09528. pt. 3
  23. Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020). RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models. Findings of EMNLP 2020. arXiv:2009.11462. pt. 1
  24. Cheng, M., Durmus, E., & Jurafsky, D. (2023). Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models. ACL 2023. arXiv:2305.18189. pt. 1
  25. Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019). The Woman Worked as a Babysitter: On Biases in Language Generation. EMNLP 2019. arXiv:1909.01326. pt. 1
  26. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems 33 (NeurIPS 2020). arXiv:2005.14165. pt. 1
  27. Bianchi, F., Suzgun, M., Attanasio, G., Röttger, P., Jurafsky, D., Hashimoto, T., & Zou, J. (2024). Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions. ICLR 2024. arXiv:2309.07875. pt. 2