LLMs in the Wild · Part 5 / 5
When it goes wrong
These models are fluent, confident, and sometimes wrong, biased, or unsafe. Knowing how they fail is part of using them.
The whole machine is trained to make text that sounds right.
Nothing inside it is trained to make text that is right.
That gap is where every failure in this chapter lives.
The same fluency, five ways
A confident voice is the model’s default — even when it’s wrong, biased, leaking, or just telling you what you want to hear.
Try it: open Sycophancy and notice the wrong fact isn’t one the model invented — it’s one you supplied, and it agreed anyway.
Reading the five
It states false things with a straight face. It isn’t hiding the truth — it’s the same next-token machine from Book 01, just as fluent when it has nothing real to say.
It tends to agree with you, even when you’re wrong, because agreement reads as “helpful.”
It soaks up the web’s patterns — nurse → she, engineer → he — and hands the stereotype back as the default.
Privacy leakage. Text seen often enough in training gets memorized word-for-word — and a leading prompt can pull it back out.
Over-trust. It sounds human, so it’s easy to lean on it like a person. There’s no one home.
Go deeper: why fluency and truth came apart
The objective only ever asked for the most probable next token. “Probable” is shaped by how things are usually said, not by whether they’re true, fair, or safe to repeat. So a well-formed lie, a polite agreement, a common stereotype, and a memorized phone number are all — to the loss function — just likely text. None of these are bugs in the code. They’re the training signal showing through. That’s why the fixes live around the model: better data, alignment, retrieval, evaluation, and a human who stays in the loop.
End of the trilogy
Three books, three machines made of the same parts.
Book 01 — the decoder that writes. Book 02 — the encoder that understands. Book 03 — the same models loose in the world.
You’ve seen how they’re built, how we talk to them, what they eat, how we grade them — and now, how they fail.
These tools are genuinely useful and genuinely fallible — and the same first principles explain both. Knowing where the fluency ends and the guessing begins is what it means to use them well.
Sources · 6
- Cheng, M., Yu, S., Lee, C., Khadpe, P., Ibrahim, L., & Jurafsky, D. (2025). ELEPHANT: Measuring and Understanding Social Sycophancy in LLMs. arXiv:2505.13995.
- Nadeem, M., Bethke, A., & Reddy, S. (2021). StereoSet: Measuring Stereotypical Bias in Pretrained Language Models. In Proceedings of ACL-IJCNLP 2021 (Vol. 1: Long Papers, pp. 5356–5371). arXiv:2004.09456.
- Carlini, N., Tramèr, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, Ú., Oprea, A., & Raffel, C. (2021). Extracting Training Data from Large Language Models. In 30th USENIX Security Symposium (USENIX Security 21), pp. 2633–2650. arXiv:2012.07805.
- Jurafsky, D., & Martin, J. H. (2026). Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition with Language Models (3rd ed., draft of January 6, 2026). Stanford University. Ch. 7 (Large Language Models); Ch. 10 (Masked Language Models).
- Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems 33 (NeurIPS 2020). arXiv:2005.11401.
- Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., et al. (2021). Ethical and social risks of harm from Language Models. DeepMind. arXiv:2112.04359.