How LLMs Work
What an LLM is, prompts and context, the vocabulary of agents and LLM engineering, a first run, next-token prediction, tokenizers and BPE, the math of transformers, self-attention, the transformer block, prefill, decode and the KV cache, sampling, training stages, and chat templates.
- 1.What Is a Large Language Model?
- 2.Talking to a Model: Prompts, Context, and Limits
- 3.Agents and LLM Engineering: The Vocabulary
- 4.Introduction and First Run
- 5.What a Language Model Computes
- 6.Tokens, Vocabularies, and Special Tokens
- 7.Building a BPE Tokenizer
- 8.The Math Behind Transformers, in Java
- 9.Self-Attention Step by Step
- 10.The Transformer Block and the Full Model
- 11.Generation, Prefill, Decode, and the KV Cache
- 12.Sampling: Greedy, Temperature, Top-k, Top-p, and Min-p
- 13.Penalties, Logit Bias, and Sampler Chains
- 14.How Models Are Trained, and the Variants You Download
- 15.Chat Templates, Roles, and Special Tokens