codeloopback
Guide contents

Lesson 33 of 35

Further Reading

This page lists the papers, books, and projects behind this course, grouped by topic. Each entry keeps its title, authors, venue, year, and link, followed by one plain sentence on what it covers. The sentences are based on the titles, so reading the source is the way to confirm details. A few entries have no year or venue listed.

The lists suit three kinds of reading. Papers in Architecture and Inference Optimization Research give the original ideas behind techniques taught in the course. Developer Tools and GPU Infrastructure point to documentation to keep open while building. Intelligence Evaluation lists the benchmarks that the model selection lesson refers to (choosing a model).

flowchart LR
    A["Architecture"] --> R[Inference Optimization Research]
    R --> T[Developer Tools]
    T --> G[GPU Infrastructure]
    G --> E[Intelligence Evaluation]

The diagram shows one reading path from model ideas to techniques, then tools, then hardware and operations, then measurement. Other orders work equally well.

Architecture

Papers and books that define the model families covered in the course: transformers, attention variants, diffusion, speech, vision, and embedding models.

  • “Attention is All You Need,” by Ashish Vaswani et al. (Neural Information Processing Systems, 2017), https://arxiv.org/abs/1706.03762 The paper that introduces the transformer .
  • “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” by Jacob Devlin et al. (North American Chapter of the Association for Computational Linguistics, 2019), https://arxiv.org/abs/1810.04805 Presents BERT, a deep bidirectional transformer pre-trained for language understanding.
  • “BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models,” by Junnan Li et al. (International Conference on Machine Learning, 2023), https://arxiv.org/abs/2301.12597 Shows how to connect a frozen image encoder to a frozen large language model so the combined model understands images and text.
  • Deep Learning, by Ian Goodfellow, Yoshua Bengio, and Aaron Courville (The MIT Press, 2016), https://www.deeplearningbook.org/ A textbook on deep learning foundations, available online.
  • Deep Learning with Python (2nd Edition), by François Chollet (Manning, 2021), https://www.manning.com/books/deep-learning-with-python-second-edition A book that teaches deep learning with Python code, in its second edition.
  • “Denoising Diffusion Probabilistic Models,” by Jonathan Ho et al. (ArXiv abs/2006.11239, 2020), https://arxiv.org/abs/2006.11239 Describes diffusion models that generate images by learning to remove noise step by step.
  • “DiT: Scalable Diffusion Models with Transformers,” by William Peebles and Saining Xie (International Conference on Computer Vision (ICCV), 2022), https://arxiv.org/abs/2212.09748 Presents scalable diffusion models built with transformers.
  • “FlashAttention: Fast and Memory-Efficient Exact Attention with IO Awareness,” by Tri Dao et al. (ArXiv abs/2205.14135, 2022), https://arxiv.org/abs/2205.14135 Presents an attention algorithm that is exact and memory-efficient because it plans its reads and writes of GPU memory.
  • “FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning,” by Tri Dao (ArXiv abs/2307.08691, 2023), https://arxiv.org/abs/2307.08691 Improves FlashAttention with better parallelism and a better split of work.
  • “FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision,” by Jay Shah et al. (ArXiv abs/2407.08608, 2024), https://arxiv.org/abs/2407.08608 Presents fast and accurate attention with asynchrony and low-precision arithmetic.
  • Flash-Attention-4, by Tri Dao (Dao AI Research Lab, 2025), https://github.com/Dao-AILab/flash-attention The code repository for the fourth FlashAttention generation, listed for Blackwell GPUs.
  • “Imagen Video: High Definition Video Generation with Diffusion Models,” by Jonathan Ho et al. (ArXiv abs/2210.02303, 2022), https://arxiv.org/abs/2210.02303 Describes generating high-definition video with diffusion models.
  • “Language Models Are Few-Shot Learners,” by Tom Brown et al. (ArXiv abs/2005.14165), https://arxiv.org/abs/2005.14165 Shows that language models can be few-shot learners.
  • “Learning Transferable Visual Models from Natural Language Supervision,” by Alec Radford et al. (International Conference on Machine Learning, 2021), https://arxiv.org/abs/2103.00020 Describes learning transferable visual models from natural language supervision.
  • “Longformer: The Long-Document Transformer,” by Iz Beltagy et al. (ArXiv abs/2004.05150, 2020), https://arxiv.org/abs/2004.05150 Presents a transformer for long documents.
  • “Mamba: Linear-Time Sequence Modeling with Selective State Spaces,” by Albert Gu and Tri Dao (ArXiv abs/2312.00752, 2023), https://arxiv.org/abs/2312.00752 Presents the Mamba model, a sequence model with selective state spaces whose cost grows linearly with sequence length.
  • “Matryoshka Representation Learning,” by Aditya Kusupati et al. (Neural Information Processing Systems, 2022), https://arxiv.org/abs/2205.13147 Describes embeddings whose first dimensions already work as a smaller embedding (the Matryoshka idea).
  • “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer,” by Noam Shazeer et al. (ArXiv abs/1701.06538, 2017), https://arxiv.org/abs/1701.06538 Introduces the sparsely-gated mixture-of-experts layer.
  • “Reformer: The Efficient Transformer,” by Nikita Kitaev et al. (ArXiv abs/2001.04451, 2020), https://arxiv.org/abs/2001.04451 Presents an efficient transformer.
  • “Robust Speech Recognition via Large-Scale Weak Supervision,” by Alec Radford et al. (International Conference on Machine Learning, 2022), https://arxiv.org/abs/2212.04356 Describes speech recognition trained with large-scale weak supervision.
  • “RoFormer: Enhanced Transformer with Rotary Position Embedding,” by Jianlin Su et al. (ArXiv abs/2104.09864, 2021), https://arxiv.org/abs/2104.09864 Presents a transformer with rotary position embedding (RoPE).
  • “SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis,” by Dustin Podell et al. (ArXiv abs/2307.01952, 2023), https://arxiv.org/abs/2307.01952 Describes the SDXL latent diffusion model for high-resolution images.
  • “Segment Anything,” by Alexander Kirillov et al. (2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023), https://arxiv.org/abs/2304.02643 Presents a model that can segment any object in an image.
  • “Sentence-BERT: Sentence Embeddings Using Siamese BERT-Networks,” by Nils Reimers and Iryna Gurevych (ArXiv abs/1908.10084, 2019), https://arxiv.org/abs/1908.10084 Describes a method that produces sentence embeddings with Siamese BERT networks.
  • “The Llama 3 Herd of Models,” by Aaron Grattafiori et al. (ArXiv 2407.21783, 2024), https://arxiv.org/abs/2407.21783 The technical report on the Llama 3 models. The production lesson cites it for its GPU failure counts during training.
  • “Video Diffusion Models,” by Jonathan Ho et al. (ArXiv abs/2204.03458, 2022), https://arxiv.org/abs/2204.03458 Extends diffusion models to video.
  • “Visual Instruction Tuning,” by Haotian Liu et al. (ArXiv abs/2304.08485, 2023), https://arxiv.org/abs/2304.08485 Presents visual instruction tuning, a way to teach a language model to follow instructions about images.

Developer Tools

Libraries, runtimes, profilers, and documentation for building and serving models.

Frontier Open Models

Open model families published on Hugging Face, with the organization that publishes each.

GPU Infrastructure

References on GPU hardware, networking, cluster software, and reliable systems.

Inference Optimization Research

Research papers and projects on quantization, speculative decoding, attention, caching, parallelism, and fast image generation.

  • “Adversarial Diffusion Distillation,” by Axel Sauer et al. (European Conference on Computer Vision, 2023), https://arxiv.org/abs/2311.17042 Describes adversarial diffusion distillation.
  • “Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation,” by Ofir Press, Noah Smith, and Mike Lewis (ArXiv abs/2108.12409, 2021), https://arxiv.org/abs/2108.12409 Describes attention with linear biases for input length extrapolation (training short, testing long).
  • “AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration,” by Song Han (MIT, 2024), https://github.com/mit-han-lab/llm-awq Describes AWQ, a method that quantizes weights while taking activations into account.
  • Cache-DIT by Vipshop, https://github.com/vipshop/cache-dit The Cache-DIT repository from Vipshop.
  • “CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion,” by Jiayi Yao et al. (Proceedings of the Twentieth European Conference on Computer Systems, 2024), https://arxiv.org/abs/2405.16444 Describes fast serving for retrieval-augmented generation by fusing cached knowledge.
  • “Adding Conditional Control to Text-to-Image Diffusion Models,” by Lymin Zhang et al. (International Conference on Computer Vision, 2023), https://arxiv.org/abs/2302.05543 Describes adding conditional control to text-to-image diffusion models.
  • “Beyond the Buzz: A Pragmatic Take on Inference Disaggregation,” by Tiyasa Mitra et al. (ArXiv abs/2506.05508, 2025), https://arxiv.org/abs/2506.05508 A pragmatic take on inference disaggregation.
  • “Break the Sequential Dependency of LLM Inference Using Lookahead Decoding,” by Yichao Fu et al. (ArXiv abs/2402.02057, 2024), https://arxiv.org/abs/2402.02057 Describes lookahead decoding, which breaks the sequential dependency of decoding without a separate draft model.
  • “EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty,” by Yuhui Li et al. (ArXiv abs/2401.15077, 2024), https://arxiv.org/abs/2401.15077 Describes EAGLE, speculative sampling that rethinks feature uncertainty.
  • “EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees,” by Yuhui Li et al. (Conference on Empirical Methods in Natural Language Processing, 2024), https://arxiv.org/abs/2406.16858 Describes EAGLE-2, which uses dynamic draft trees.
  • “EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test,” by Yuhui Li et al. (ArXiv abs/2503.01840, 2025), https://arxiv.org/abs/2503.01840 Describes EAGLE-3, which scales up the method with training-time testing.
  • “FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving,” by Ye et al. (ArXiv abs/2501.01005, 2025), https://arxiv.org/abs/2501.01005 Describes FlashInfer, an attention engine for LLM serving.
  • “GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers,” by Elias Frantar (ArXiv abs/2210.17323, 2022), https://arxiv.org/abs/2210.17323 Describes GPTQ, a post-training quantization method for generative transformers.
  • “High-Resolution Image Synthesis with Latent Diffusion Models,” by Robin Rombach et al. (Conference on Computer Vision and Pattern Recognition (CVPR), 2021), https://arxiv.org/abs/2112.10752 Describes latent diffusion models, which denoise in a smaller latent space.
  • “Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference,” by Simian Luo (ArXiv abs/2310.04378, 2023), https://arxiv.org/abs/2310.04378 Describes latent consistency models, which generate images in few steps.
  • “LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale,” by Tim Dettmers et al. (ArXiv abs/2208.07339, 2022), https://arxiv.org/abs/2208.07339 Describes LLM.int8(), a way to run 8-bit matrix multiplication in large transformers.
  • “Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads,” by Tianle Cai et al. (ArXiv abs/2401.10774, 2024), https://arxiv.org/abs/2401.10774 Describes Medusa, which adds several decoding heads to a model to draft multiple tokens.
  • “Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism,” by Mohammad Shoeybi et al. (ArXiv abs/1909.08053, 2019), https://arxiv.org/abs/1909.08053 Describes Megatron-LM and its model parallelism for training very large models.
  • “Efficient Memory Management for Large Language Model Serving with PagedAttention,” by Woosuk Kwon et al. (Proceedings of the 29th Symposium on Operating Systems Principles, 2023), https://arxiv.org/abs/2309.06180 Describes PagedAttention for memory management in LLM serving.
  • “Fast Inference from Transformers via Speculative Decoding,” by Yaniv Leviathan et al. (International Conference on Machine Learning, 2022), https://arxiv.org/abs/2211.17192 Describes fast transformer inference through speculative decoding.
  • “Ring Attention with Blockwise Transformers for Near-Infinite Context,” by Hao Lin et al. (ArXiv abs/2310.01889, 2023), https://arxiv.org/abs/2310.01889 Describes ring attention, which passes blocks around a ring of devices to support very long context.
  • “SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration,” by Jintao Zhang et al. (ArXiv abs/2410.02367, 2024), https://arxiv.org/abs/2410.02367 Describes SageAttention, an accurate 8-bit attention method.
  • Sequence/Context Parallelism, by Megatron-LM for NVIDIA https://github.com/NVIDIA/Megatron-LM The Megatron-LM implementation of sequence and context parallelism.
  • SmoothQuant by Song Han (MIT), https://github.com/mit-han-lab/smoothquant The SmoothQuant repository.
  • “SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot,” by Elias Frantar and Dan Alistarh, (ArXiv abs/2301.00774, 2023), https://arxiv.org/abs/2301.00774 Describes SparseGPT, a one-shot pruning method for large language models.
  • “SpecVLM: Fast Speculative Decoding in Vision-Language Models,” Haiduo Huang et al. (ArXiv abs/2509.11815, 2025), https://arxiv.org/abs/2509.11815 Describes speculative decoding for vision-language models.
  • “TeaCache: Efficient KV Cache Compression via Tensor Decomposition,” by Feng Lu et al. (Alibaba TongYi Vision Intelligence Lab, ArXiv abs/2411.19108, 2025), https://github.com/ali-vilab/TeaCache Describes TeaCache. The title mentions KV cache compression through tensor decomposition.

Intelligence Evaluation

Benchmarks and guides for measuring model quality.

  • ARC AGI Prize by Greg Kamradt (2025), https://arcprize.org/ The ARC AGI Prize website.
  • Evals for AI Engineers: Systematically Measuring and Improving AI Applications, by Shreya Shankar and Hamel Husain (O’Reilly Media, forthcoming 2026) https://www.oreilly.com/library/view/evals-for-ai/9798341660717/ A book on measuring and improving AI applications with evals, listed as forthcoming in 2026.
  • Grade School Math: Training Verifiers to Solve Math Word Problems, by Karl Cobbe and Vineet Kosaraju (ArXiv abs/2110.14168, 2021), https://github.com/openai/grade-school-math A math word problem dataset with a method to train verifiers.
  • “How to Fine-Tune Qwen3 to GPT-4o Level Performance,” by Greg Schoeninger (Fine-Tune Fridays, Oxen AI, 2025), https://ghost.oxen.ai/how-to-fine-tune-qwen3-to-gpt-4o-level-performance/ A guide to fine-tuning Qwen3 to GPT-4o level performance.
  • “Humanity’s Last Exam,” by Long Phan et al. (Center for AI Safety and Scale AI, ArXiv abs/2501.14249, 2025), https://www.lastexam.ai/ The benchmark named Humanity's Last Exam.
  • “HumanEval: Evaluating Large Language Models Trained on Code,” by Michelle Pokrass, Qiming Yuan, and Yichen Xu (OpenAI, 2021), https://github.com/openai/human-eval A benchmark for evaluating language models on code generation.
  • “MMLU: Measuring Massive Multitask Language Understanding,” by Dan Hendrycks et al. (Proceedings of the International Conference on Learning Representations (ICLR), 2021), https://github.com/hendrycks/test The MMLU benchmark, which measures knowledge across many subjects.
  • “MTEB: Massive Text Embedding Benchmark,” by Niklas Muenninghoff et al. (Conference of the European Chapter of the Association for Computational Linguistics, 2022), https://arxiv.org/abs/2210.07316 A benchmark for text embedding models.
  • “SWE-Bench: Can Language Models Resolve Real-World Github Issues?” by Carlos Jimenez et al. (Proceedings of the International Conference on Learning Representations (ICLR), 2024), https://www.swebench.com/ A benchmark that asks models to resolve real GitHub issues.

Summary

  • The list has 109 entries in six groups: Architecture (27), Developer Tools (20), Frontier Open Models (14), GPU Infrastructure (12), Inference Optimization Research (27), and Intelligence Evaluation (9).
  • Every entry lists its title, authors, venue, year, and link where available.
  • The Llama 3 paper is the source of the GPU failure numbers in the multi-cloud lesson.