Home
Data Structures and Algorithms
About

Guide contents

LLM Fundamentals

Kinds of Models

1.AI, Machine Learning, and Neural Networks
2.Large and Small Language Models
3.Vision-Language Models
4.Multimodality
5.Frontier Models
6.Artificial General Intelligence (AGI)

How Models Work

7.Parameters and Weights
8.Tokens
9.Tokenizers and Tokenization
10.The Context Window
11.Context: What Goes Into a Request
12.Temperature and Sampling
13.Mixture of Experts (MoE)

Training Models

14.Training vs Inference
15.Pre-training vs Post-training
16.Fine-tuning
17.LoRA and Parameter-Efficient Fine-tuning
18.Synthetic Data
19.Reinforcement Learning from Human Feedback (RLHF)
20.Distillation
21.Data Poisoning

Running Models

22.Reasoning Models
23.Caching
24.Quantization
25.Model Routing

Prompts and APIs

26.Prompts
27.Prompt Engineering
28.Context Engineering
29.Structured Outputs
30.APIs for AI Models

Agents

31.Models vs AI Agents
32.AI Agents
33.Tools
34.The Agent Loop
35.The Harness

Knowledge and Memory

36.Hallucinations
37.Grounding
38.Retrieval-Augmented Generation (RAG)
39.Vector Databases and Embeddings
40.Memory
41.Compaction
42.Context Rot

Agent Tooling

43.Model Context Protocol (MCP)
44.Agent Skills
45.Computer Use
46.Sandboxes
47.Agentic Search
48.Human in the Loop
49.Agentic Engineering
50.Vibe Coding

Quality and Safety

51.Benchmarks
52.Evals
53.Prompt Injection
54.Data Exfiltration
55.Open Weights and Open-Source Models

Interview Practice

56.LLM Fundamentals Interview Questions: Concepts
57.LLM Fundamentals Interview Questions: Scenarios and Design

LLM Fundamentals

Kinds of Models

1.AI, Machine Learning, and Neural Networks
2.Large and Small Language Models
3.Vision-Language Models
4.Multimodality
5.Frontier Models
6.Artificial General Intelligence (AGI)

How Models Work

7.Parameters and Weights
8.Tokens
9.Tokenizers and Tokenization
10.The Context Window
11.Context: What Goes Into a Request
12.Temperature and Sampling
13.Mixture of Experts (MoE)

Training Models

14.Training vs Inference
15.Pre-training vs Post-training
16.Fine-tuning
17.LoRA and Parameter-Efficient Fine-tuning
18.Synthetic Data
19.Reinforcement Learning from Human Feedback (RLHF)
20.Distillation
21.Data Poisoning

Running Models

22.Reasoning Models
23.Caching
24.Quantization
25.Model Routing

Prompts and APIs

26.Prompts
27.Prompt Engineering
28.Context Engineering
29.Structured Outputs
30.APIs for AI Models

Agents

31.Models vs AI Agents
32.AI Agents
33.Tools
34.The Agent Loop
35.The Harness

Knowledge and Memory

36.Hallucinations
37.Grounding
38.Retrieval-Augmented Generation (RAG)
39.Vector Databases and Embeddings
40.Memory
41.Compaction
42.Context Rot

Agent Tooling

43.Model Context Protocol (MCP)
44.Agent Skills
45.Computer Use
46.Sandboxes
47.Agentic Search
48.Human in the Loop
49.Agentic Engineering
50.Vibe Coding

Quality and Safety

51.Benchmarks
52.Evals
53.Prompt Injection
54.Data Exfiltration
55.Open Weights and Open-Source Models

Interview Practice

56.LLM Fundamentals Interview Questions: Concepts
57.LLM Fundamentals Interview Questions: Scenarios and Design

Home/LLM Fundamentals/Knowledge and Memory

Knowledge and Memory

How models get facts right and keep track of long work: hallucinations, grounding, RAG, vector databases and embeddings, memory, compaction, and context rot.

7 lessons

36.Hallucinations
37.Grounding
38.Retrieval-Augmented Generation (RAG)
39.Vector Databases and Embeddings
40.Memory
41.Compaction
42.Context Rot

© 2026 codeloopback