Lesson 56 of 57
LLM Fundamentals Interview Questions: Concepts
Short answers to concept questions about large language models and the systems built around them. The groups follow the course sections in order, and the first question on each topic links to the lesson that explains it in full. A line that starts with "Follow-up:" is the next question an interviewer is likely to ask, with its answer in the paragraph below it.
Kinds of models
1. How do AI, machine learning, neural networks, and deep learning relate to each other?
AI is the goal: computers doing work that normally needs human thinking. Machine learning is one way to reach it, where the system learns its rules from data instead of a person writing them. A neural network is one kind of machine learning model, made of layers of adjustable numbers, and deep learning means neural networks with many layers. A large language model is a neural network. See AI, Machine Learning, and Neural Networks.
Follow-up: When is a hand-written rule list still the right choice?
When the set of possible situations is fixed and small, so a person can predict every case. A game enemy that follows written rules is an example, and it is not machine learning.
2. What is the difference between a model and a product?
The model reads text and writes text. The agent or harness around it carries out actions such as reading files, and the product is the application on top. A feature such as chat history or file upload belongs to the software, not to the model.
3. What is a large language model?
A large language model (LLM) is a neural network, usually a transformer, that takes tokens in and produces tokens out. It is trained on a very large amount of text and learns how tokens relate to each other. "Large" refers to the number of parameters. See Large and Small Language Models.
Follow-up: How can one model translate, classify, and choose tools?
Every one of those jobs is text in and text out. A label or a tool name is text, so the same operation covers all of them.
4. When does a small language model fit better than a large one?
A small language model has the same design with far fewer parameters, so it fits in the memory of one machine. It suits one narrow job, especially after fine-tuning, and it suits cases where data must stay on local hardware or where per-request cost must be zero. The consequence of fewer parameters is less capacity for detail and nuance on broad tasks.
5. What is a vision-language model?
A vision-language model (VLM) takes images, video, and text as input and writes text as output. The inputs are read together, so the model can relate the words of a question to the right part of a picture. A video is processed as a sequence of frames. See Vision-Language Models.
6. How does a VLM differ from an older single-purpose vision model?
A single-purpose model, such as a face detector, answers one fixed question and returns fixed structured output such as bounding boxes. A new question needed a new model trained on a new image set. With a VLM the task arrives with the prompt, so a new question is a new prompt.
7. What does multimodal mean?
A modality is a type of data: text, images, audio, or video. A multimodal model or assistant handles more than one, and it reasons about the inputs together instead of in separate steps. See Multimodality.
8. An assistant can read images. Does that mean it can create them?
No. Input modalities and output modalities are separate abilities. An assistant that both reads and creates several modalities usually combines several models, such as a language model and an image model, and the application routes each request to the right one.
Follow-up: What decides what an assistant can do?
The model or models behind it and the software that connects them. The look of the chat screen says nothing about it.
9. What is a frontier model?
A frontier model is one of the most capable general-purpose models available at a given time. The term has two parts: leading edge in capability, and general purpose instead of specialist. See Frontier Models.
10. Who decides which models count as frontier?
Nobody officially. The category is agreed on informally from two signals that have to agree: benchmark results and the preference of people who use the models daily. Membership changes with each release, and few labs take part because the work needs very large budgets, hardware, and data.
11. What is AGI?
AGI means artificial general intelligence: AI that can do almost any thinking task a person can, at human level or better, including tasks it was not trained for. There is no agreed definition and no agreed test. A practical reading is that a model handles any screen-based task about as well as a person. See Artificial General Intelligence (AGI).
Follow-up: What is still open at the time of writing?
Top models do a large share of everyday computer work, often faster than people. Doing everything, working unsupervised all the time, and matching the user's intent without guidance are not settled.
12. Why do capable models write a program to add up large numbers?
Models are unreliable at large calculations done internally, because they generate likely tokens and do not compute. A model that writes and runs a program gets an exact result from the program and reports it.
How models work
13. What are parameters and weights?
A parameter is one of the numbers stored inside a model. Training sets its value, starting from a random value and adjusting it a little after each example. A weight is the strength of one connection in the network and is the most common kind of parameter, so the two words are used as synonyms. Biases are another, far less numerous kind. See Parameters and Weights.
14. Does a model with more parameters always give better answers?
More parameters allow more layers and connections, which lets a model capture more nuance. Data quality and the training method also decide the result, so the count alone does not settle it. A larger model also needs more memory and costs more to run.
15. What is a token?
A token is a chunk of text: a word, a piece of a word, a number, or a punctuation mark. In English one token is about three-quarters of a word. The model receives token IDs, which are numbers, and the vocabulary maps each token to its ID. See Tokens.
16. Why are words split into pieces instead of kept whole?
Pieces keep the vocabulary a manageable size. Common pieces appear often in training, so the model learns them well, and a word the model has never seen can be built from known pieces. A long number is handled the same way.
17. How are tokens billed?
There are two counts: input tokens, which is everything sent in the request, and output tokens, which is everything the model writes. Cost is input tokens times the input price plus output tokens times the output price, and output tokens usually cost more. A subscription hides the counts but still measures usage in tokens, which is why plans have limits.
Follow-up: Why can the same text cost a different amount on two models?
Each model has its own tokenizer, so the same text splits into a different number of tokens. The price per token also differs between models.
18. What does a tokenizer do?
A tokenizer turns raw text into tokens and then into token IDs, which is called encoding. It also turns the model's output IDs back into text, which is called decoding. This runs on every request and every response. See Tokenizers and Tokenization.
19. Can a model be used with a different tokenizer than the one it was trained with?
No. The tokenizer is built before training because the training text has to be tokenized first, and the model learns relations between those specific IDs. IDs from another tokenizer are meaningless to it.
20. What is the context window?
The context window is the maximum number of tokens a model can handle in one request. For most models it covers the input and the output together. The term is also used for the content itself. See The Context Window.
Follow-up: Why does the limit exist?
Answer quality falls as the input grows, long inputs need more compute and memory, and models are trained up to a certain length.
21. Why does an assistant forget something said early in a long chat?
The model has no memory between requests, so the application sends the whole conversation again each time. When the window fills, the request fails or the application drops or summarises older messages. The early detail is then no longer in what the model reads.
22. What is a system prompt?
A system prompt is a hidden instruction written by the company or developer behind an assistant. It is sent with every request, comes before the conversation, and uses tokens. Two products on the same model can behave differently because their system prompts differ.
23. What does the context of one request contain?
Eight typical parts: system instructions, developer instructions, conversation history, the user's message, attached files, retrieved documents, saved memory, and tool definitions. Only the message and the attachment come from the user. The application adds the rest. See Context: What Goes Into a Request.
Follow-up: What does the model answer from?
Two sources only: the context of the current request and what it learned in training.
24. How does an attached file reach the model?
There is one input stream. The application copies the text of the file into the input next to the message, usually under a label with the file name, and images become image tokens in the same input. The whole document takes space in the window, because the model cannot read only parts of it.
25. What is sampling, and why is a model not deterministic?
For each position the model produces a probability for every token in its vocabulary. Sampling picks one token from that list at random, weighted by the probabilities. Because the pick is random, the same input can give different outputs on different runs. See Temperature and Sampling.
26. What does temperature change?
Temperature is a setting on the sampling step. A low value sharpens the probabilities so the most likely token is chosen almost every time, and at 0 it is always chosen. A high value flattens them, so less likely tokens are chosen more often and answers vary more. It does not change the model's parameters or knowledge.
Follow-up: How is a value chosen?
Low values fit tasks with one correct answer, such as extraction, classification, and code. Higher values fit brainstorming and creative writing. A very high value lets poorly fitting tokens through and the text can become incoherent.
27. What is a mixture of experts model?
A mixture of experts (MoE) model is an LLM in which only part of the parameters is used for each token. Inside a layer, the single block of parameters is replaced by several experts, and a small router picks a few of them. The experts are not assigned human topics, because the division of work is learned in training. See Mixture of Experts (MoE).
28. What is the difference between total and active parameters?
Total parameters are everything the model holds, and they decide capacity and memory needs. Active parameters are the ones used for one token, and they decide the computation per token. An MoE model is therefore faster and cheaper at inference than a dense model of the same total size.
Follow-up: Does an MoE model run on a machine with less memory?
No. Any expert may be needed for the next token, so every parameter must still be loaded.
Training models
29. What is the difference between training and inference?
Training is the phase in which the parameters are worked out from examples. Inference is the phase in which the finished model, with fixed parameters, computes outputs for new inputs. Training is one very long job on many GPU machines, and inference is many short requests on machines that each hold the whole model in memory. See Training vs Inference.
30. Does a model learn from a conversation while it answers?
No. Parameters do not change during inference. Earlier messages are available only because the application sends them again as context.
Follow-up: Why can small models run on personal computers and large ones cannot?
Inference needs every parameter in memory, and the memory needed grows with the parameter count.
31. What are pre-training and post-training?
Pre-training is the first and largest stage. It trains the model to predict the next token on a very large amount of text, and the correct answers come from the text itself, so no manual labelling is needed. Post-training builds on that network with much smaller, chosen data to shape behaviour, specialisation, reasoning, and safety. See Pre-training vs Post-training.
32. What is a base model, and why is it not an assistant?
A base model is what comes out of pre-training. It continues text and has no built-in idea of answering, stopping, or refusing. Given a question, it may continue with more questions, because its training data contains many questions that are not followed by answers.
Follow-up: Why are refusals of harmful requests not perfectly reliable?
Refusals are behaviour learned in post-training and held in the parameters. They are not a separate hard rule in the software.
33. What is fine-tuning?
Fine-tuning is further training of an already trained model on a smaller, chosen set of input and output examples. A finished model goes in and an updated version of the same model comes out. It can teach a writing style, a behaviour, or knowledge that was absent from pre-training, such as internal codes. See Fine-tuning.
Follow-up: Does fine-tuning erase what the model already knew?
No. The new examples shift the existing parameters, so the new patterns dominate in the cases they cover. Too little data changes almost nothing, and heavy training on very narrow data can damage general abilities.
34. What is needed to fine-tune a model, and when is fine-tuning a poor fit?
It needs hardware with enough memory to hold the model while it trains, a model whose weights may be downloaded and modified (or a provider's fine-tuning service), and one narrow job to improve. It fits poorly for facts that change often, because each change needs retraining.
35. What is the difference between full fine-tuning and LoRA?
Full fine-tuning may change every parameter, needs memory and compute for the whole model, and produces a complete new copy. LoRA (low-rank adaptation) is the most widely used parameter-efficient fine-tuning (PEFT) method. It freezes the original parameters and trains small added blocks beside existing layers, often around 1% of the model or less. See LoRA and Parameter-Efficient Fine-tuning.
Follow-up: What does "low rank" mean here?
Each correction is expressed as two narrow tables of numbers instead of one full-size table, which is what cuts the number of trained parameters.
36. What is an adapter?
An adapter is the small file of new parameters that LoRA produces. It works only together with the base model it was trained for. One base model can serve many adapters, one per job, so only one copy of the large model is stored and loaded.
37. What is synthetic data, and when is it used?
Synthetic data is training data created on purpose, usually by an AI model, instead of collected from human-written material. It is used when a narrow task has no existing examples, for instance a library that is newer than the model. A model generates examples from source material such as documentation, people review them, and the result is used for fine-tuning. See Synthetic Data.
38. What are the limits of synthetic data?
It can lack variety, and it cannot exceed the quality of its source material. Errors in generated examples are learned by the fine-tuned model, which is why human review comes before training.
Follow-up: When is synthetic data unnecessary?
When enough private documents exist. They were never crawled, so they hold information public models have not seen, and fine-tuning can use them directly.
39. What is RLHF?
RLHF stands for reinforcement learning from human feedback. It is a form of fine-tuning in which the model attempts a problem, a person grades the attempt, and the grade nudges the weights. Then the loop repeats with the next problem. See Reinforcement Learning from Human Feedback (RLHF).
40. How does RLHF differ from ordinary fine-tuning?
Ordinary fine-tuning gives the model finished examples with correct outputs. RLHF has no answer key: outputs are judged better or worse, often by comparing two of them. This makes it possible to train qualities for which nobody can write the ideal output in advance, because judging is less demanding than writing.
Follow-up: What are the consequences at scale?
A reward model trained on human judgements does most of the grading, and the resulting model inherits the preferences and biases of its graders.
41. What is distillation?
Distillation is training one model, the student, on the outputs of another already trained model, the teacher. A broad variety of prompts is sent to the teacher, and the prompt and output pairs become training data. Labs use it to turn their own large models into smaller ones that are cheaper and faster at inference. See Distillation.
42. What are the limits of distillation, and where does the dispute lie?
The student inherits the teacher's errors, covers only what the prompts covered, and generally does not exceed the teacher. The technique is neutral and accepted on a lab's own models. The dispute concerns distilling another company's model without permission, which terms of service typically forbid, and the legal position is not settled.
43. What is data poisoning?
Data poisoning is the deliberate insertion of harmful examples into training data so that the trained model behaves in a way its builders did not intend. It works because training adjusts the model towards whatever data it is given and does not judge the data. The attack changes the parameters, so it affects every user until the model is retrained or replaced. See Data Poisoning.
44. What is a backdoor, and why are fine-tuning sets easier targets than pre-training data?
A backdoor is planted behaviour that appears only when a specific trigger is in the input, so normal tests pass. Fine-tuning sets are small, so a few rows are a meaningful share, and they sit inside one organisation where an insider can edit them. On the open web an attacker can only publish pages and hope they are collected.
Follow-up: What are the defences?
Access control and review for training data, knowledge of data sources, filtering, testing of the trained model, restricted tool permissions, and monitoring of the running system.
Running models
45. What is a reasoning model?
A reasoning model is an LLM that writes out its working first and its final answer second. The architecture and the token-by-token generation are the same as in a standard model. The step helps because each generated token becomes input for the next, so written intermediate results can be used when the answer is produced. See Reasoning Models.
Follow-up: Where did the idea come from?
From chain-of-thought prompting, where users asked a model to think step by step. Answers written after the working scored better in tests, so model makers built the step in.
46. What does the thinking step cost, and can the visible working be trusted?
Reasoning tokens are usually billed as output tokens, even when only a summary is shown, and they add latency and take window space. The gain is on multi-step problems, and lookups and short rewrites gain little. The visible working is generated text: it helps a reader check the answer, but it is not a guaranteed record of the model's internal computation.
47. What does prompt caching do?
The provider keeps the processed input and output of a request. When a later request starts with the same text, the stored work for that beginning is reused and only the new tail goes through the network. A cache hit lowers latency and cost, and cached input tokens are usually billed at a reduced rate. See Caching.
Follow-up: Does caching free space in the context window?
No. The tokens are still in the request. Caching reduces how many tokens are computed, and output tokens are generated and billed normally.
48. How should a prompt be ordered to get cache hits?
Matching starts at the first token and stops at the first difference. Stable content such as instructions and tool descriptions therefore belongs at the start, and changing content belongs at the end. Entries also expire, and after expiry the next request is computed in full.
Follow-up: How does prompt caching differ from response caching?
Prompt caching reuses work for a shared prefix and still generates a new answer. Response caching returns a stored answer for an identical request.
49. What is quantization?
Quantization stores each weight with fewer bits, commonly 8 or 4 instead of 16 or 32. The architecture and the number of weights stay the same. Memory for weights is roughly the number of weights times the bits per weight, so going from 16 bits to 4 bits cuts it to a quarter. See Quantization.
50. Why is the quality loss from quantization usually small?
Each weight is rounded to the nearest value in a small set of allowed values. Outputs depend on sums over very many weights, so the small errors mostly cancel out. The loss grows as the bit count drops.
Follow-up: How is a quantization level chosen?
Take the highest precision that fits in the available memory and meets the speed requirement, then test quality on the real task.
51. What is model routing?
Model routing means sending each request to a suitable model instead of sending everything to the most capable one. The same request uses about the same number of tokens on any model, but the price per token differs widely. Light tasks such as summarising, rewriting, and extracting go to small models, and design and debugging go to the largest. See Model Routing.
52. What is the difference between manual and automatic routing, and what can go wrong?
In manual routing the user or developer picks the model for each task. In automatic routing a router, usually a small model, classifies the request and forwards it, and the user is not told which model answered. A router can misclassify and send a hard task to a weak model. A manual override or a retry on a larger model addresses that.
Prompts and APIs
53. What is a prompt?
A prompt is what the user sends to the model in one turn. It can hold a task, which says what to do, and material, which is the text, files, or images the task is about. Attachments are merged into the same input, so they count toward tokens and the window. See Prompts.
54. How does a prompt differ from the context?
The prompt is one part of the context. A request typically holds system instructions, tool descriptions, the conversation so far, and then the new prompt last. In a chat app the user controls only the prompt, and developers control the system prompt and the rest.
55. What is prompt engineering?
Prompt engineering is writing a prompt so that the first response is the intended one. Current models infer much more than early ones did, but a model cannot know information that exists only in the user's head, such as length, audience, or team-specific meanings. See Prompt Engineering.
Follow-up: What does a vague prompt cost?
Extra round trips. Each one resends the whole conversation, so it costs time and tokens.
56. What does a complete prompt state?
The task, the constraints, the output format, definitions of terms that could be read in more than one way, and the style. Naming the audience and giving an example are common additions. It is the same information a colleague would need: what the job is, what it must not be, and what finished looks like.
57. What is context engineering?
Context engineering is deciding everything a model receives before it answers: instructions, documents, tool definitions, memory, and history, as well as the prompt. Prompt engineering is the subset about wording. A fact that is missing from the context and from training cannot appear in the answer, however well the prompt is worded. See Context Engineering.
58. Why not send the model everything that is available?
Too much context costs tokens, slows the response, and can reduce accuracy because relevant details are buried among unrelated ones. Good context is selective: the few documents, instructions, and tools that bear on the task.
Follow-up: What does too little context produce?
A generic answer that would fit anyone, because the model had no specifics to work from.
59. What is structured output, and why is it needed?
Structured output is model text in a fixed, predefined shape, usually JSON, meant for a program to read. A person can read any phrasing, but software reads specific fields, so it needs the same field names and allowed values every time. See Structured Outputs.
60. What is the difference between asking for JSON in the prompt, a JSON mode, and a structured output mode?
| Method | What it guarantees |
|---|---|
| Asking in the prompt | Nothing. It can fail through extra prose, renamed or missing fields, unexpected values, or broken syntax |
| JSON mode | Valid JSON, but not a particular set of fields |
| Structured output mode with a schema | An answer that matches the schema |
A structured output mode typically uses constrained decoding: at each step only tokens that keep the output valid can be chosen.
Follow-up: Does a schema make the answer correct?
No. The guarantee is about shape. A valid answer can hold a wrong value, so the content needs its own testing.
61. What is a model API?
An API is a documented interface through which one program asks another for something. For a hosted model, a program sends an HTTP POST with a JSON body to an endpoint, authenticated with an API key, and reads a JSON response that contains the generated text. Usage is billed per input and output token. See APIs for AI Models.
Follow-up: Why can the same model behave differently through the API than in the chat app?
The chat app adds its own system instructions, history, and tools. A basic API call is stateless and contains only what the calling program sends.
62. Which settings does a request carry?
Temperature controls variation. A maximum output limit caps length and cost. Stop sequences end generation at chosen text. A format setting requests structured output. A request can also list tool definitions, and the calling program runs any tool the model asks for and returns the result in a follow-up request.
Agents
63. What is the difference between a model and an AI agent?
A model is a function from tokens to tokens, with no persistent memory and no ability to act on the outside world. An agent is the software around a model that is allowed to carry out actions the model asks for. See Models vs AI Agents.
64. What does "deciding and doing are separate" mean?
The model reads the situation, works out the next step, and writes it down as text. Opening files, running searches, and sending requests need a program. A bare model asked to perform an action can only describe it.
65. What is a tool call?
A tool call is text generated by the model that names a tool and its arguments. Generating it performs no action. The agent executes it and returns the raw result to the model as a tool result. See AI Agents.
66. What are the three jobs of an agent?
Offering the tools, running the chosen tool with the given arguments, and holding the environment of files, results, and history. The model can request only the tools it was offered, so the tool list and the permissions define what the agent can do and what can go wrong.
67. What does a tool definition contain?
A name, the inputs the model must fill in, and a plain description of what the tool does. The software owns the code, and the model receives only the description. The tool list is text in the context, so it uses tokens. See Tools.
68. Why does the same model have different abilities in different products?
The tool lists differ. The software decides which tools exist, and the model cannot add any. Because every action passes through the software, the software is also the place to enforce permissions and limits.
69. Describe the agent loop.
The software builds one combined message with the system prompt, the tool descriptions, the conversation so far, and the prompt. It asks the model, runs the requested tool with its own code, adds the result to the transcript, and asks again. See The Agent Loop.
Follow-up: Why does the input cost of an agent task grow faster than the transcript?
The model keeps no memory, so every round sends the full transcript again, including all earlier tool calls and results.
70. How does the agent loop end?
The exit test is mechanical: if the model's output contains no tool call, there is nothing to run, so the text is shown to the user as the final answer. Production agents add a maximum number of rounds, a budget, or a stop control, because the model's own judgment is the only other exit.
71. What is a harness?
A harness is the software wrapped around a model. It builds each request, runs the agent loop, executes tool calls, and stores and reloads memory. The model receives only what the harness sends. See The Harness.
72. How do the words "agent" and "harness" differ?
Both are used for the same application. Strictly, the harness is the software without the model, and the agent is the model plus the harness. Permissions, round limits, and context handling are harness decisions, which is why the same model can perform differently in two products.
Knowledge and memory
73. What is a hallucination, and why does it happen?
A hallucination is output that sounds plausible but is false or made up. A model picks each next token by probability and has no built-in step that checks a statement against the world. Likely and true usually agree, and hallucinations appear where the model lacks knowledge but a likely continuation still exists. See Hallucinations.
Follow-up: Can tone reveal a hallucination?
No. Hallucinated text has the same fluency and confidence as correct text.
74. Why is a mostly correct answer the highest risk, and how is an answer checked?
Correct claims build trust that carries one wrong detail through. The method is to split the answer into individual claims and test each against a trusted source. Names, numbers, dates, quotations, and citations deserve the most attention, and citations must themselves be verified.
75. What is grounding?
Grounding ties an answer to sources outside the model, such as documents or web pages, so that each statement can be checked. The software adds instructions to the request: work from the supplied documents, cite the location of each statement, and say when the documents do not support something. See Grounding.
Follow-up: Why is a reference stronger than a list of sources?
A reference points to a specific page or paragraph. A general list does not say where the statement is.
76. Does grounding eliminate hallucinations?
No. The model can misread a source, give a wrong reference, or faithfully repeat an error in the source. The practical benefit is that verification becomes fast, because the reference says where to look.
77. What is RAG?
RAG stands for retrieval-augmented generation. Software searches a document store for passages that match the question, adds them to the prompt, and the model generates from the enlarged prompt. The model is unchanged and sees only the pasted passages. See Retrieval-Augmented Generation (RAG).
78. When does RAG fit, and what does its quality depend on?
RAG suits knowledge that is private, large, or frequently updated, because changing a document is cheaper than retraining or fine-tuning. Sending only relevant passages also keeps prompts inside the window and lowers cost. Answer quality depends on retrieval quality: a wrong or missing passage leads to a wrong or ungrounded answer.
79. What is an embedding?
A vector is a list of numbers, and an embedding is a vector that stands for the meaning of a piece of text, produced by an embedding model. Texts with similar meaning have embeddings that are close together, even when they share no words. See Vector Databases and Embeddings.
80. How does a vector database serve a RAG system?
At ingestion, text is extracted, cut into chunks that each cover one topic, and each chunk gets one embedding. At query time the question is embedded with the same embedding model, and the nearest chunks are returned, commonly measured by cosine similarity. The language model receives the text of those chunks and never talks to the database.
Follow-up: Why chunk, and when is keyword search still stronger?
One embedding for a whole document blurs many topics into one position. Keyword search is stronger for exact identifiers, so the two are sometimes combined.
81. How does memory work in an AI tool?
The model is stateless, so memory is a feature of the tool around it. The tool saves information such as preferences, decisions, and results, and inserts it into later prompts. Very often the store is a few plain Markdown files, and the tool, not the model, writes to them. See Memory.
82. What kinds of memory are there?
| Pair | Meaning |
|---|---|
| Short-term and long-term | The current session's context, and anything stored outside the model that survives across sessions |
| Project and global | Stored beside one project and applied only there, and stored in a user-wide location and applied everywhere |
Follow-up: What does loading the whole memory file cost?
The file's full token count on every prompt. Selective loading searches and sends only matching items.
83. What is compaction?
Compaction makes room in a full context window by removing or shortening earlier content. The harness runs it automatically when the context nears its limit, and the answering model only sees the result. See Compaction.
84. Compare truncation and summarisation.
Truncation drops the oldest turns. It is free and immediate, but it chooses by age and not by importance. Summarisation replaces earlier turns with a summary written by a model, which costs an extra request and keeps the main points of all turns.
Follow-up: What does a summary lose?
Specific conditions, numbers, and wording. Repeated compaction compounds the loss, so details that must persist are safer in memory files or project documents.
85. What is context rot?
Context rot is the decline in answer quality as a context becomes longer and more cluttered. It is a quality problem, separate from running out of window space, and it begins well below the advertised limit. Clutter misleads the model, and length alone reduces precision even when all content is relevant. See Context Rot.
86. What are the symptoms of context rot, and how is it prevented?
Symptoms are vaguer answers, hedging between options, revived rejected ideas, and invented details. Prevention has three habits: start a fresh session carrying only conclusions, trigger compaction deliberately before the limit, and include content only when leaving it out would change the answer.
Agent tooling
87. What is MCP?
MCP, the Model Context Protocol, is an open standard for connecting AI applications to external tools and data. A service provider runs an MCP server that exposes tools, which are actions, and resources, which are data to read. The protocol standardises how they are described, listed, and called, not what any tool does. See Model Context Protocol (MCP).
Follow-up: What changes for the model?
Nothing. MCP tools appear as tool descriptions in the context, and the model requests them like built-in tools. The server runs the code.
88. What problem does MCP solve, and what risks does it bring?
Without it, an agent is limited to the tools it was built with, and each integration needs custom code. A shared standard needs one implementation per application and one server per service. The risks are trusting third-party servers, tools with real side effects, and context space used by many tool descriptions.
89. What is an agent skill?
An agent skill is a folder with a required SKILL.md file and optional references/, scripts/, and assets/ folders. The file is Markdown: metadata with name and description on top, instructions below. It acts like a prompt that is written once and attached by the application. See Agent Skills.
Follow-up: How does a skill differ from MCP?
MCP connects the agent to outside systems through tools. A skill supplies written instructions for how to do a job.
90. What is staged loading, and why does the description matter?
All skill descriptions are always in the context. The model matches the request against them, the matching skill's file is loaded in full, and extra files are opened only if the instructions call for them. The description is the only part the model sees before choosing, so a vague one matches too much and one that only repeats the name gives no signal.
91. What is computer use?
Computer use is a tool that lets a model operate a graphical interface. The model receives a screenshot, names a point by x and y coordinates, and asks for a click or typing, which the agent software performs. It exists because much software has no API. See Computer Use.
92. Why is computer use slower than a purpose-built tool?
The model cannot know the state of the screen without a new screenshot after every action. Opening one file takes eight steps, four of which send a full screenshot, where a read_file tool takes one call. Computer use works with any software that has a screen and is the slowest option, so agents use precise tools when they fit.
93. What is a sandbox?
A sandbox is an enforced boundary that limits what an agent can reach. A locally installed agent otherwise runs with the permissions of the user who started it and can read, change, or delete anything that user can. The boundary is enforced by software outside the model, so it holds even when the model does not follow its instructions. See Sandboxes.
94. What does a sandbox not do?
It does not make the agent behave correctly, and resources inside it are still at risk. It limits how much damage a mistake can cause. Sandbox implementations have had flaws and have been escaped, so it is one layer of defence among several.
Follow-up: Who is responsible for the sandbox in a custom agent?
The builder. The builder's code decides where each tool call runs, and a sandbox exists only if the builder adds one.
95. What is agentic search?
Agentic search is search in which the model decides step by step what to list and what to open. The agent starts with a folder and no description of its contents, lists it with a tool such as list_dir, picks files by name, and opens them with read_file. It can go back and open more if the first files lack the answer. See Agentic Search.
96. How does agentic search compare with vector database retrieval?
Agentic search needs no index and always reads current files, but it uses more tool calls and more context per question. It fits project folders and codebases with meaningful names. Vector database retrieval needs preparation and handles very large collections and files whose names carry no information.
97. What is human in the loop?
Human in the loop means an agent pauses before certain tool calls and a person allows or denies each one. The criterion is the side effect of the action, not its difficulty: reading and searching change nothing, while deleting, sending, and paying leave changes that may not be reversible. See Human in the Loop.
98. Where is approval implemented, and what happens with too many prompts?
In the harness, because the model only requests tool calls and cannot reliably pause itself. The policy is commonly one setting per tool, auto or ask, and a denied call does not run while the model is told so it can continue differently. Asking too little lets harmful actions through. Asking too much slows the agent and trains users to approve without reading.
99. What is agentic engineering?
Agentic engineering is software development in which an agent writes most of the code and an experienced developer directs it and reviews the result. The developer steers with guardrails: written conventions, precise requests, tests, and reading the diff. The developer still decides what is built and stays responsible for the accepted code. See Agentic Engineering.
Follow-up: Why must project conventions be written into a file?
Unwritten knowledge is not in the agent's context. A file makes it part of every request.
100. What is vibe coding, and how does it differ from agentic engineering?
Vibe coding means describing the desired result to an AI agent, accepting the output, and not reading the code. In agentic engineering the person steers the code. In vibe coding the person steers the result, and every instruction is about behaviour or appearance. See Vibe Coding.
101. Where does vibe coding fit, and what are its risks?
It suits rapid prototypes and personal tools, and it makes building possible for people who cannot code. Testing by hand does not show security flaws, unhandled edge cases, or accumulated technical debt. Software that handles other people's data or money needs review by someone who can read the code.
Quality and safety
102. What is a benchmark?
A benchmark is a fixed set of questions or tasks with a scoring rule, used to compare models on the same footing. A high score is real for the questions on the list and does not show how the model performs on everything else. See Benchmarks.
103. Why can accurate launch benchmark scores still mislead?
The company that built the model produces and selects the results, so unfavourable ones can be left out. Different launch posts show different sets of benchmarks, so they cannot be compared directly. Most benchmarks are public, so a model can be tuned for them and their questions can enter training data.
Follow-up: How should benchmarks be used?
To make a shortlist. The deciding comparison is running the candidate models on examples from one's own work.
104. What is an eval?
An eval runs an AI system on a fixed test set and scores each output against written requirements. Generated text has no single expected value, so the test measures behaviour. The thing under test is the whole system as shipped: prompts, data, tools, model, and settings. See Evals.
105. How is an eval scored, and how does it differ from a benchmark?
Code checks cover fixed rules, a judge model covers requirements that need language understanding, and human review validates the judge. A benchmark gives one general score for a model, while an eval scores a specific setup on a specific use case. Run on every change, evals expose regressions before customers meet them.
Follow-up: What does a judge model cost?
One call to answer each input and one call to judge it doubles the model calls per run, and runs are repeated often.
106. What is prompt injection?
Prompt injection is an attack in which text from an outside source contains instructions that the model follows as if they came from its user or developer. The cause is that the model receives instructions and data as one stream of text. Any content the system reads can carry it: emails, PDFs, web pages, search results, and tool results. See Prompt Injection.
Follow-up: How does it differ from a hallucination?
A hallucination originates in the model. An injection arrives with the input.
107. How is prompt injection defended against?
By layers, because no single measure is complete. Decide which sources the model may read, give it only the data and tools the task needs (least privilege), screen incoming text, and require human approval for risky actions. Role labels and delimiters help the model weigh sources but are not a guarantee.
108. What is data exfiltration through an agent?
Data exfiltration is data leaving its owner's control for a destination the owner did not choose. An injected instruction tells the agent to gather files and include them in its output, and no break-in is needed because the agent uses its own permissions. It requires three things together: access to private data, exposure to untrusted content, and an outbound channel. Removing one closes the route. See Data Exfiltration.
109. What leaves the organisation when a hosted model is used, with no attacker involved?
Everything in the context window is sent to the provider's servers, and it is sent again on every turn. Provider settings control training use and retention, but they do not stop the sending. Defaults differ by provider and account type, and a person has to change them.
110. What is the difference between a closed model and an open-weights model?
A closed model runs only on its provider's servers, and users receive an API address. An open-weights model has its trained weights published for download, so anyone with enough memory and usually a GPU can run it. Local running keeps data on that hardware, works offline, allows fine-tuning, and avoids per-token fees, and it moves hardware cost and operating work to the user. See Open Weights and Open-Source Models.
111. Is an open-weights model open source?
Not necessarily. Open weights covers the weights only. Open source in the strict sense also covers training code, information about training data, and a licence that permits free use and modification. Each open-weights model has its own licence, and some restrict commercial or other uses.