Grounding

The previous lesson showed that a language model can write a confident answer that is false, because it writes likely text and has no built-in check for truth. It also showed the defence: check each claim against a source. That raises a practical problem. If the answer comes only from the model's training, there is no source to check it against.

There are two ways to reduce hallucinations. One is to build better models, which is the job of the companies that train them. The other works on the software around the model instead of on the model. It is called grounding, and it is the subject of this lesson.

Grounding means giving the model real source material to answer from, and tying each statement in the answer to the place it came from. After this lesson you can recognise grounding when a tool does it, do it yourself on purpose, and explain why a grounded answer is quicker to check but still needs checking.

1. What an answer is tied to

The software a model runs inside, such as a chat app, tries to tie every answer to something a person could go and open.

The last row is the rule that defines grounding. An answer whose only source is the model has no source. "The model said so" cannot be opened, read, or compared with anything. A page on a website can. A paragraph on page 7 of a report can.

The word comes from everyday speech, where a "grounded" claim is one that rests on something solid. An ungrounded answer rests only on the model's learned patterns. A grounded answer rests on a text that exists outside the model.

2. An example: summarising a PDF

Attach a document, ask for a summary, and look at what comes back. Suppose the file is a quarterly report called northwind-q3.pdf.

flowchart LR
    subgraph S["The summary that comes back"]
        direction TB
        s1["Several sentences of summary"]
        s2["A reference: see p. 7"]
        s1 --- s2
    end
    subgraph D["northwind-q3.pdf"]
        direction TB
        p3["p. 3"]
        p7["p. 7"]
        p11["p. 11"]
    end
    s2 -- "points back at" --> p7

The diagram shows the summary on the left and the document on the right. Next to one of the statements in the summary there is a small marker: "see p. 7". That marker points back at one specific page of the document.

The marker is called a reference or citation. It is the part of the answer that a reader can act on, because it says which page to turn to. Without it, checking one sentence of the summary means reading the whole report again. With it, checking takes a minute: open page 7 and compare.

3. What the software adds to the request

The model does not add page references because it has a special grounding mode. It adds them because it was told to. The software puts extra wording into the request before sending it to the model. Something like this:

When you answer this request:

  - work from the attached documents
  - after each statement, name the
    page it came from
  - if nothing in them supports it,
    say so rather than fill the gap

This is a stand-in for the real wording, which differs between products and is usually not shown. What matters is who wrote it. The wording is added for you by the software. You did not type it.

The three lines do three different jobs:

1. Work from the documents. The model should take its facts from the supplied text and not from what it recalls from training.
2. Name the page. Each statement should carry a reference that a person can follow.
3. Admit gaps. If the documents do not support a statement, the model should say so and not fill the hole with a guess. This line targets hallucinations directly.

This is the same mechanism as the system prompt from earlier lessons: text the software adds to the context. Grounding instructions are ordinary instructions. There is no separate machinery behind them.

4. An answer together with its evidence

The purpose of grounding is to hand over two things at once: an answer, and the evidence for it. A useful picture is a receipt. A receipt does not prove that a purchase was wise, but it shows exactly what was bought and where, so the purchase can be checked.

The quality of the evidence matters. A list of documents at the end of an answer, such as "sources: the Q3 report", is a reading list. It tells the reader where to start looking, not where the statement is. Good grounding points at the concrete statement that a claim rests on: this page, this paragraph, this sentence.

5. Grounding you do yourself

Grounding needs something to ground in. Some of that material is supplied by the person asking.

Summarise these two for tomorrow's board pack.

  [attached]  northwind-q3.pdf
  [attached]  harlow-q3.pdf

And check it against example.com/quarterly-notes

In this request the person attached two reports and pasted a link to a web page. Both actions are grounding. They are the deliberate part of it, because the person chose the sources.

Supplying the source changes what the answer rests on. Two ways of asking about a contract show the difference:

• Asking "what is the usual notice period in a rental contract" gets an answer from training data about contracts in general.
• Attaching your own contract and asking "what is the notice period in this contract, and which clause says so" gets an answer tied to the text that applies to you.

The second request gives the model the facts and asks for the reference. A person who supplies the source and asks where each claim comes from has done most of the grounding by hand.

6. Grounding the tool does without being asked

Grounding also happens when the person supplies nothing. Many chat tools, including ChatGPT and Claude at the time of writing, can search the web when a question calls for it.

flowchart LR
    Q["Your question"] --> T["The tool<br/>for example ChatGPT or Claude"]
    T -- "search" --> W["The web"]
    W -- "pages" --> T
    T --> M["The model"]
    M --> A["The answer<br/>carrying its sources"]

The diagram reads from left to right. Your question goes to the tool. The tool sends a search to the web and gets pages back, without you asking for it. It passes those pages to the model together with your question. The model writes an answer that carries its sources, usually as links you can click.

Web search grounding helps with one weakness in particular. A model's training stops at a certain date, so it knows nothing about later events. A search brings in pages written after that date.

Whether a tool searches depends on the product, the settings, and the question. A tool may decide that a question about arithmetic needs no search and a question about yesterday's news does.

7. What the word covers

Grounding is a broad word. All four things from this lesson belong to it:

• the wording the software adds to your request
• the page reference in the answer
• the document you attached or the link you pasted
• the search the tool ran by itself

What they share is the goal: the answer rests on material that exists outside the model and can be opened.

8. What grounding does not guarantee

Grounding lowers the chance of a hallucination. It does not remove it. Three limits apply.

The model can still misread the source. It may summarise page 7 incorrectly, or combine two sentences into a claim that neither one makes.

The reference can be wrong. A model can attach "p. 7" to a statement that is on page 9, or that is nowhere in the document. A reference is produced by the same token prediction as the rest of the answer.

The source can be wrong. If a web page contains an error, a well-grounded answer repeats that error faithfully. Grounding ties the answer to the source. It does not judge the source.

So the rule from the previous lesson still holds: an answer is a claim to check. What grounding changes is the cost of checking. Following a page reference takes a minute, so checking becomes something a person will really do.

Interview points

• Grounding ties a model's answer to sources outside the model, such as documents or web pages, so that each statement can be checked.
• It is the second way to reduce hallucinations. Better models lower the rate. Grounding works in the software around the model, with any model.
• An answer whose only source is the model itself is ungrounded, because there is nothing to open and compare.
• A reference points from a statement to a specific location such as a page or paragraph. A general list of sources is weaker because it does not say where the statement is.
• Tools achieve grounding by adding instructions to the request: work from the supplied documents, cite the location of each statement, and say when the documents do not support something.
• Users ground deliberately by attaching documents or pasting links, and by asking for the location of each claim.
• Tools ground automatically when they run a web search and pass the pages to the model. This also covers events after the model's training ended.
• Grounding reduces hallucinations but does not eliminate them. The model can misread a source, give a wrong reference, or faithfully repeat an error in the source.
• The main practical benefit is that verification becomes fast, because the reference says where to look.

Summary

• Grounding means answering from real sources and pointing at them.
• A source is something a person can open: a web page, a document, a paragraph.
• The model alone never counts as a source.
• Software grounds answers by adding instructions to the request.
• Attaching a file or pasting a link is grounding done by the user.
• A tool that searches the web for you is grounding done by the tool.
• Grounded answers still need checking, and the references make checking quick.