Large and Small Language Models

A large language model (LLM) is the piece that almost everything else in modern AI is built on. Chat assistants, coding tools, and AI agents all have an LLM at the centre.

The previous lesson showed that a neural network is layers of adjustable numbers, trained on examples. An LLM is a neural network trained on text. This lesson explains what goes into an LLM and what comes out, how it cuts text into pieces, and why one model can do many different jobs.

The second half covers small language models (SLMs). They are built the same way as large ones but have far fewer numbers inside, which changes where they can run and what they are good at.

After this lesson you can explain what an LLM does in one sentence, say why text is split into tokens, and decide when a small model is a reasonable choice.

1. What a large language model is

An LLM is a neural network for language. Most LLMs use a design called the transformer, which is a particular kind of neural network that works well on text. The idea of an LLM is short: it takes input text and produces output text.

That sounds narrow, but text carries more than facts. Think of someone planning how to get to a nine o'clock meeting:

The 7:40 gets me there with ten minutes to spare.
But that means leaving before the bakery opens.
Coffee at the station, then. Take the 7:40.

These three lines are thinking, written out as text. Nobody spoke them. A large part of how people reason can be written down this way, as one sentence leading to the next.

This is why LLMs matter more than earlier kinds of AI. An LLM is not built for one narrow task. It is a thinking process that can be handed to a machine. You point it at a problem, it works through that problem in text, and then you can point it at a different problem.

2. Tokens in, tokens out

You type a question and read an answer:

flowchart LR
    A["What you type<br/>What time does the bakery open?"] --> T1["Split into tokens<br/>What | time | does | the | bakery | open | ?"]
    T1 --> M["The model"]
    M --> T2["Tokens come out"]
    T2 --> B["What you read<br/>Seven on weekdays,<br/>nine at the weekend."]

Neither side of the model ever sees a sentence. Before your text reaches the model, it is cut into small pieces called tokens. The model reads tokens and writes tokens. The tokens it writes are joined back into text for you.

So the precise description of an LLM is "tokens in, tokens out". The lesson on tokens covers them in detail. The next sections give the first picture.

3. What the model was trained on

An LLM is trained on text, and a very large amount of it. A page from the web about a bakery's opening hours. A page from a novel about a river rising in spring. Then another page, and another. The training text is close to the whole public internet, with books added.

The model is not given a list of facts or grammar rules. It is given text, and it learns patterns from that text.

4. How a model cuts text

The obvious way to cut text is into words:

large | language | models | are | trained | on | text

That gives seven words. No model cuts text this way. A model cuts text into tokens, and a token can be a whole word or a fragment of one. The cut might look like this:

lar | ge | lang | uage | mod | els | are | trained | on | text

Here "are", "trained", "on", and "text" stay whole, and the other words are split into two pieces each. This split is an illustration. In real models, very common words such as "large" usually stay as one token, and each model splits text slightly differently.

The reason for cutting this way is the size of the list of pieces. A model has a fixed list of all the tokens it knows, called its vocabulary. If the vocabulary held every word ever written, it would be enormous, and the model would be stuck on any word it had never seen. With pieces, the model can build any word from a list of manageable size. That includes new words, names, and typos.

5. What the cutting is for: learning how tokens relate

Take one piece of text in tokens:

under | stand | ing | how | text | is | for | med

During training, the model learns how tokens relate to each other: which ones appear together, in what order, and what they mean next to each other. Every pair of tokens in the text carries a learned relation. "under" relates to "stand". "for" relates to "med". "stand" relates to "text", further away.

flowchart LR
    t1["under"] --- t2["stand"] --- t3["ing"] --- t4["how"] --- t5["text"] --- t6["is"] --- t7["for"] --- t8["med"]
    t1 -. "related" .- t4
    t2 -. "related" .- t7
    t5 -. "related" .- t8

The diagram draws only a few of the relations. The model learns these for every piece of text it was ever shown. What an LLM "knows" is this very large collection of learned relations between tokens.

6. Why numbers are split into pieces too

Long numbers show clearly why pieces work better than whole items. Suppose the model received this number as a single token:

8472913605

Then it would be one single thing, and the model could only work with it if it had met that exact number in training. The same would be true for 4180, 9263, 3017, and every other number. There are endlessly many numbers, so no amount of training could cover them all.

Now split the same number into small pieces:

84 | 72 | 91 | 36 | 05

Pieces such as "84" and "05" turn up again and again in everything the model read, in every language. The model has met each piece many times, next to many other tokens, so it knows them well. This means it can handle any number built from those pieces, even a number it has never seen whole.

The number is the same in both cases. Only the size of the pieces changed, and that change gives a much more capable model.

7. One model, a different job each time

You do not need a separate AI for every task. One LLM can do very different jobs.

Translating. It reads a sentence in one language and writes the same meaning in another.

Der Zug fährt um sieben.   ->   The train leaves at seven.

Writing. It produces new text, such as code, a summary, or the three facts buried in a long page. For example, asked for a function that adds up amounts, it writes:

def total(rows):
    return sum(
        r.amount
        for r in rows
    )

Sorting. It reads a customer message and decides which team it belongs to.

The model can do all three because each one works the same way underneath: text goes in and text comes out. A label is a decision, and a decision is more text. When the model decides "this message is about sales", it writes the word sales.

8. Choosing what to do next

The same idea lets a model choose actions. You give the model a question and a list of tools it could use. You do not say which tool fits.

Nothing tells the model which tool this task needs. It reads the question, works out that searching invoices is the right step, and writes the name of that tool. The choice itself is the output, and it is text again.

The model does not run the tool. The software around the model sees the name and runs the tool, as described in the previous lesson. This is the basis of AI agents.

9. Small language models: "large" is a measurement

Large language models have a counterpart at the other end of the same scale: the small language model (SLM). The term is semi-official. There is no fixed line between "small" and "large".

The word "large" is a measurement. It does not say how good the model is. It counts the model's parameters. Because it is a measurement, it can also come out small.

A parameter is a number inside the model. It is not a setting that anyone chose. It is a number that training left behind. Here are sixteen of them from one layer of a model:

 0.0142  -0.8831   0.4407  -0.0026
-0.3319   1.2065  -0.7714   0.0901
 0.5528  -0.1176  -0.9302   0.2415
 0.0763   0.6491   0.1358  -0.4870

The parameters start as random values. Training nudges each one up or down, over and over, until the model gives good answers. Whatever values are left when training stops are the parameters. A modern large model holds billions of them, and the largest are reported to reach a trillion or more. When someone says a model has "70 billion parameters", this is what is being counted.

Parameters and tokens are often confused:

• Parameters are the numbers inside the model. They are what the model learned. The count is fixed once training is done.
• Tokens are the chunks of text the model reads and writes. The model does not contain tokens. Text passes through it as tokens.

Three token-related numbers are often confused with the parameter count:

One way to hold the difference: tokens are the words on the page, and parameters are the trained brain that reads the page.

10. Where all those numbers have to sit

Before a model can produce a single word, all of its parameters have to be loaded into a computer's memory at once.

This is why the most capable models run in data centres and reach you over the internet. They are too large for any normal computer.

11. A small model is the same thing with far less of it

A small language model has the same design as a large one. Only the parameter count changes. Picture a large model as a wall covered with hundreds of blocks of parameters, with hardware to match. A small model is the same drawing with only a few of those blocks.

Fewer parameters need less memory. That brings the model back within reach of a machine you own: a good laptop or desktop.

The count is the capacity the model has to work with, so reducing it has a cost.

"Shades of meaning" has a concrete sense. Take one line of a reply: "Sure, whatever you think is best." It can be read in three ways.

Room in the network is room to keep near-identical meanings like these apart. A large model is more likely to pick the right reading from the surrounding text. A small model has fewer parameters to store such differences, so it misses them more often.

12. Small, then sharpened: fine-tuning

A small model on its own has less general knowledge and fewer shades of meaning to draw on. It can be improved for one job in three steps:

1. Start with a small model.
2. Fine-tune it. Fine-tuning means training the model further on the kind of material it will be given in real use, such as your company's support tickets, documents, or code.
3. Result: it becomes sharp inside that one area. It is still not as capable as a large model in general. For that one job it does not have to be.

A small fine-tuned model suits jobs that are narrow and repeat the same pattern:

For the sorting job, the result looks like this:

These jobs fit a small model because each one is focused and predictable. The model does not need broad knowledge of the world. It needs to do one clear task well, many times.

13. Why small models are attractive: the model sits where the work is

A small model can run on your own device or inside your company's own network.

flowchart LR
    subgraph L["Your device or your network"]
        T["Your text<br/>never copied out"] --> S["Small model<br/>running locally"] --> A["The answer"]
    end
    R["Somebody else's hardware"]
    L -. "no request leaves,<br/>so no request is billed" .- R

The whole round trip happens inside the box:

• Your text stays where it is. It goes to the model on your machine, and the answer comes back on the same machine.
• Nothing is sent outside. Your data never travels to another company's servers, so it stays private.
• There is no usage bill. No request goes to an outside service, so nothing is charged per request.

Privacy and cost are the main reasons people choose small models. Not every job needs the largest model in the world. Some jobs only need a model that runs where the work already is.

Interview points

• An LLM is a neural network, usually a transformer, that takes text in and produces text out. More precisely, it takes tokens in and produces tokens out.
• LLMs are trained on text at very large scale, close to the whole public internet plus books.
• Text is split into tokens, which are whole words or fragments. Fragments keep the vocabulary a manageable size and let the model handle words it has never seen.
• A long number treated as one token would have to be seen in training in that exact form. Split into short pieces, it is built from pieces the model has met many times.
• Training teaches the model how tokens relate to each other. Those learned relations are what the model knows.
• Translation, writing, classification, and tool choice are the same operation: text in, text out. A label or a tool name is text.
• The model names a tool. The software around the model runs it.
• "Large" counts parameters. A parameter is a number that training set. Parameters are inside the model, and tokens pass through it.
• All parameters must be in memory before the model can answer, so the largest models run in data centres.
• A small language model has the same design with fewer parameters. It fits on one machine but has less capacity for detail and nuance.
• Fine-tuning a small model on one kind of work makes it good at that narrow job. Running it locally keeps data private and removes per-request cost.

Summary

• An LLM reads tokens and writes tokens. A token is a word or a piece of a word.
• The model learns relations between tokens from a very large amount of text.
• One model does many jobs because every job is expressed as text in and text out.
• Parameters are the trained numbers inside a model. Large models have billions or more.
• Large models need data-centre hardware. Small models fit on a machine you own.
• Fewer parameters means less capacity. Fine-tuning recovers quality for one narrow job.
• Small local models keep data on your own hardware and have no per-request charge.