What Is a Large Language Model?

This course teaches you to run language models on your own computer and to build a coding assistant around them in Java. Before installing anything, you need a clear picture of what a language model is and what happens when you "talk" to one. This lesson and the next one give you that picture.

There is nothing to install in these two lessons and no program to run. Every idea comes with small worked examples that you can follow with a pen and paper. The numbers in the examples are small on purpose, so you can check every step yourself.

The only thing this course assumes is that you can read basic Java. You do not need to know machine learning, statistics, or anything about graphics cards. Every term is explained the first time it appears, and the glossary at the end of the lesson collects them.

After this lesson you can explain in your own words:

• what a large language model does, and what it does not do,
• why the same question can get different answers,
• what a "token" is and why it matters,
• what "a 7B model" means and how much memory it needs,
• how a model learns, and why a "base" model and an "instruct" model behave differently.

Example 1: the phone keyboard, a tiny language model

Open a messaging app on a phone and type See you. Above the keyboard, the phone suggests three words, for example tomorrow, soon, and later. Tap one, and it suggests three more.

The phone is doing the job of a language model. A language model is a program that is given some text and predicts what comes next. That is the whole job. It does not "understand" in the human sense and it does not look anything up. It predicts a continuation that fits the text before it.

The phone's suggestions come from patterns: after See you, people often type tomorrow or soon, so those words are suggested first.

A large language model (an LLM) does the same thing at a much bigger scale. This table shows the difference:

The phone keyboard is a good comparison for the core idea, "predict a likely next piece of text". It stops being a good comparison when it comes to ability. An LLM has seen so much text, and looks at so much context, that its continuations can solve real problems. Continuing the text Here is a Java method that sorts a list of names: well requires writing a correct Java method.

Example 2: every prediction is a list of probabilities

When a model predicts the next piece of text, it does not produce one answer. It produces a probability for every possible next piece. A probability is a number between 0 and 1 that says how likely something is: 0 means "never", 1 means "certain", 0.5 means "half the time". All the probabilities for one prediction add up to exactly 1.

Here is what a model's prediction could look like for the text The capital of France is. These numbers are made up for illustration, but real models produce output of exactly this shape:

Text so far: "The capital of France is"

Next piece      Probability
" Paris"        0.91
" a"            0.03
" the"          0.02
" located"      0.01
" known"        0.01
...             (tens of thousands of other pieces, each very small)
                ----
Total           1.00

The model puts almost all of its probability on Paris, because in the text it learned from, The capital of France is was nearly always followed by Paris.

The other candidates are not wrong, either. The capital of France is a city of two million people and The capital of France is located on the Seine are both fine sentences. The model gives them smaller probabilities because they are less common continuations.

Look closely at Paris: it starts with a space. The model predicts pieces of text, and the space in front of a word usually belongs to the piece. Example 4 explains these pieces.

Example 3: the same word, different context, different prediction

The previous text decides everything. Here are three texts that end with the same word, of, and the predictions a trained model could make for each (illustrative numbers):

"I drank a cup of"              -> " coffee" 0.38, " tea" 0.34, " water" 0.08, ...
"The Lord of"                   -> " the" 0.93, ...                 (as in "The Lord of the Rings")
"Java has two kinds of"         -> " types" 0.21, " memory" 0.09, " exceptions" 0.07, ...

The word of alone tells the model almost nothing. The words before it carry the information. This is the most important thing to know about how these models work: the prediction depends on the entire text so far, and a longer, clearer text produces better predictions.

This is also why the way you write a request to a model matters. Compare two requests:

Request A: "Write a sort function."
Request B: "Write a Java 21 method that sorts a List<String> of names alphabetically,
            ignoring case, without changing the original list."

For Request A, many continuations are likely: Python, C, Java, bubble sort, quicksort. For Request B, the extra words make one kind of continuation far more likely than the others. The text you give a model is called the prompt, and writing a clear prompt narrows the probabilities toward what you want.

Example 4: text is cut into tokens

A computer does arithmetic on numbers, not on letters. So before the model runs, the text is cut into pieces called tokens, and each token is replaced by a number called its token id.

The program that does this cutting is the tokenizer. Each model comes with its own tokenizer and its own fixed list of all the tokens it knows, called the vocabulary. Vocabularies typically hold between about 30,000 and 260,000 tokens.

Common words are usually a single token. Rare or long words are split into several tokens. Here is how a tokenizer might cut four different texts. The exact cuts and ids depend on the model, so treat these as illustrations:

Text:    "The cat sat on the mat."
Tokens:  "The" | " cat" | " sat" | " on" | " the" | " mat" | "."
Count:   7 tokens

Text:    "Tokenizers split unbelievable words."
Tokens:  "Token" | "izers" | " split" | " un" | "believ" | "able" | " words" | "."
Count:   8 tokens

Text:    "int total = price * 12;"
Tokens:  "int" | " total" | " =" | " price" | " *" | " " | "12" | ";"
Count:   8 tokens

Text:    "System.out.println(\"Hi\");"
Tokens:  "System" | ".out" | ".println" | "(\"" | "Hi" | "\");"
Count:   6 tokens

And here is the same first sentence as the model receives it, as token ids:

"The cat sat on the mat."  ->  [791, 8415, 7731, 389, 279, 5634, 13]

The model never sees the letters. It sees [791, 8415, 7731, ...]. After the model predicts a new token id, the tokenizer turns that id back into text for you to read.

Why not use whole words? Because there are too many: every name, every typo, every identifier like getUserById would need its own entry. Why not single letters? Because then a sentence would be very long, and the model would need many more steps to read and write it. Pieces of words sit in between: a manageable vocabulary that can still spell any text, including words the model has never seen, by combining pieces.

Example 5: why tokens matter to you

Tokens are not a hidden detail. They affect cost, limits, speed, and even the mistakes a model makes. Four worked cases:

Estimating size. In English, one token is about four characters, or about three quarters of a word, on average. Code usually needs more tokens per character because of symbols and indentation. So:

A 750-word English email      -> about 1,000 tokens
A 300-line Java file          -> about 300 lines × 10 tokens per line ≈ 3,000 tokens
                                 (rough; short lines need fewer, long lines more)

Limits. A model can only look at a fixed number of tokens at once (the next lesson calls this the context window and explains it in Example 5). If the limit is 8,192 tokens, the 300-line file plus your question plus the answer must fit within that.

Speed. A model writes one token per step, so a 400-token answer takes 400 steps. Speeds are reported in tokens per second. At 20 tokens per second, that answer takes 20 seconds.

Mistakes. Ask a model how many times the letter r appears in strawberry. The correct answer is 3. Models often get this wrong, and tokens explain why. The model might see the word as two tokens:

"strawberry"  ->  "straw" | "berry"

The model receives two ids, not ten letters. To count letters, it has to "remember" how each token is spelled, which it learned only indirectly. A program that loops over the characters of a String gets it right every time, which is one reason coding agents let the model run code instead of guessing.

The lessons Tokens and Tokenizers and Byte-Pair Encoding show real tokenizer output and build a tokenizer in Java.

Example 6: an answer is written one token at a time

A model predicts only one token per run. To write a whole answer, the program around the model repeats a loop:

1. Give the model all the tokens so far.
2. The model returns a probability for every token in the vocabulary.
3. Pick one token from those probabilities (Example 7 shows how).
4. Append the picked token to the text.
5. Go back to step 1, until the model picks a special end-of-text token or a length limit is reached.

Here is that loop traced by hand. The prompt asks for a Java method, and the model has already written int add(int a, int b) {. The probabilities are illustrative:

Step  Text so far (end of it)                    Most likely next token   Probability
1     ...int add(int a, int b) {                  "\n"                     0.97
2     ...int add(int a, int b) {\n                "    return"             0.88
3     ...{\n    return                            " a"                     0.94
4     ...{\n    return a                          " +"                     0.96
5     ...{\n    return a +                        " b"                     0.99
6     ...{\n    return a + b                      ";"                      0.98
7     ...{\n    return a + b;                     "\n"                     0.95
8     ...{\n    return a + b;\n                   "}"                      0.97
9     ...{\n    return a + b;\n}                  <end of text>            0.81

After step 9 the text reads:

int add(int a, int b) {
    return a + b;
}

Three things to take from this trace:

• Every token is a full run of the model. Nine tokens needed nine runs. A long answer of 1,000 tokens needs 1,000 runs, which is why speed matters so much and why later lessons spend time on making each run fast.
• The model reads its own output. At step 5, the text includes return a +, which the model itself wrote in steps 2 to 4. Each new token is predicted from the prompt plus everything generated so far.
• The model does not plan the whole answer first. It commits to one token at a time. If it picks a poor token early, every later prediction builds on that choice. This is one reason a model sometimes starts a sentence confidently and then contradicts itself.

Example 7: picking the next token, greedy or sampling

Step 3 of the loop says "pick one token". There are two basic ways to pick.

Greedy decoding always takes the token with the highest probability. It is fully predictable: the same prompt gives the same answer every time. Its weakness is that the most likely word at every step does not add up to the best text. Greedy output tends to be bland and can get stuck repeating itself:

Greedy, prompt "My favorite food is":
"My favorite food is pizza. I love pizza. I love pizza. I love pizza. ..."

Sampling picks a token at random, but in proportion to the probabilities. A token with probability 0.6 is picked 60 percent of the time, one with 0.3 is picked 30 percent of the time.

You can picture sampling as a spinning wheel with one slice per token, where each slice's size is the token's probability:

Prediction after "My favorite food is":   " pizza" 0.6, " sushi" 0.3, " pasta" 0.1

Wheel:   [ pizza pizza pizza pizza pizza pizza | sushi sushi sushi | pasta ]
          <---------------- 60% --------------> <----- 30% -----> <-10%->

Ten spins might give:  pizza, sushi, pizza, pizza, pasta, pizza, sushi, pizza, pizza, sushi

Over many spins, pizza comes up about six times in ten, but sushi and pasta get their turns. This is why asking a chat assistant the same question twice can give two different answers: the program sampled different tokens.

Each picked token changes all later predictions (Example 6), so one different early token can lead to a completely different answer.

Example 8: temperature, the randomness dial

Most programs that run models have a setting called temperature. It controls how strongly sampling favors the likely tokens.

To see what it does, you need one more fact. The model does not output probabilities directly. It outputs a raw score for each token, called a logit. A higher logit means "more likely". A small formula called softmax turns logits into probabilities: raise the number e (about 2.718) to the power of each logit, then divide each result by their total, so they add up to 1. Temperature divides every logit by the temperature value before softmax.

Here is the full arithmetic for three tokens with logits 2.0, 1.0, and 0.0:

Temperature 1.0  (logits unchanged: 2.0, 1.0, 0.0)
  e^2.0 = 7.389    e^1.0 = 2.718    e^0.0 = 1.000     total = 11.107
  probabilities:   7.389/11.107 = 0.665   2.718/11.107 = 0.245   1.000/11.107 = 0.090

Temperature 0.5  (divide by 0.5: 4.0, 2.0, 0.0)
  e^4.0 = 54.598   e^2.0 = 7.389    e^0.0 = 1.000     total = 62.987
  probabilities:   0.867                 0.117                  0.016

Temperature 2.0  (divide by 2.0: 1.0, 0.5, 0.0)
  e^1.0 = 2.718    e^0.5 = 1.649    e^0.0 = 1.000     total = 5.367
  probabilities:   0.506                 0.307                  0.186

Read the three rows side by side:

A temperature close to 0 makes the top token's probability close to 1, which behaves like greedy decoding. For code, where one wrong token breaks the program, low temperatures (0 to 0.3) are common. For brainstorming or stories, values around 0.7 to 1.0 are common.

The lessons on sampling implement softmax, temperature, and the other sampling settings in Java.

Example 9: where the probabilities come from

So far the probabilities have appeared from nowhere. Here is the simplest possible way to get them: count. Take this tiny piece of training text, three sentences long:

the cat sat .  the cat ran .  the dog sat .

Now count, for each word, which word came right after it:

After "the":   "cat" 2 times, "dog" 1 time               (3 in total)
After "cat":   "sat" 1 time,  "ran" 1 time               (2 in total)
After "dog":   "sat" 1 time                              (1 in total)
After "sat":   "."   2 times                             (2 in total)

Divide each count by the total to get probabilities:

After "the":   "cat" 2/3 = 0.67,  "dog" 1/3 = 0.33
After "cat":   "sat" 1/2 = 0.50,  "ran" 1/2 = 0.50
After "dog":   "sat" 1/1 = 1.00

That is a working language model. Start it with the, sample, and it can produce the cat ran . or the dog sat ., sentences that follow the patterns of its training text.

Real LLMs are far more capable than this counting table, but the idea is the same: the probabilities reflect patterns in the training text. The counting table only looks at one previous word. It has no way to use the whole context, and it has nothing to say about a word it never saw. An LLM replaces the table with a large mathematical function that looks at all the previous tokens and generalizes to text it has never seen. The next lessons build exactly this progression in Java, starting from a counting model in What a Language Model Computes.

Example 10: parameters, and what "7B" means

An LLM computes its probabilities with arithmetic: mostly multiplying long lists of numbers and adding the results. Most of the numbers in that arithmetic are fixed values that the model learned during training. These learned values are the model's parameters, also called its weights.

Here is the idea at the smallest possible size. Suppose a "model" scores the next token with this formula, where the two weights were learned:

score = weight1 × input1 + weight2 × input2

weights (learned):  weight1 = 0.8,  weight2 = -0.3
inputs:             input1  = 2.0,  input2  = 1.0

score = 0.8 × 2.0 + (-0.3) × 1.0 = 1.6 - 0.3 = 1.3

This toy model has 2 parameters. A real LLM uses the same kind of multiply-and-add, repeated across billions of weights. Model names state the count, with B for billion:

Qwen2.5-Coder-7B        about 7 billion parameters
Llama-3.1-8B            about 8 billion parameters
Llama-3.1-70B           about 70 billion parameters

A model file on disk is mostly these numbers, stored one after another. Its size follows directly from the count, which Example 11 calculates.

More parameters give the model more room to store patterns, so larger models are usually more capable. They also need more memory and run more slowly. Choosing a model is mostly a trade between those two.

Example 11: will this model fit on my computer?

Every parameter must be stored in memory while the model runs. So the memory a model needs is, to a first approximation:

memory for the weights = number of parameters × bytes per parameter

How many bytes per parameter? That depends on the number format. Training uses 16-bit numbers (2 bytes each). Storing each parameter with fewer bits is called quantization. A common choice for running models at home is about 4 bits per parameter (half a byte), plus a little extra for scale values stored alongside, so real 4-bit files come to about 0.6 bytes per parameter.

Here is the arithmetic for common sizes:

Parameters    16-bit (2 bytes each)        4-bit (about 0.6 bytes each)
1 billion     1 × 2   =   2 GB             1 × 0.6  =  0.6 GB
3 billion     3 × 2   =   6 GB             3 × 0.6  =  1.8 GB
7 billion     7 × 2   =  14 GB             7 × 0.6  =  4.2 GB
14 billion    14 × 2  =  28 GB             14 × 0.6 =  8.4 GB
32 billion    32 × 2  =  64 GB             32 × 0.6 = 19.2 GB
70 billion    70 × 2  = 140 GB             70 × 0.6 = 42   GB

Now a worked question. You have a laptop with 16 GB of memory, and the operating system, your editor, and a browser already use about 6 GB. That leaves about 10 GB. The model also needs some working memory beyond its weights (for the conversation, as the next lesson explains), so budget about 1 to 2 GB for that.

Available:              16 GB - 6 GB = 10 GB
7B at 16-bit:           14 GB + 1 GB  -> does not fit
7B at 4-bit:           4.2 GB + 1 GB  -> fits, with room to spare
14B at 4-bit:          8.4 GB + 1.5 GB -> fits, with little room left
32B at 4-bit:         19.2 GB         -> does not fit

Quantization is the reason a model that needs a large server at 16 bits can run on an ordinary laptop at 4 bits. It costs a little quality, and the section Hardware, Numbers, and Quantization measures how much and explains how it works from the bits up.

Example 12: training, nudging billions of numbers

A new model starts with random parameters, so its predictions are noise: every token gets roughly the same tiny probability. Training is the process that adjusts the parameters until the predictions match real text.

One training step looks like this:

1. Take a piece of real text:       "The capital of France is Paris"
2. Hide the last token and ask the model to predict it from "The capital of France is".
3. The model (still poorly trained) answers:  " Paris" 0.02, " London" 0.04, " a" 0.05, ...
4. The correct token was " Paris". Its probability, 0.02, is low, so the error is large.
5. Compute, for every parameter, which direction would have made " Paris" more likely.
6. Move every parameter a tiny step in that direction.
7. Now the same text would give " Paris" 0.021 instead of 0.02.

One step changes almost nothing. Training repeats it at every position of every text, trillions of times. Step 5 uses a method called gradient descent, which computes for each parameter how the error changes when that parameter changes; the lesson What a Language Model Computes runs it in Java.

To predict the next token well across trillions of tokens of varied text, the model has to absorb a great deal: spelling, grammar, facts that appear often, the structure of Java code, how arguments are built. None of this is programmed in by hand. It is stored, spread across the parameters, as a side effect of getting better at prediction.

Training a large model takes thousands of high-end graphics cards running for weeks or months. Only a small number of companies and research labs do it. Many of them publish the finished parameters as files that anyone can download. This course downloads those files and runs them. Running a finished model to get predictions is called inference, and it needs far less hardware than training.

Example 13: base models and instruct models

Training happens in stages, and the stages produce different kinds of models that you will see when you download one.

Stage 1, pretraining, is the next-token training from Example 12 on a huge amount of general text. The result is a base model. A base model is excellent at continuing text, but it does not know it is supposed to help you. It continues whatever you give it, in the style it most often saw:

Prompt to a base model:
  "What is a Java record?"

A likely continuation:
  " What is a Java interface? What is the difference between an abstract class
   and an interface? These are common Java interview questions ..."

The base model saw many pages where a question is followed by more questions (lists of interview questions, quizzes), so it continues the pattern.

Stage 2, instruction tuning, continues training on many examples of a request followed by a helpful answer. It is often followed by training on judgments of which of two answers is better, made by people or by automated checks. The result is an instruct model, also called a chat model:

Prompt to an instruct model:
  "What is a Java record?"

A likely answer:
  "A record is a class meant to hold data. Declaring
     record Point(int x, int y) {}
   gives you a constructor, the accessors x() and y(), and equals, hashCode,
   and toString, all generated by the compiler."

Same knowledge, different behavior. Model files usually say which kind they are in the name:

Qwen2.5-Coder-7B              base model: continues text and code
Qwen2.5-Coder-7B-Instruct     instruct model: follows requests, answers questions

For a chat assistant or a coding agent, you want the instruct version. Base models are used for completing code inside an editor and as a starting point for further training. The lesson How Models Are Trained covers the stages and the other variants, such as "thinking" models, in detail.

Example 14: what is inside the model, at a glance

You can use a model without knowing its internals, but a rough map makes the later lessons easier to follow. Almost every current LLM uses a design called the transformer. Here is the path of one prediction through it:

"The cat sat"
     |
     v
 tokenizer           -> token ids [791, 8415, 7731]
     |
     v
 embedding lookup    -> each id becomes a list of numbers (a vector),
                        for example 4,096 numbers per token
     |
     v
 layer 1             attention: each token gathers information from earlier tokens
                     feed-forward: a calculation on each token's numbers
 layer 2             (same structure, different learned weights)
 ...
 layer N             (real models have from about 16 to over 100 layers)
     |
     v
 output              -> one score (logit) for every token in the vocabulary
     |
     v
 softmax             -> probabilities: " on" 0.31, " down" 0.22, " still" 0.05, ...

Three terms in this diagram are new.

A vector is a list of numbers. The model represents each token as a vector, and during training the numbers come to encode features such as meaning and grammatical role. Tokens used in similar ways end up with similar vectors: the vectors for cat and dog are closer to each other than to the vector for Tuesday.

An embedding is the starting vector for a token, read from a big table with one row per vocabulary entry. Token id 8415 ( cat) selects row 8415.

Attention is the step where each token looks back at the tokens before it and pulls in information from the ones that matter. Consider:

"I sat on the bank of the river."      -> "bank" means the side of a river
"I deposited cash at the bank."        -> "bank" means a financial company

The embedding for bank is the same in both sentences, because it comes from the same table row. Attention lets the vector for bank take in information from river in the first sentence and from cash in the second, so after a few layers the two vectors are different. Attention is the reason a model can use context that appeared thousands of tokens earlier, for example a variable declared at the top of a long Java file.

Each layer refines every token's vector a little more. After the last layer, the vector of the last token is turned into one score per vocabulary token, and softmax turns those scores into the probabilities you saw in Example 2.

The foundations lessons build each piece in Java: the math, self-attention, and the transformer block.

The next lesson, Talking to a Model: Prompts, Context, and Limits, shows how this text predictor is used in practice: chat messages, context windows, prompt engineering, RAG, and the ways models fail. The lesson after it, Agents and LLM Engineering, covers tools, agents, and the rest of the vocabulary you will meet.

Glossary

Interview points

• A language model outputs a probability for every token in its vocabulary, given the text so far. Generation repeats that prediction one token at a time and appends each chosen token.
• The prediction depends on the whole context. The same final word gets very different predictions after different text, which is why prompt wording changes results.
• Models work on tokens, not characters or words. One English token is about four characters. Letter counting is hard for a model because it sees token ids, not letters.
• Every generated token needs one full run of the model, so answer length directly drives response time.
• Greedy decoding is repeatable but dull and repetitive. Sampling adds variety. Temperature divides the logits before softmax: below 1 it sharpens the distribution, above 1 it flattens it.
• Weight memory is about parameter count times bytes per parameter: a 7B model needs about 14 GB at 16 bits and about 4 GB at 4 bits.
• A base model continues text. An instruct model has been trained further to follow requests. Chat assistants and agents use instruct models.
• Attention is what lets a token's representation depend on the other tokens in the context, so the same word can be read differently in different sentences.

Summary

• A large language model predicts the next token of a text, as a probability for every token in its vocabulary.
• The prediction depends on all the text before it. A clearer prompt makes the wanted continuation more likely.
• A tokenizer splits text into tokens from a fixed vocabulary and turns them into ids, which are what the model actually receives.
• An answer is generated one token per model run, and each new token is predicted from the prompt plus everything generated so far.
• Greedy decoding always takes the top token. Sampling picks in proportion to probability. Temperature controls how strongly sampling favors the top tokens.
• The probabilities reflect patterns in the training text, from simple counting in a toy model to a learned function in an LLM.
• Parameters are the learned numbers. Their count times the bytes per parameter gives the memory the weights need, and quantization lowers the bytes per parameter.
• Pretraining produces a base model that continues text, and instruction tuning produces an instruct model that follows requests.
• A transformer turns tokens into vectors and refines them through many layers, using attention to bring in context, and ends with a score for every possible next token.