Open Weights and Open-Source Models

When you use ChatGPT or Claude, your message travels over the internet to a data centre, a model there produces the answer, and the answer travels back. The model never comes near your computer. You could not copy it onto your laptop even if you wanted to.

Other models work differently. Their makers publish the model as files that anyone can download. With suitable software and hardware, such a model runs on your own machine, with no internet connection needed.

The first kind is called a closed model. The second kind is called an open-weights model. The many models used in AI systems and agents fall into one group or the other, and the group decides where the model can run.

After this lesson you can tell the two kinds apart, explain what is downloaded with an open-weights model, estimate whether a model fits on a given machine, and say what "open source" adds to "open weights".

1. Two things a provider can do with a model

A company that has trained a model has two ways of letting others use it.

Hosting means the provider runs the model on its own servers and lets customers send requests to it over the internet. The lesson on APIs for AI models described this arrangement. The customer gets answers. The customer never gets the model.

Publishing means the provider makes the model's files available for download. Anyone can fetch a copy and run it wherever they like.

The two are not exclusive. A company that publishes weights often also hosts the same model as a paid service, for customers who do not want to run it themselves. A company can also keep its largest models closed and publish some smaller ones.

2. Where the weights sit

To see what is published, recall what a model is. The lesson on parameters and weights explained that a model consists of a very large set of numbers, called weights, that were adjusted during training. The weights are the result of the training. Whoever has the weights has the model.

flowchart LR
    subgraph CLOSED["Closed model"]
        direction LR
        A1["Your machine<br/>no model on it"] --> B1["Their server<br/>holds the weights"]
        B1 --> C1["An answer"]
    end
    subgraph OPEN["Open-weights model"]
        direction LR
        A2["The weights<br/>published, downloadable"] -- "downloaded" --> B2["Your machine<br/>holds the same weights"]
        B2 --> C2["An answer"]
    end

The box labelled "Closed model" shows the first case. Your machine has no model on it. It sends a request to the provider's server, the weights there are used to compute an answer, and the answer comes back. The weights never leave the provider's server.

The box labelled "Open-weights model" shows the second case. The weights, which are the result of the training, are published. You download them once. From then on your own machine holds the same weights and computes the answers. The model runs on your hardware, not on the provider's.

3. What is downloaded

Downloading a model is done with software made for the purpose. Ollama and LM Studio are two widely used programs for this, and there are others. They fetch the weights from a public source, store them, and run the model. Product names and features change, so check what is current.

After a download, the disk holds something like this:

~/models/                               a folder on your own disk
    an-open-weights-model/              one model, downloaded once
        weights-00001-of-00004.bin      the training result, now on your machine

The folder ~/models/ is an ordinary folder in your home directory. Inside it is one folder per model. Inside that are the weight files. The file name weights-00001-of-00004.bin shows that this model's weights are split across four files, of which this is the first. Large models are commonly split this way. The ending .bin stands for binary, which means the file holds raw numbers and is not readable text. Real files use various formats and names.

These files are large. The size follows from the number of weights and from how much space each weight takes. Here is a rough calculation with round numbers chosen for the example:

7 billion weights x 2 bytes each   = 14 billion bytes  = about 14 GB
7 billion weights x 0.5 bytes each = 3.5 billion bytes = about 3.5 GB

GB stands for gigabyte, about one billion bytes. The first line stores each weight in 2 bytes. The second line uses quantization, which stores each weight with less precision so that it takes less space, at some cost in quality. Quantized versions are what most people download for use on a personal computer.

4. Having the file is not the same as running it

A downloaded weight file does nothing by itself. It has to be executed, and that takes the right hardware. The weights are only half of what is needed.

To produce an answer, the computer has to load all the weights into memory and do a very large number of calculations with them for every token it generates. Two things decide whether a machine can do this at a usable speed:

• Memory. The weights have to fit. A model whose files take 14 GB needs somewhat more than 14 GB of free memory.
• A fast processor for this kind of calculation. This is usually a GPU, a graphics processing unit. GPUs were made for drawing images and are well suited to the many parallel calculations a model needs. On a GPU, the relevant memory is the GPU's own.

This leads to a rough guide:

The lesson on large and small language models covered why smaller models need less hardware and what they give up in ability.

For the largest models, an ordinary computer is not enough. The alternative is to rent a server with strong hardware from a cloud company and run the model there. The model then runs on a machine you control, though not on the one on your desk. You pay rent for the server for as long as it runs.

One point about rented servers: the data now passes through the cloud company's machines. That is a different arrangement from sending it to a model provider, because you control the server and the software on it, but the data is no longer only on hardware you own.

5. The two kinds compared

"An address to send requests to" means the web address of the provider's API. That address and a key are everything a customer of a closed model receives.

At the time of writing, the main models of OpenAI and Anthropic are closed models. Companies and labs that have published open-weights models include Meta, Google, Mistral, DeepSeek, and Alibaba, among others. This list changes, and OpenAI has also released some open-weights models, so treat any such list as a snapshot.

An open-weights model is as much a large language model as a closed one. Both are trained in the same way, built from the same kind of neural network, and used in the same way. Nothing about the model itself differs between the two columns. Open-weights models are not a lesser category by nature, although at any given time the most capable models available have often been closed ones.

6. What changes in practice

So the whole difference is whose machine the model runs on: the provider's, or yours if yours has the hardware for it. Several practical consequences follow from that one difference.

The first row connects to the previous lesson on data exfiltration. With a hosted model, everything in the context is sent to the provider on every turn. With a model that runs on your own machine, there is no sending. For work with confidential material, this is a common reason for organisations to choose open-weights models.

The criteria for choosing follow from the table. A hosted closed model fits when the highest capability is needed, when nobody on the team wants to manage hardware, and when sending the data to a provider is acceptable. A self-run open-weights model fits when the data must not leave your own machines, when work has to continue offline, when the model needs to be modified, or when the volume of requests is high enough that owning the hardware costs less than paying per token.

Running a model yourself also moves responsibility. Safety filters that a provider applies on its servers are not there unless you add them. Updates, monitoring, and capacity are your job.

7. Open weights and open source

The two terms are often used as if they meant the same thing. They do not.

Open weights means one thing: the trained weights can be downloaded. You can run the model and usually adjust it. You cannot necessarily see how it was made.

Open source is a term from software. In its strict sense it means that the material needed to study, change, and rebuild a piece of software is published, under a licence that lets anyone use it for any purpose. For a model, the equivalent would cover more than the weights:

An analogy: open weights is like receiving a finished cake. You can eat it, share it, and add your own icing. Open source would also give you the recipe and the list of ingredients, so that you could bake the same cake yourself and change the recipe. The analogy has a limit. Even with the full recipe for a large model, baking it again needs training hardware that very few organisations have.

Most models that are loosely called "open source" are, strictly speaking, open weights. The weights are published, while the training data and often the training code are not.

The licence is the other part that differs from model to model. Publishing weights does not mean that every use is allowed. Each model comes with a licence that states what is allowed. Some licences permit almost any use. Others forbid commercial use, restrict certain applications, or set conditions for large companies. A product built on an open-weights model has to stay within that model's licence, so the licence is read before the model is chosen.

Interview points

• A closed model runs only on its provider's servers. Users receive an API address and never the weights. An open-weights model has its trained weights published for download.
• The weights are the result of training. Whoever holds the weights can run the model, so publishing the weights lets the model run outside the provider's servers.
• Programs such as Ollama and LM Studio download weight files and run them locally. The files are large, often split into several parts, and commonly quantized to reduce their size.
• Downloading the weights is not enough. Running a model requires enough memory to hold the weights and a fast processor, usually a GPU. The largest open-weights models need server hardware, which can be rented.
• Closed and open-weights models are the same kind of model. The difference is where the model runs and who controls it.
• Running an open-weights model on one's own hardware keeps prompts and documents on that hardware, works offline, allows modification such as fine-tuning, and avoids per-token fees. It also moves the cost of hardware and the work of operating the model to the user.
• Open weights and open source are different. Open weights covers the weights only. Open source in the strict sense also covers training code, information about training data, and a licence that permits free use and modification.
• Each open-weights model has its own licence, and some licences restrict commercial or other uses.

Summary

• Providers either host a model, publish its weights, or do both.
• A closed model stays on the provider's servers.
• An open-weights model can be downloaded as weight files.
• The weights are the trained model.
• Running the weights requires enough memory and usually a GPU.
• Large open-weights models may need a rented server.
• Both kinds are large language models of the same nature.
• A locally run model keeps your data on your own machine.
• Open weights means the weights are published. Open source means more than that.
• The licence of an open-weights model states what it may be used for.