Lesson 27 of 73
The safetensors Format
A model is a very large collection of numbers called weights. Those numbers have to be saved to a file so that another program can load them later. The file format decides two things: how fast and cheaply the weights can be loaded, and whether loading an unknown file is safe.
safetensors is the format most model repositories on the Hugging Face Hub use for weights. It was designed to replace PyTorch's pickle-based checkpoints. A checkpoint is a saved copy of a model, and pickle is Python's way of saving objects, which can execute code when a file is loaded. This lesson starts with that safety problem, using Java's own serialization as a familiar comparison.
Then it reads and writes safetensors files byte by byte in Java. You will learn where each byte sits, how a program finds one tensor (one named block of numbers, such as a matrix) without reading the whole file, which checks make a file trustworthy, and how to read only the header of a file that sits on a remote server. The lesson ends with Project 6, SafetensorsInfo, a command-line inspector for one or more files. The first table shows the three parts of every file.
| Part of the file | Content |
|---|---|
| Bytes 0 to 7 | N, the header length, unsigned 64-bit little-endian |
| Bytes 8 to 8 + N | UTF-8 JSON header, starting with {, padded with spaces |
| Rest of the file | Tensor data, packed, row-major, little-endian |
The format rules in this lesson come from the specification in the README of the safetensors crate (a crate is a Rust library) and from its Rust source (safetensors/src/tensor.rs, version 0.8.0). Files written by the Java examples were checked by opening them with the Python safetensors package while the lesson was prepared.
Example 1: Java's own deserialization runs code
Most people expect that loading a file only reads data. That is not always true. If the file format can describe objects, the loader may have to run code to build them, and the file then influences which code runs.
Java has the same problem, so it is a good place to see it. Serialization turns an object into bytes and back. The program below saves an object, then loads it. The class defines a readObject method, which Java calls during loading. Look at the order of the printed lines: the message from readObject appears before the line that reports the loaded value.
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable;
public class LoadRunsCode {
static class Checkpoint implements Serializable {
float[] weights = {0.1f, 0.2f};
private void readObject(ObjectInputStream in) throws Exception {
in.defaultReadObject();
System.out.println("readObject ran during load: this could have been any code");
}
}
public static void main(String[] args) throws Exception {
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
try (ObjectOutputStream out = new ObjectOutputStream(bytes)) { out.writeObject(new Checkpoint()); }
System.out.println("saved " + bytes.size() + " bytes");
try (ObjectInputStream in = new ObjectInputStream(new ByteArrayInputStream(bytes.toByteArray()))) {
Checkpoint c = (Checkpoint) in.readObject();
System.out.println("loaded weights[1] = " + c.weights[1]);
}
}
}saved 90 bytes
readObject ran during load: this could have been any code
loaded weights[1] = 0.2The first line says 90 bytes were saved. The second line comes from inside readObject, so code ran while the file was being loaded. The third line shows the weights were restored too.
ObjectInputStream.readObject does more than copy bytes into fields. When the stream names a class that has a private readObject method, the JVM calls that method during loading. Here it prints a line; in an attack, a class already on the class path does something harmful, and the stream is crafted to reach it. The data in the file decides which code runs.
Python's pickle works the same way, with fewer limits. A pickle is a small program for a stack machine (a simple machine that pushes values onto a stack and pops them to use). Its opcodes (single-byte instructions) build objects, and some opcodes import a function by name and call it. PyTorch's traditional .bin and .pt checkpoints are pickles inside a zip file. Loading one with full unpickling runs whatever the pickle says, which is why a model file from an unknown source was a security risk. The next example shows those opcodes.
Example 2: the pickle opcodes that call a function
To see why pickle is dangerous, it helps to read one. In Python, an object can define __reduce__ to tell pickle how to rebuild it. The method returns a function and its arguments, and the loader calls that function. An attacker can choose os.system (run a shell command) with the argument "echo pwned".
The program below walks through the 42 bytes that pickle.dumps(obj, protocol=2) produces for such an object and names each opcode. Look for the two lines marked with arrows: one imports a function by name, the other calls it.
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.charset.StandardCharsets;
import java.util.HexFormat;
public class PickleOpcodes {
public static void main(String[] args) {
// pickle.dumps(obj, protocol=2) for an object whose __reduce__ returns (os.system, ("echo pwned",))
byte[] p = HexFormat.of().parseHex("800263706f7369780a73797374656d0a7100580a0000006563686f2070776e656471018571025271032e");
ByteBuffer b = ByteBuffer.wrap(p).order(ByteOrder.LITTLE_ENDIAN);
while (b.hasRemaining()) {
int pos = b.position(), op = b.get() & 0xFF;
String text = switch (op) {
case 0x80 -> "PROTO " + b.get();
case 'c' -> "GLOBAL " + line(b) + " " + line(b) + " <- imports a callable by name";
case 'q' -> "BINPUT " + b.get();
case 'X' -> { byte[] s = new byte[b.getInt()]; b.get(s); yield "BINUNICODE '" + new String(s, StandardCharsets.UTF_8) + "'"; }
case 0x85 -> "TUPLE1";
case 'R' -> "REDUCE <- calls the callable with the tuple";
case '.' -> "STOP";
default -> "opcode 0x" + Integer.toHexString(op);
};
System.out.printf("%2d: %s%n", pos, text);
}
}
static String line(ByteBuffer b) {
StringBuilder s = new StringBuilder();
for (byte c; (c = b.get()) != '\n'; ) s.append((char) c);
return s.toString();
}
} 0: PROTO 2
2: GLOBAL posix system <- imports a callable by name
16: BINPUT 0
18: BINUNICODE 'echo pwned'
33: BINPUT 1
35: TUPLE1
36: BINPUT 2
38: REDUCE <- calls the callable with the tuple
39: BINPUT 3
41: STOPEach output line shows the byte position and the opcode found there. BINPUT stores a value in a memo table and has no effect on the attack, so it can be ignored. The important opcodes are listed next.
Read top to bottom, the opcodes say:
GLOBAL posix system: import the moduleposix(whatosresolves to on Linux) and push its attributesystem.BINUNICODE 'echo pwned'andTUPLE1: push the string and wrap it in a 1-element tuple.REDUCE: pop the tuple and the callable and callsystem("echo pwned").
Unpickling this file runs a shell command. Any object can ask to be saved this way through __reduce__, and the loader has no way to tell harmless calls (rebuilding a tensor) from harmful ones without restricting which globals it allows. That restriction is what torch.load(weights_only=True) and pickle scanners do: they reject GLOBAL names outside an allow-list (a list of the only names that are permitted). safetensors avoids the problem entirely, because its reader parses JSON and copies bytes, and nothing in the file is ever executed.
Example 3: the layout: header size, JSON header, data
Before reading any code that parses the format, it helps to see the bytes. A safetensors file has three parts: 8 bytes that say how long the header is, the header itself as JSON text, and then the raw numbers. Putting the description first means a reader learns where every tensor is before it touches any numbers.
The program below builds the smallest useful file in memory, one tensor named x with two float values, and prints a hex dump. A hex dump shows each byte as two hexadecimal digits (00 to ff), with the same bytes as text on the right. Match the three parts to the dump as you read the output.
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.charset.StandardCharsets;
public class LayoutBytes {
public static void main(String[] args) {
String header = "{\"x\":{\"dtype\":\"F32\",\"shape\":[2],\"data_offsets\":[0,8]}}";
byte[] json = header.getBytes(StandardCharsets.UTF_8);
int padded = (json.length + 7) / 8 * 8; // pad with spaces to a multiple of 8
ByteBuffer file = ByteBuffer.allocate(8 + padded + 8).order(ByteOrder.LITTLE_ENDIAN);
file.putLong(padded).put(json);
for (int i = json.length; i < padded; i++) file.put((byte) ' ');
file.putFloat(1.5f).putFloat(-2.0f);
byte[] bytes = file.array();
for (int row = 0; row < bytes.length; row += 16) {
StringBuilder hex = new StringBuilder(), ascii = new StringBuilder();
for (int i = row; i < Math.min(row + 16, bytes.length); i++) {
hex.append(String.format("%02x ", bytes[i]));
ascii.append(bytes[i] >= 32 && bytes[i] < 127 ? (char) bytes[i] : '.');
}
System.out.printf("%04x %-48s %s%n", row, hex, ascii);
}
System.out.println("header length " + json.length + " padded to " + padded + ", file " + bytes.length + " bytes");
}
}0000 38 00 00 00 00 00 00 00 7b 22 78 22 3a 7b 22 64 8.......{"x":{"d
0010 74 79 70 65 22 3a 22 46 33 32 22 2c 22 73 68 61 type":"F32","sha
0020 70 65 22 3a 5b 32 5d 2c 22 64 61 74 61 5f 6f 66 pe":[2],"data_of
0030 66 73 65 74 73 22 3a 5b 30 2c 38 5d 7d 7d 20 20 fsets":[0,8]}}
0040 00 00 c0 3f 00 00 00 c0 ...?....
header length 54 padded to 56, file 72 bytesThe last line gives the sizes. The JSON is 54 bytes, it is padded to 56, and the whole file is 56 + 8 + 8 = 72 bytes.
The dump shows the three parts:
- Bytes
0000to0007:38 00 00 00 00 00 00 00, the number 56 as a little-endian 64-bit integer. Little-endian means the least significant byte comes first, so 56 (0x38) is the first byte. - Bytes
0008to003f: the JSON header, 54 bytes of text followed by 2 spaces (20 20), 56 bytes in total. The specification allows trailing spaces, and the reference writer pads the header to a multiple of 8 so the data that follows starts at an 8-byte boundary. - Bytes
0040to0047: the data.00 00 c0 3fis thefloat1.5 and00 00 00 c0is −2.0, both little-endian.
data_offsets: [0, 8] is relative to the start of the data section, not the file. The tensor therefore starts at file byte 8 + N + 0 = 8 + 56 + 0 = 64, which is 0040 in hex. The whole file is 72 bytes, and its JSON can be read with any text viewer.
Example 4: writing a one-tensor file in Java
Gluing JSON together as strings is error-prone, so a real writer uses a JSON library. Jackson is a Java library that builds JSON as a tree of nodes: an ObjectNode is a JSON object, and putArray adds a list.
The program below writes a file with one tensor, embed.weight, a 2 by 3 matrix of float values (2 rows, 3 columns). It builds the header with Jackson, pads it, writes the values, and then reads the file back to print the header with spaces shown as underscores, so the padding is visible.
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.file.Files;
import java.nio.file.Path;
public class WriteOneTensor {
static void write(Path path, String name, int[] shape, float[] values) throws IOException {
ObjectMapper json = new ObjectMapper();
ObjectNode header = json.createObjectNode();
ObjectNode info = header.putObject(name);
info.put("dtype", "F32");
var dims = info.putArray("shape");
for (int d : shape) dims.add(d);
info.putArray("data_offsets").add(0).add(values.length * 4L);
byte[] h = json.writeValueAsBytes(header);
int padded = (h.length + 7) / 8 * 8;
ByteBuffer buf = ByteBuffer.allocate(8 + padded + values.length * 4).order(ByteOrder.LITTLE_ENDIAN);
buf.putLong(padded).put(h);
while (buf.position() < 8 + padded) buf.put((byte) ' ');
for (float v : values) buf.putFloat(v);
Files.write(path, buf.array());
}
public static void main(String[] args) throws Exception {
Path path = Files.createTempFile("one", ".safetensors");
write(path, "embed.weight", new int[] {2, 3}, new float[] {0.5f, -1f, 2f, 0f, 0.25f, 3f});
byte[] all = Files.readAllBytes(path);
long n = ByteBuffer.wrap(all, 0, 8).order(ByteOrder.LITTLE_ENDIAN).getLong();
System.out.println("file size " + all.length + ", header size " + n);
System.out.println(new String(all, 8, (int) n).replace(' ', '_'));
}
}file size 104, header size 72
{"embed.weight":{"dtype":"F32","shape":[2,3],"data_offsets":[0,24]}}____The output says the file is 104 bytes and the header is 72 bytes. The underscores at the end of the header line are the padding spaces.
The writer follows the rules from Example 3. Jackson keeps the insertion order of fields in an ObjectNode, so the output is the same on every run. The header is 68 bytes and is padded to 72 with 4 spaces; the file is 8 + 72 + 24 = 104 bytes.
Each tensor entry needs exactly three fields (the names are fixed by the format): dtype as a string ("F32"), shape as an array of integers ([2, 3]), and data_offsets as [begin, end) in bytes. The values are written row by row: [0.5, -1, 2] is row 0 and [0, 0.25, 3] is row 1. Row-major (or C order) means the last index changes fastest, so element [r][c] of a matrix with cols columns is value number r * cols + c.
Example 5: reading the header length and the JSON
A reader meets a file without knowing how long its header is. The first 8 bytes answer that, so reading takes two steps: read the length, then read that many bytes. The header may also contain a special entry called __metadata__, a small table of text notes about the file, such as which library wrote it.
The program below writes a file with a __metadata__ entry, then reads it back with a FileChannel (Java's random-access file reader) in two positioned reads. Look at the three numbers in the first output line and how they relate.
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.channels.FileChannel;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
public class ReadHeader {
public static void main(String[] args) throws Exception {
Path path = Files.createTempFile("model", ".safetensors");
String json = "{\"__metadata__\":{\"format\":\"pt\"},\"w\":{\"dtype\":\"F32\",\"shape\":[2,2],\"data_offsets\":[0,16]}}";
int padded = (json.length() + 7) / 8 * 8;
ByteBuffer file = ByteBuffer.allocate(8 + padded + 16).order(ByteOrder.LITTLE_ENDIAN);
file.putLong(padded).put(json.getBytes(StandardCharsets.UTF_8));
while (file.position() < 8 + padded) file.put((byte) ' ');
file.putFloat(1).putFloat(2).putFloat(3).putFloat(4);
Files.write(path, file.array());
try (FileChannel ch = FileChannel.open(path)) {
ByteBuffer len = ByteBuffer.allocate(8).order(ByteOrder.LITTLE_ENDIAN);
ch.read(len, 0);
long n = len.flip().getLong();
ByteBuffer header = ByteBuffer.allocate((int) n);
ch.read(header, 8);
String text = new String(header.array(), StandardCharsets.UTF_8);
System.out.println("N = " + n + ", file size " + ch.size() + ", data starts at byte " + (8 + n));
System.out.println("first char '" + text.charAt(0) + "', trailing spaces " + (text.length() - text.stripTrailing().length()));
System.out.println(text.stripTrailing());
}
}
}N = 88, file size 112, data starts at byte 96
first char '{', trailing spaces 0
{"__metadata__":{"format":"pt"},"w":{"dtype":"F32","shape":[2,2],"data_offsets":[0,16]}}In the output, N = 88 is the header length. The data starts at byte 8 + 88 = 96, and the file is 112 bytes, so 112 - 96 = 16 bytes of data remain. The second line shows that the header starts with { and has no trailing spaces. The third line is the header JSON.
Reading takes two steps. First read 8 bytes at position 0 and decode them as a little-endian long: here N = 88. Then read N bytes at position 8. A FileChannel read with an explicit position does not move a shared file pointer, so the same channel can serve several readers.
Three numbers come out of the first read: the header length (88), the start of the data (8 + 88 = 96), and, together with the file size (112), the length of the data section (16). Here the header text happened to be exactly 88 bytes, a multiple of 8, so there are no trailing spaces. A reader must still accept spaces at the end, which is why the program calls stripTrailing() before printing.
A reader should check N against the file size before allocating N bytes. A corrupted or malicious file can claim any N; Example 11 shows the limits the reference implementation enforces.
Example 6: parsing dtype, shape, data_offsets, and __metadata__
Once the header text is read, the program needs a usable list of tensors, each with a name, a type, a shape, and a byte range. A shape is the list of sizes along each dimension: [896, 896] is a square matrix, and [896] is a plain list of 896 numbers.
The program below parses a header with Jackson. It prints the __metadata__ entries separately, collects the tensors, and sorts them by where their data begins. Notice that the two tensors appear in the JSON in the opposite order to their data.
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
public class ParseHeader {
record TensorInfo(String name, String dtype, long[] shape, long begin, long end) {}
public static void main(String[] args) throws Exception {
String header = """
{"__metadata__": {"format": "pt", "producer": "example"},
"model.norm.weight": {"dtype": "BF16", "shape": [896], "data_offsets": [1605632, 1607424]},
"model.embed_tokens.weight": {"dtype": "BF16", "shape": [896, 896], "data_offsets": [0, 1605632]}}
""";
JsonNode root = new ObjectMapper().readTree(header);
List<TensorInfo> tensors = new ArrayList<>();
for (Map.Entry<String, JsonNode> e : root.properties()) {
if (e.getKey().equals("__metadata__")) {
e.getValue().properties().forEach(m -> System.out.println("metadata " + m.getKey() + " = " + m.getValue().asText()));
continue;
}
JsonNode v = e.getValue();
long[] shape = new long[v.get("shape").size()];
for (int i = 0; i < shape.length; i++) shape[i] = v.get("shape").get(i).asLong();
tensors.add(new TensorInfo(e.getKey(), v.get("dtype").asText(), shape,
v.get("data_offsets").get(0).asLong(), v.get("data_offsets").get(1).asLong()));
}
tensors.sort(Comparator.comparingLong(TensorInfo::begin));
for (TensorInfo t : tensors)
System.out.printf("%-26s %-5s %-11s bytes [%d, %d) = %d%n", t.name(), t.dtype(),
java.util.Arrays.toString(t.shape()), t.begin(), t.end(), t.end() - t.begin());
}
}metadata format = pt
metadata producer = example
model.embed_tokens.weight BF16 [896, 896] bytes [0, 1605632) = 1605632
model.norm.weight BF16 [896] bytes [1605632, 1607424) = 1792The output starts with the two metadata lines. Then come the tensors in data order: the embedding occupies bytes 0 to 1,605,632 and the norm vector follows it. In the brackets, [a, b) means "from byte a up to but not including byte b", so the size is b - a.
The header is a JSON object whose keys are tensor names, plus at most one special key, __metadata__. The specification allows only a string-to-string map there; Transformers writes {"format": "pt"}, and tools add their own entries.
JSON objects have no defined order, and in this header the norm weight appears before the embedding although its data comes after. The reference reader therefore sorts tensors by data_offsets before it validates them (tensors.sort_by_key(|(_, left)| left.data_offsets) in tensor.rs, with a comment that previous versions used a different order). The program does the same with Comparator.comparingLong(TensorInfo::begin).
The offsets also give each tensor's byte size, and the size matches the shape. The BF16 embedding has 896 * 896 values at 2 bytes each, and 896 * 896 * 2 = 1605632. The norm vector has 896 values, and 896 * 2 = 1792. JsonNode.properties() iterates the object's fields as map entries.
Example 7: reading a tensor's values
A reader that wants one tensor should jump straight to its bytes. For that it needs the absolute position of the tensor in the file, and a rule for finding element [r][c] inside it.
The program below builds a file with a 3-value bias and a 2 by 3 weight, then reads the weight by computing its absolute position from the header. Look at the starting byte and at how the two printed rows match the numbers that were written.
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.charset.StandardCharsets;
public class ReadValues {
public static void main(String[] args) throws Exception {
String json = "{\"bias\":{\"dtype\":\"F32\",\"shape\":[3],\"data_offsets\":[0,12]},"
+ "\"weight\":{\"dtype\":\"F32\",\"shape\":[2,3],\"data_offsets\":[12,36]}}";
int padded = (json.length() + 7) / 8 * 8;
ByteBuffer file = ByteBuffer.allocate(8 + padded + 36).order(ByteOrder.LITTLE_ENDIAN);
file.putLong(padded).put(json.getBytes(StandardCharsets.UTF_8));
while (file.position() < 8 + padded) file.put((byte) ' ');
for (float v : new float[] {0.1f, 0.2f, 0.3f, 1, 2, 3, 4, 5, 6}) file.putFloat(v);
byte[] bytes = file.array();
// Reader: header, then the tensor's absolute position = 8 + N + begin.
ByteBuffer in = ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN);
long n = in.getLong(0);
JsonNode w = new ObjectMapper().readTree(new String(bytes, 8, (int) n, StandardCharsets.UTF_8)).get("weight");
long begin = w.get("data_offsets").get(0).asLong();
int rows = w.get("shape").get(0).asInt(), cols = w.get("shape").get(1).asInt();
long base = 8 + n + begin;
System.out.println("weight starts at file byte " + base);
for (int r = 0; r < rows; r++) {
StringBuilder row = new StringBuilder();
for (int c = 0; c < cols; c++) row.append(in.getFloat((int) (base + 4L * (r * cols + c)))).append(' '); // row-major
System.out.println("row " + r + ": " + row.toString().trim());
}
}
}weight starts at file byte 140
row 0: 1.0 2.0 3.0
row 1: 4.0 5.0 6.0The output says the weight starts at file byte 140 and prints two rows. The numbers 1 to 6 were written in order after the three bias values, and the reader recovers them as a 2 by 3 grid.
The absolute position of a tensor is 8 + N + begin. This header is 120 bytes of JSON, already a multiple of 8, so N = 120, and the weight begins after the bias's 12 bytes, so it starts at 8 + 120 + 12 = 140. The program reads N from the file instead of computing it from the JSON length, because padding (and any other writer's formatting) changes the real value.
ByteBuffer.getFloat(index) with ByteOrder.LITTLE_ENDIAN decodes 4 bytes at an absolute index. Element [r][c] is at base + 4 * (r * cols + c) because the data is row-major; the output shows rows 1 2 3 and 4 5 6 in the order they were written.
Example 8: several tensors: contiguous offsets and header padding
A real model has hundreds of tensors, so a writer must decide their order and give each one its own byte range. The reference writer in the safetensors crate has a rule for this order, and the rule has a practical purpose, explained after the output.
The program below sorts four tensors of different types with that rule, assigns offsets one after another, and pads the header. Look at the order of the output lines, then at how each offset equals the end of the previous line.
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
public class SeveralTensors {
// Dtype enum order in the safetensors crate, from smallest alignment to largest.
static final List<String> ORDER = List.of("BOOL", "F4", "F6_E2M3", "F6_E3M2", "U8", "I8", "F8_E5M2", "F8_E4M3", "F8_E8M0",
"F8_E4M3FNUZ", "F8_E5M2FNUZ", "I16", "U16", "F16", "BF16", "I32", "U32", "F32", "C64", "F64", "I64", "U64");
record T(String name, String dtype, long[] shape, int bytesPerValue) {
long size() { long n = 1; for (long d : shape) n *= d; return n * bytesPerValue; }
}
public static void main(String[] args) throws Exception {
List<T> tensors = new ArrayList<>(List.of(
new T("norm.weight", "F32", new long[] {4}, 4),
new T("lm_head.weight", "BF16", new long[] {2, 3}, 2),
new T("embed.weight", "BF16", new long[] {3, 2}, 2),
new T("step", "I64", new long[] {}, 8)));
// serialize(): descending dtype order, then by name
tensors.sort(Comparator.comparingInt((T t) -> -ORDER.indexOf(t.dtype())).thenComparing(T::name));
ObjectMapper json = new ObjectMapper();
ObjectNode header = json.createObjectNode();
long offset = 0;
for (T t : tensors) {
ObjectNode info = header.putObject(t.name());
info.put("dtype", t.dtype());
var shape = info.putArray("shape");
for (long d : t.shape()) shape.add(d);
info.putArray("data_offsets").add(offset).add(offset + t.size());
System.out.printf("%-15s %-4s offset %3d .. %3d%n", t.name(), t.dtype(), offset, offset + t.size());
offset += t.size();
}
int len = json.writeValueAsBytes(header).length;
System.out.println("data bytes " + offset + ", header " + len + " bytes, padded to " + (len + 7) / 8 * 8);
}
}step I64 offset 0 .. 8
norm.weight F32 offset 8 .. 24
embed.weight BF16 offset 24 .. 36
lm_head.weight BF16 offset 36 .. 48
data bytes 48, header 260 bytes, padded to 264The output lists step first, then norm.weight, then the two BF16 matrices in name order. The ranges are 0 to 8, 8 to 24, 24 to 36, and 36 to 48, so the data is 48 bytes. The header is 260 bytes and is padded to 264.
The reference writer (prepare in tensor.rs) sorts tensors by dtype in descending order of the Dtype enum, then by name. The enum is declared "in increasing alignment order" (from BOOL and F4 up to I64 and U64), so the largest types come first: the 8-byte I64 scalar, then the F32 vector, then the two BF16 matrices in name order. Because every earlier tensor has a size that is a multiple of the next one's element size, every tensor starts at an offset aligned to its own element size. Alignment means a value of size k starts at an address that is a multiple of k. Processors read aligned values faster, and some require it. A reader that maps the file (Example 10) can then view the data as long, float, or short arrays without copying, provided the data section itself is aligned, which the 8-byte header padding ensures.
The offsets are contiguous: each tensor begins exactly where the previous one ended (0 to 8, 8 to 24, 24 to 36, 36 to 48). step has shape [], a 0-rank tensor (a scalar), which the specification allows; its size is one element. The header JSON is 260 bytes and is padded to 264.
Example 9: the dtype table and decoding BF16
The bytes in the data section are only numbers once you know their dtype (data type). The dtype tells the reader how many bits each value uses and how to interpret them. A bit is one binary digit, and 8 bits make a byte.
The program below prints the bit widths of common dtypes. It then decodes four BF16 values and shows one 16-bit pattern read both as F16 and as BF16. BF16 (brain floating point, 16 bits) is a number format that keeps the range of a 32-bit float but fewer digits of precision. Number Formats explains the formats in detail.
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.util.Locale;
public class DtypeDecode {
static float bf16(short bits) { return Float.intBitsToFloat((bits & 0xFFFF) << 16); }
public static void main(String[] args) {
String[][] table = {{"F64", "64"}, {"F32", "32"}, {"F16", "16"}, {"BF16", "16"}, {"F8_E4M3", "8"}, {"F8_E5M2", "8"},
{"I64", "64"}, {"I32", "32"}, {"I16", "16"}, {"I8", "8"}, {"U8", "8"}, {"BOOL", "8"}, {"F4", "4"}};
StringBuilder row = new StringBuilder();
for (String[] t : table) row.append(t[0]).append('=').append(t[1]).append(' ');
System.out.println("bits per value: " + row.toString().trim());
// Four BF16 values as stored in a file: little-endian 16-bit words.
byte[] stored = {(byte) 0x80, 0x3F, 0x49, 0x40, (byte) 0xC0, (byte) 0xBF, 0x00, 0x00};
ByteBuffer b = ByteBuffer.wrap(stored).order(ByteOrder.LITTLE_ENDIAN);
StringBuilder out = new StringBuilder();
while (b.hasRemaining()) {
short bits = b.getShort();
out.append(String.format(Locale.ROOT, "0x%04x=%s ", bits & 0xFFFF, bf16(bits)));
}
System.out.println("BF16: " + out.toString().trim());
short f16 = (short) 0x4248;
System.out.println("F16 0x4248 = " + Float.float16ToFloat(f16) + ", same bits as BF16 = " + bf16(f16));
}
}bits per value: F64=64 F32=32 F16=16 BF16=16 F8_E4M3=8 F8_E5M2=8 I64=64 I32=32 I16=16 I8=8 U8=8 BOOL=8 F4=4
BF16: 0x3f80=1.0 0x4049=3.140625 0xbfc0=-1.5 0x0000=0.0
F16 0x4248 = 3.140625, same bits as BF16 = 50.0The first output line lists the widths, from 64 bits down to 4 bits. The second line shows the four BF16 words decoded. The third line shows one bit pattern, 0x4248, giving two different numbers depending on the format.
The dtype names are the ones the Dtype enum serializes to, with bitsize() giving the widths. The full list at version 0.8.0 is BOOL, F4, F6_E2M3, F6_E3M2, U8, I8, F8_E5M2, F8_E4M3, F8_E8M0, F8_E4M3FNUZ, F8_E5M2FNUZ, I16, U16, F16, BF16, I32, U32, F32, C64, F64, I64, and U64. F4 and the F6 types are smaller than a byte, which is why the reader checks that a tensor's total bits are a multiple of 8 (Example 11).
BF16 (Number Formats) is the top 16 bits of an FP32 value, so decoding is a shift: Float.intBitsToFloat(bits << 16). Put the 16 stored bits in the upper half of a 32-bit pattern, fill the lower half with zeros, and read the result as a normal float. The bytes 80 3f read little-endian are 0x3f80, which is 1.0; 0x4049 is 3.140625, the nearest BF16 value to π; 0xbfc0 is −1.5.
The last line shows why the dtype field matters. The same 16 bits 0x4248 mean 3.140625 as F16 and 50.0 as BF16. A reader that ignores dtype produces numbers that look plausible and are completely wrong.
Example 10: memory-mapping the file with FileChannel.map
Large models are many gigabytes. They do not fit in a Java byte[], and reading a whole file to use one tensor is slow. Memory mapping solves this: the operating system makes the file look like a region of memory, and loads a piece from disk only when the program reads it.
The program below writes a 1 MiB tensor (1024 rows of 256 float values), maps the file, and reads one row through a slice (a view of part of a buffer). Look at the byte where row 700 starts and at the values printed for it.
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
public class MapFile {
public static void main(String[] args) throws Exception {
int rows = 1024, cols = 256;
String json = "{\"big\":{\"dtype\":\"F32\",\"shape\":[" + rows + "," + cols + "],\"data_offsets\":[0," + rows * cols * 4 + "]}}";
int padded = (json.length() + 7) / 8 * 8;
ByteBuffer file = ByteBuffer.allocate(8 + padded + rows * cols * 4).order(ByteOrder.LITTLE_ENDIAN);
file.putLong(padded).put(json.getBytes(StandardCharsets.UTF_8));
while (file.position() < 8 + padded) file.put((byte) ' ');
for (int i = 0; i < rows * cols; i++) file.putFloat(i * 0.001f);
Path path = Files.createTempFile("big", ".safetensors");
Files.write(path, file.array());
try (FileChannel ch = FileChannel.open(path)) {
MappedByteBuffer map = ch.map(FileChannel.MapMode.READ_ONLY, 0, ch.size());
map.order(ByteOrder.LITTLE_ENDIAN);
long n = map.getLong(0);
long dataStart = 8 + n;
int row = 700; // read one row without reading the rest
ByteBuffer slice = map.slice((int) (dataStart + (long) row * cols * 4), cols * 4).order(ByteOrder.LITTLE_ENDIAN);
System.out.println("mapped " + map.capacity() + " bytes, row " + row + " starts at file byte " + (dataStart + (long) row * cols * 4));
System.out.printf("row %d: %.3f %.3f ... %.3f%n", row, slice.getFloat(0), slice.getFloat(4), slice.getFloat((cols - 1) * 4));
}
}
}mapped 1048656 bytes, row 700 starts at file byte 716880
row 700: 179.200 179.201 ... 179.455The output says 1,048,656 bytes were mapped: 1,048,576 of data plus 80 for the length and the header. Row 700 begins at byte 716,880 and holds the numbers 179.200 to 179.455, which are 0.001 times the positions 179,200 to 179,455.
FileChannel.map(READ_ONLY, 0, size) returns a MappedByteBuffer: the operating system maps the file's pages into the process's address space, and a page is read from disk only when the program touches it. Row 700 starts at 8 + N + 700 * 256 * 4 = 716880; slice(index, length) (Java 13 and later) creates a view of only those 1,024 bytes, and reading it touches one page of the file. The values 179.200 ... 179.455 are 0.001 * (700 * 256 + c).
This is the access pattern safetensors was designed for, and the reason its data is not compressed or encrypted: an engine can map the file and hand tensor regions to the GPU without a separate copy in Java or Python memory. A MappedByteBuffer is limited to 2 GiB per mapping because its indexes are int. Section 7 maps multi-gigabyte model files with the Foreign Function and Memory API (MemorySegment), which has long offsets.
Example 11: validation rules from the specification
A file downloaded from the internet cannot be trusted. A damaged or hostile file might claim a huge header, offsets outside the file, or tensors that overlap. A reader must check the header before it uses any offset.
The program below implements the checks of the reference implementation and runs nine small files through them. One file is valid and each of the others breaks exactly one rule. Read each output line as the name of the broken rule and the message the validator returns.
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
public class Validate {
static final Map<String, Integer> BITS = Map.of("F32", 32, "F16", 16, "BF16", 16, "I64", 64, "U8", 8, "F4", 4);
static final ObjectMapper JSON = new ObjectMapper().enable(JsonParser.Feature.STRICT_DUPLICATE_DETECTION);
static String validate(byte[] file) {
if (file.length < 8) return "header too small";
long n = ByteBuffer.wrap(file).order(ByteOrder.LITTLE_ENDIAN).getLong();
if (n > 100_000_000L) return "header too large";
if (8 + n > file.length) return "invalid header length";
String text = new String(file, 8, (int) n, StandardCharsets.UTF_8);
if (!text.startsWith("{")) return "header must start with '{'";
JsonNode root;
try { root = JSON.readTree(text); } catch (Exception e) { return "invalid JSON: " + e.getMessage().lines().findFirst().orElse(""); }
List<long[]> ranges = new ArrayList<>(); // {begin, end, expected size}
for (var e : root.properties()) {
if (e.getKey().equals("__metadata__")) {
for (var m : e.getValue().properties()) if (!m.getValue().isTextual()) return "metadata values must be strings";
continue;
}
JsonNode v = e.getValue();
long count = 1;
for (JsonNode d : v.get("shape")) count *= d.asLong();
long bits = count * BITS.get(v.get("dtype").asText());
if (bits % 8 != 0) return "misaligned slice: " + e.getKey();
ranges.add(new long[] {v.get("data_offsets").get(0).asLong(), v.get("data_offsets").get(1).asLong(), bits / 8});
}
ranges.sort((a, b) -> Long.compare(a[0], b[0]));
long start = 0;
for (long[] r : ranges) {
if (r[0] != start || r[1] < r[0]) return "invalid offset (gap or overlap) at " + r[0];
if (r[1] - r[0] != r[2]) return "size does not match dtype and shape";
start = r[1];
}
if (8 + n + start != file.length) return "incomplete metadata, file not fully covered";
return "ok";
}
static byte[] file(String header, int dataBytes) {
int padded = (header.length() + 7) / 8 * 8;
ByteBuffer b = ByteBuffer.allocate(8 + padded + dataBytes).order(ByteOrder.LITTLE_ENDIAN);
b.putLong(padded).put(header.getBytes(StandardCharsets.UTF_8));
while (b.position() < 8 + padded) b.put((byte) ' ');
return b.array();
}
public static void main(String[] args) {
String a = "\"a\":{\"dtype\":\"F32\",\"shape\":[2],\"data_offsets\":[0,8]}";
System.out.println("valid: " + validate(file("{" + a + "}", 8)));
System.out.println("extra bytes: " + validate(file("{" + a + "}", 12)));
System.out.println("gap: " + validate(file("{\"a\":{\"dtype\":\"F32\",\"shape\":[2],\"data_offsets\":[4,12]}}", 12)));
System.out.println("overlap: " + validate(file("{" + a + ",\"b\":{\"dtype\":\"F32\",\"shape\":[1],\"data_offsets\":[4,8]}}", 8)));
System.out.println("wrong size: " + validate(file("{\"a\":{\"dtype\":\"F16\",\"shape\":[2],\"data_offsets\":[0,8]}}", 8)));
System.out.println("duplicate: " + validate(file("{" + a + "," + a + "}", 8)));
System.out.println("metadata: " + validate(file("{\"__metadata__\":{\"epoch\":3}," + a + "}", 8)));
System.out.println("F4 odd count: " + validate(file("{\"q\":{\"dtype\":\"F4\",\"shape\":[3],\"data_offsets\":[0,2]}}", 2)));
byte[] huge = file("{" + a + "}", 8);
ByteBuffer.wrap(huge).order(ByteOrder.LITTLE_ENDIAN).putLong(0, 200_000_000L);
System.out.println("huge header: " + validate(huge));
}
}valid: ok
extra bytes: incomplete metadata, file not fully covered
gap: invalid offset (gap or overlap) at 4
overlap: invalid offset (gap or overlap) at 4
wrong size: size does not match dtype and shape
duplicate: invalid JSON: Duplicate field 'a'
metadata: metadata values must be strings
F4 odd count: misaligned slice: q
huge header: header too largeThe first file is accepted. Every other line shows one failure: extra bytes after the last tensor, a gap, an overlap, a size that does not match the dtype, a duplicate key, a non-string metadata value, a half-byte F4 tensor, and a header length of 200 million. The table lists all checks and their sources.
The checks and where they come from:
| Check | Source in tensor.rs or the README |
|---|---|
| At least 8 bytes | HeaderTooSmall |
N at most 100,000,000 |
MAX_HEADER_SIZE; the README calls it a limit "to prevent parsing extremely large JSON" |
8 + N within the file |
InvalidHeaderLength |
Header starts with { |
README: "The header data MUST begin with a { character" |
| Valid JSON | InvalidHeaderDeserialization |
| No duplicate keys | README: "Duplicate keys are disallowed. Not all parsers may respect this." tensor.rs has no separate duplicate check |
__metadata__ values are strings |
README: "Arbitrary JSON is not allowed, all values must be strings" |
| Offsets sorted start at 0 and each begins where the last ended | validate, InvalidOffset |
end - begin equals elements times dtype size |
TensorInvalidInfo |
| Total bits a multiple of 8 | MisalignedSlice |
| Last offset equals the data length | MetadataIncompleteBuffer: "the data offsets of the tensor does not fully cover the buffer part of the file" |
The last rule forbids extra bytes after the last tensor ("extra bytes" in the output) and holes between tensors. The README gives the reason: a file whose every byte belongs to exactly one tensor cannot hide a second file format inside it. A polyglot file is a file that is valid in two formats at once, which attackers use to slip content past scanners. Together with the offset checks it also guarantees that no two tensors overlap, so a reader never returns the same bytes as two tensors.
Jackson accepts duplicate keys by default and keeps the last one; enabling STRICT_DUPLICATE_DETECTION makes it reject them as the specification requires. The README also notes what is not checked: values may be NaN or infinite, and a tensor with a 0 in its shape is allowed and takes no bytes.
Example 12: sharded checkpoints and model.safetensors.index.json
Hosting services and file systems handle very large files poorly, so models above a few gigabytes are split into several files called shards. A loader then needs a lookup table: for a tensor name, which shard holds it? A small JSON file, model.safetensors.index.json, provides it.
The program below reads an excerpt of such an index, counts the tensors per shard, looks one tensor up, and prints the total size. Look at how a tensor name leads to a shard file.
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.util.Map;
import java.util.TreeMap;
public class ShardIndex {
public static void main(String[] args) throws Exception {
String index = """
{
"metadata": {"total_size": 15231233024},
"weight_map": {
"lm_head.weight": "model-00004-of-00004.safetensors",
"model.embed_tokens.weight": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.31.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
"model.norm.weight": "model-00004-of-00004.safetensors"
}
}
""";
JsonNode root = new ObjectMapper().readTree(index);
Map<String, Integer> perShard = new TreeMap<>();
for (var e : root.get("weight_map").properties()) perShard.merge(e.getValue().asText(), 1, Integer::sum);
perShard.forEach((shard, n) -> System.out.println(shard + ": " + n + " tensor(s) in this excerpt"));
System.out.println("model.norm.weight is in " + root.get("weight_map").get("model.norm.weight").asText());
System.out.printf("total_size %d bytes = %.2f GiB of tensor data%n", root.get("metadata").get("total_size").asLong(),
root.get("metadata").get("total_size").asLong() / Math.pow(2, 30));
}
}model-00001-of-00004.safetensors: 2 tensor(s) in this excerpt
model-00002-of-00004.safetensors: 1 tensor(s) in this excerpt
model-00003-of-00004.safetensors: 1 tensor(s) in this excerpt
model-00004-of-00004.safetensors: 3 tensor(s) in this excerpt
model.norm.weight is in model-00004-of-00004.safetensors
total_size 15231233024 bytes = 14.19 GiB of tensor dataThe output counts how many of the 7 excerpt tensors sit in each of the 4 shards, answers that model.norm.weight is in the fourth shard, and converts total_size to 14.19 GiB (15,231,233,024 divided by 2^30).
A sharded checkpoint has files named model-00001-of-00004.safetensors to model-00004-of-00004.safetensors and an index:
metadata.total_sizeis the total size of the tensor data in bytes, across all shards (14.19 GiB here).weight_mapmaps every tensor name to the shard file that contains it.
Each shard is a complete, valid safetensors file with its own header; only the index ties them together. The excerpt shows the usual pattern: the embedding and early layers in the first shard, the final norm and lm_head in the last, and a layer's tensors in consecutive shards. A loader reads the index, opens each shard once, and looks up tensors by name. A tool that only needs the header data (Example 13) can read the shard headers without reading the index at all, because every tensor appears in exactly one shard header. The values in this excerpt are illustrative.
Example 13: parameter count from headers alone
A parameter is one weight, one number in the model. The size in a model's name, such as "0.5B", counts them. A tensor with shape [896, 896] holds 896 * 896 = 802,816 parameters, so the shapes in the header are enough to count them without touching the data.
The program below first builds the header of a model with the Qwen2 shapes from The Hugging Face Hub (Example 2). A reader would receive this JSON from a file. It then counts parameters from the header alone. Look at the totals and at the difference when the embedding matrix is left out.
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;
import java.util.Locale;
public class CountParameters {
public static void main(String[] args) throws Exception {
// Build the header of a Qwen2-shaped model (lesson 22, Example 2); a reader sees only this JSON.
ObjectMapper json = new ObjectMapper();
ObjectNode h = json.createObjectNode();
long d = 896, ff = 4864, kv = 128, vocab = 151936, offset = 0;
java.util.List<Object[]> specs = new java.util.ArrayList<>();
specs.add(new Object[] {"model.embed_tokens.weight", new long[] {vocab, d}});
for (int i = 0; i < 24; i++) {
String p = "model.layers." + i + ".";
specs.add(new Object[] {p + "self_attn.q_proj.weight", new long[] {d, d}});
specs.add(new Object[] {p + "self_attn.q_proj.bias", new long[] {d}});
specs.add(new Object[] {p + "self_attn.k_proj.weight", new long[] {kv, d}});
specs.add(new Object[] {p + "self_attn.k_proj.bias", new long[] {kv}});
specs.add(new Object[] {p + "self_attn.v_proj.weight", new long[] {kv, d}});
specs.add(new Object[] {p + "self_attn.v_proj.bias", new long[] {kv}});
specs.add(new Object[] {p + "self_attn.o_proj.weight", new long[] {d, d}});
for (String m : new String[] {"gate_proj", "up_proj"}) specs.add(new Object[] {p + "mlp." + m + ".weight", new long[] {ff, d}});
specs.add(new Object[] {p + "mlp.down_proj.weight", new long[] {d, ff}});
specs.add(new Object[] {p + "input_layernorm.weight", new long[] {d}});
specs.add(new Object[] {p + "post_attention_layernorm.weight", new long[] {d}});
}
specs.add(new Object[] {"model.norm.weight", new long[] {d}});
for (Object[] s : specs) {
long n = 1; for (long x : (long[]) s[1]) n *= x;
ObjectNode info = h.putObject((String) s[0]);
info.put("dtype", "BF16");
var shape = info.putArray("shape"); for (long x : (long[]) s[1]) shape.add(x);
info.putArray("data_offsets").add(offset).add(offset + 2 * n);
offset += 2 * n;
}
String header = json.writeValueAsString(h);
long params = 0, embed = 0; int tensors = 0;
for (var e : json.readTree(header).properties()) {
long n = 1;
for (JsonNode x : e.getValue().get("shape")) n *= x.asLong();
params += n; tensors++;
if (e.getKey().contains("embed_tokens")) embed += n;
}
System.out.println(tensors + " tensors, header " + header.length() + " bytes");
System.out.println(String.format(Locale.ROOT, "parameters %,d (%.2fB), without embeddings %.2fB", params, params / 1e9, (params - embed) / 1e9));
System.out.println(String.format(Locale.ROOT, "data %.2f GiB in BF16", offset / Math.pow(2, 30)));
}
}290 tensors, header 32248 bytes
parameters 494,032,768 (0.49B), without embeddings 0.36B
data 0.92 GiB in BF16The output reports 290 tensors, a header of 32,248 bytes, 494,032,768 parameters, and 0.92 GiB of data at 2 bytes per BF16 value.
A tensor's parameter count is the product of its shape, and the model's count is the sum over tensors. The header of this model has 290 tensors and is 32,248 bytes, and from it alone the program finds 494,032,768 parameters, 0.49 billion, the size in the model's name. Without the embedding matrix it is 0.36 billion.
This count includes what the estimate in The Hugging Face Hub left out: the biases of the query, key, and value projections (a feature of Qwen2 models) and the norm weights. Because the embedding is tied, there is no lm_head.weight tensor; a model with untied embeddings has one more vocab x d tensor.
The data size, 0.92 GiB in BF16, is the sum of all end - begin values. Together with Example 14, this gives a way to count the parameters of any model on the Hub by downloading only a few kilobytes per shard.
Example 14: reading only the header with HTTP Range requests
A model file can be many gigabytes, but its header is a few kilobytes. If the file is on a remote server, downloading all of it only to see tensor names and sizes would waste time and bandwidth. HTTP allows a client to ask for part of a file with a Range header.
The program below serves a 1 MiB safetensors file from a small HTTP server inside the program, one that supports Range requests. The client reads the header in two requests: first the 8 length bytes, then the header. At the end, it prints how many bytes the server actually sent.
import com.fasterxml.jackson.databind.ObjectMapper;
import com.sun.net.httpserver.HttpServer;
import java.net.InetSocketAddress;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.atomic.AtomicLong;
public class RangeHeader {
public static void main(String[] args) throws Exception {
String json = "{\"w\":{\"dtype\":\"F32\",\"shape\":[512,512],\"data_offsets\":[0,1048576]}}";
int padded = (json.length() + 7) / 8 * 8;
ByteBuffer f = ByteBuffer.allocate(8 + padded + 1048576).order(ByteOrder.LITTLE_ENDIAN);
f.putLong(padded).put(json.getBytes(StandardCharsets.UTF_8));
while (f.position() < 8 + padded) f.put((byte) ' ');
byte[] file = f.array();
AtomicLong sent = new AtomicLong();
HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
server.createContext("/model.safetensors", ex -> { // serves "Range: bytes=a-b" with 206
String[] r = ex.getRequestHeaders().getFirst("Range").replace("bytes=", "").split("-");
int a = Integer.parseInt(r[0]), b = Math.min(Integer.parseInt(r[1]), file.length - 1);
ex.getResponseHeaders().add("Content-Range", "bytes " + a + "-" + b + "/" + file.length);
ex.sendResponseHeaders(206, b - a + 1);
ex.getResponseBody().write(file, a, b - a + 1);
sent.addAndGet(b - a + 1);
ex.close();
});
server.start();
HttpClient client = HttpClient.newHttpClient();
String url = "http://127.0.0.1:" + server.getAddress().getPort() + "/model.safetensors";
HttpResponse<byte[]> first = client.send(HttpRequest.newBuilder(URI.create(url)).header("Range", "bytes=0-7").build(),
HttpResponse.BodyHandlers.ofByteArray());
long n = ByteBuffer.wrap(first.body()).order(ByteOrder.LITTLE_ENDIAN).getLong();
HttpResponse<byte[]> second = client.send(HttpRequest.newBuilder(URI.create(url)).header("Range", "bytes=8-" + (8 + n - 1)).build(),
HttpResponse.BodyHandlers.ofByteArray());
var header = new ObjectMapper().readTree(new String(second.body(), StandardCharsets.UTF_8));
System.out.println(first.statusCode() + " " + first.headers().firstValue("Content-Range").orElse("") + " -> N = " + n);
System.out.println(second.statusCode() + " " + second.headers().firstValue("Content-Range").orElse("") + " -> tensors " + header.size()
+ ", w shape " + header.get("w").get("shape"));
System.out.println("transferred " + sent.get() + " of " + file.length + " bytes");
server.stop(0);
}
}206 bytes 0-7/1048656 -> N = 72
206 bytes 8-79/1048656 -> tensors 1, w shape [512,512]
transferred 80 of 1048656 bytesThe first output line shows the first request: status 206, a range of 0 to 7, and the decoded N = 72. The second shows bytes 8 to 79 and the parsed tensor. The third shows that 80 bytes were transferred out of 1,048,656.
An HTTP Range request asks for part of a resource: Range: bytes=0-7 asks for the first 8 bytes (both ends inclusive). A server that supports ranges answers 206 Partial Content with a Content-Range: bytes 0-7/1048656 header that also states the full size.
The client first fetches bytes 0 to 7 and decodes N = 72, then fetches bytes 8 to 79 and parses the header. It transferred 80 of 1,048,656 bytes. The Hub's resolve URLs and the CDNs behind them support range requests; convert_hf_to_gguf.py --remote uses this approach ("Read safetensors file remotely without downloading to disk", in its help text), and the next section's download code uses ranges to resume interrupted downloads.
A real client also handles a server that ignores Range and answers 200 with the whole body; it must then read only the first bytes and close the connection.
Example 15: Project 6: SafetensorsInfo
This project combines the lesson into a tool. SafetensorsInfo reads one or more safetensors files, checks that each header is sane and that its data ends exactly at the end of the file, and prints a tensor table, counts per dtype, and totals. It has three parts: read parses one file, report merges and prints, and write creates test files.
The demo writes a two-shard model into a temporary directory and inspects both shards. Read the output as a table with one row per tensor, followed by totals.
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.channels.FileChannel;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Comparator;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.TreeMap;
public class SafetensorsInfo {
record Tensor(String file, String name, String dtype, long[] shape, long bytes) {
long count() { long n = 1; for (long d : shape) n *= d; return n; }
}
static List<Tensor> read(Path path) throws IOException {
try (FileChannel ch = FileChannel.open(path)) {
ByteBuffer len = ByteBuffer.allocate(8).order(ByteOrder.LITTLE_ENDIAN);
ch.read(len, 0);
long n = len.flip().getLong();
if (n > 100_000_000L || 8 + n > ch.size()) throw new IOException(path + ": bad header length " + n);
ByteBuffer h = ByteBuffer.allocate((int) n);
ch.read(h, 8);
JsonNode root = new ObjectMapper().readTree(new String(h.array(), StandardCharsets.UTF_8));
List<Tensor> out = new ArrayList<>();
long end = 0;
for (var e : root.properties()) {
if (e.getKey().equals("__metadata__")) continue;
JsonNode v = e.getValue();
long[] shape = new long[v.get("shape").size()];
for (int i = 0; i < shape.length; i++) shape[i] = v.get("shape").get(i).asLong();
long b = v.get("data_offsets").get(1).asLong() - v.get("data_offsets").get(0).asLong();
end = Math.max(end, v.get("data_offsets").get(1).asLong());
out.add(new Tensor(path.getFileName().toString(), e.getKey(), v.get("dtype").asText(), shape, b));
}
if (8 + n + end != ch.size()) throw new IOException(path + ": data does not end at the end of the file");
return out;
}
}
static void report(List<Path> files) throws IOException {
List<Tensor> all = new ArrayList<>();
for (Path p : files) all.addAll(read(p));
all.sort(Comparator.comparing(Tensor::name));
Map<String, long[]> byDtype = new TreeMap<>();
long params = 0, bytes = 0;
System.out.printf("%-40s %-5s %-12s %10s %s%n", "tensor", "dtype", "shape", "params", "file");
for (Tensor t : all) {
System.out.printf("%-40s %-5s %-12s %10d %s%n", t.name(), t.dtype(), Arrays.toString(t.shape()), t.count(), t.file());
long[] c = byDtype.computeIfAbsent(t.dtype(), k -> new long[2]);
c[0]++; c[1] += t.count();
params += t.count(); bytes += t.bytes();
}
byDtype.forEach((d, c) -> System.out.println(d + ": " + c[0] + " tensors, " + c[1] + " params"));
System.out.println(String.format(Locale.ROOT, "total: %d tensors, %d params, %d bytes of data", all.size(), params, bytes));
}
static void write(Path path, Map<String, Object[]> tensors) throws IOException { // name -> {dtype, shape, bytesPerValue}
StringBuilder json = new StringBuilder("{\"__metadata__\":{\"format\":\"pt\"}");
long offset = 0;
for (var e : tensors.entrySet()) {
long[] shape = (long[]) e.getValue()[1]; long n = 1; for (long d : shape) n *= d;
long size = n * (Integer) e.getValue()[2];
json.append(",\"").append(e.getKey()).append("\":{\"dtype\":\"").append(e.getValue()[0]).append("\",\"shape\":")
.append(Arrays.toString(shape).replace(" ", "")).append(",\"data_offsets\":[").append(offset).append(',').append(offset + size).append("]}");
offset += size;
}
byte[] h = json.append('}').toString().getBytes(StandardCharsets.UTF_8);
int padded = (h.length + 7) / 8 * 8;
ByteBuffer b = ByteBuffer.allocate((int) (8 + padded + offset)).order(ByteOrder.LITTLE_ENDIAN);
b.putLong(padded).put(h);
while (b.position() < 8 + padded) b.put((byte) ' ');
Files.write(path, b.array());
}
public static void main(String[] args) throws IOException {
Path dir = Files.createTempDirectory("shards");
Map<String, Object[]> s1 = new TreeMap<>(), s2 = new TreeMap<>();
s1.put("model.embed_tokens.weight", new Object[] {"BF16", new long[] {1000, 64}, 2});
s1.put("model.layers.0.self_attn.q_proj.weight", new Object[] {"BF16", new long[] {64, 64}, 2});
s1.put("model.layers.0.input_layernorm.weight", new Object[] {"F32", new long[] {64}, 4});
s2.put("model.layers.0.mlp.down_proj.weight", new Object[] {"BF16", new long[] {64, 256}, 2});
s2.put("model.norm.weight", new Object[] {"F32", new long[] {64}, 4});
s2.put("lm_head.weight", new Object[] {"BF16", new long[] {1000, 64}, 2});
write(dir.resolve("model-00001-of-00002.safetensors"), s1);
write(dir.resolve("model-00002-of-00002.safetensors"), s2);
report(List.of(dir.resolve("model-00001-of-00002.safetensors"), dir.resolve("model-00002-of-00002.safetensors")));
}
}tensor dtype shape params file
lm_head.weight BF16 [1000, 64] 64000 model-00002-of-00002.safetensors
model.embed_tokens.weight BF16 [1000, 64] 64000 model-00001-of-00002.safetensors
model.layers.0.input_layernorm.weight F32 [64] 64 model-00001-of-00002.safetensors
model.layers.0.mlp.down_proj.weight BF16 [64, 256] 16384 model-00002-of-00002.safetensors
model.layers.0.self_attn.q_proj.weight BF16 [64, 64] 4096 model-00001-of-00002.safetensors
model.norm.weight F32 [64] 64 model-00002-of-00002.safetensors
BF16: 4 tensors, 148480 params
F32: 2 tensors, 128 params
total: 6 tensors, 148608 params, 297472 bytes of dataThe table lists the six tensors sorted by name, with the shard that holds each. Below it, the program prints a count per dtype: four BF16 tensors with 148,480 parameters and two F32 tensors with 128. The last line is the grand total.
The program combines the lesson's pieces:
readdoes the two positioned reads from Example 5, rejects header lengths over 100,000,000 or beyond the file, parses the JSON with Jackson, and checks that the largestendoffset plus8 + Nequals the file size (the "fully covered" rule from Example 11).reportmerges the tensors of all shards, sorts them by name, and prints one row per tensor with its shard, then totals per dtype and overall.writebuilds test files with aStringBuilderheader and aTreeMap, so tensor order and output are the same on every run.
The output lists 6 tensors with 148,608 parameters in 297,472 bytes: the BF16 tensors take 2 bytes per parameter and the two F32 norm vectors 4. report takes a list of paths, so pointing it at the shards of a real checkpoint gives the same table for a 7-billion-parameter model, after reading only the headers. loop show in section 4 uses this class for safetensors repositories and the GGUF reader from The GGUF Format for GGUF files.
Interview points
- Pickle checkpoints can run code on load, because the
GLOBALandREDUCEopcodes import and call arbitrary functions. safetensors contains only JSON and raw bytes, so nothing in the file is executed. - A safetensors file is an 8-byte little-endian header length
N, thenNbytes of JSON that start with{and are padded with spaces, then the data. data_offsetsare relative to the start of the data section, so a tensor's position in the file is8 + N + begin.- Data is little-endian and row-major with no strides, and tensors are packed one after another with no gaps.
- The reference writer sorts tensors by descending dtype order, then by name, and pads the header to a multiple of 8, so every tensor is aligned to its element size.
- A reader validates the file: header at most 100,000,000 bytes, valid JSON without duplicate keys, string-only
__metadata__, contiguous offsets from 0, sizes that match shape and dtype, and a last offset at the end of the file. - The "no holes" rule prevents polyglot files and overlapping tensors.
- Sharded checkpoints use
model.safetensors.index.jsonwithweight_mapandmetadata.total_size, and each shard is a complete valid file. - Parameter counts, dtypes, and sizes come from headers alone, and over HTTP two Range requests per shard are enough.
- Memory mapping gives access without copying, and a
MappedByteBufferis limited to 2 GiB per mapping.
Summary
- Java serialization and Python pickle both run code that the file chooses during loading.
torch.load(weights_only=True)restricts unpickling to allowed types.- The safetensors header length is an unsigned 64-bit little-endian integer.
- The JSON header holds
dtype,shape, anddata_offsetsfor each tensor, and an optional__metadata__map of strings. - Padding the header with spaces to a multiple of 8 aligns the data section.
- BF16 decodes with
Float.intBitsToFloat(bits << 16), and F16 decodes withFloat.float16ToFloat. - The same 16 bits mean different numbers in F16 and BF16, so a reader must always use the
dtypefield. - Readers sort tensors by offset, because JSON object order is not defined.
- 0-rank tensors (shape
[]) and empty tensors (a 0 in the shape) are allowed. - Sub-byte dtypes such as
F4require the total bit count to be a multiple of 8. FileChannel.mapandMappedByteBuffer.sliceread one tensor without reading the whole file.- The Qwen2-shaped header gives 494,032,768 parameters from 290 tensors.
- A Range request for
bytes=0-7followed by one for the header reads 80 bytes of a 1 MiB file. SafetensorsInfoprints per-tensor, per-dtype, and total counts for any set of shards.