Virtual Memory, Page Faults, and perf

This lesson explains how Linux gives a JVM its memory and what that costs at run time. After it, a reader can measure page faults and resident memory of a Java process from /proc, decide when to pre-touch the heap or use huge pages, and read perf output.

Idea	What it is
Virtual memory	Each process sees its own address space; the kernel maps it to physical memory
Page	The unit of mapping, 4 KiB on x86-64 by default
Page table	The per-process structure that maps virtual pages to physical frames
Page fault	A trap into the kernel when a virtual page has no mapping yet
`-XX:+AlwaysPreTouch`	Touch every heap page at startup so faults happen before work starts
Transparent huge pages	2 MiB pages the kernel creates without `hugetlbfs`
Swap	Disk space the kernel moves cold pages to under memory pressure
`perf`	Linux profiler that reads hardware and software counters

Examples print checks instead of sizes and counts, because those differ between machines.

Example 1: virtual size, resident size, and the page size

A virtual address space is the range of addresses a process can use. Reserving a range (mmap) costs no physical memory; the kernel assigns a physical frame only when a page is first touched. /proc/self/status shows both numbers: VmSize is the virtual size and VmRSS (resident set size) is the part backed by physical memory now.

Java

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;

public class VmSizes {
    static long kb(String field) throws IOException {
        for (String l : Files.readAllLines(Path.of("/proc/self/status")))
            if (l.startsWith(field + ":")) return Long.parseLong(l.split("\\s+")[1]);
        throw new IllegalStateException(field);
    }

    public static void main(String[] args) throws Exception {
        Process p = new ProcessBuilder("getconf", "PAGESIZE").start();
        String page = new String(p.getInputStream().readAllBytes()).trim();
        p.waitFor();
        System.out.println("page size: " + page);
        long maxHeapKb = Runtime.getRuntime().maxMemory() / 1024;
        System.out.println("VmSize > VmRSS: " + (kb("VmSize") > kb("VmRSS")));
        System.out.println("VmSize covers the reserved max heap: " + (kb("VmSize") > maxHeapKb));
    }
}

Output

page size: 4096
VmSize > VmRSS: true
VmSize covers the reserved max heap: true

The JVM reserves address space for the maximum heap at startup but commits and touches only part of it, so the virtual size is far above the resident size. A page table maps each virtual page to a physical frame; the CPU caches recent translations in the TLB (Caches and Locality covers TLB misses).

Example 2: minor page faults when memory is first touched

A page fault is a trap into the kernel when the CPU touches a virtual page without a valid mapping. A minor fault is served from memory: the kernel zeroes a free frame or maps a page already in the page cache. A major fault needs disk I/O, for example reading a file page or a swapped-out page. Fields 10 (minflt) and 12 (majflt) of /proc/self/stat count them for the whole process.

Java

import java.nio.ByteBuffer;
import java.nio.file.Files;
import java.nio.file.Path;

public class MinorFaults {
    static long[] faults() throws Exception {
        String s = Files.readString(Path.of("/proc/self/stat"));
        String[] f = s.substring(s.lastIndexOf(')') + 2).split(" ");
        return new long[] { Long.parseLong(f[7]), Long.parseLong(f[9]) }; // fields 10 and 12
    }

    public static void main(String[] args) throws Exception {
        int size = 64 * 1024 * 1024;
        ByteBuffer buf = ByteBuffer.allocateDirect(size);  // zeroed by the JDK: pages touched here
        long[] a = faults();
        for (int i = 0; i < size; i += 4096) buf.put(i, (byte) 1); // already touched: few faults
        long[] b = faults();
        System.out.println("second pass faults < 1000: " + (b[0] - a[0] < 1000));

        ByteBuffer fresh = ByteBuffer.allocateDirect(size);
        long[] c = faults();
        System.out.println("allocating 64 MiB caused >= 16384 minor faults: " + (c[0] - b[0] >= 16384));
        System.out.println("no major faults: " + (c[1] - a[1] == 0));
        System.out.println("second buffer capacity ok: " + (fresh.capacity() == size));
    }
}

Output

second pass faults < 1000: true
allocating 64 MiB caused >= 16384 minor faults: true
no major faults: true
second buffer capacity ok: true

allocateDirect zeroes the new memory, which touches each 4 KiB page once: 64 MiB is 16384 pages, so at least 16384 minor faults. The second pass over the first buffer touches pages that are already mapped and causes almost none. Each minor fault adds kernel work (finding, zeroing, and mapping a frame); a request that touches fresh memory pays that cost on its critical path.

Example 3: AlwaysPreTouch

A Java heap behaves the same way: the first write to each heap page faults. -XX:+AlwaysPreTouch makes the JVM write to every page of the committed heap at startup, so startup is slower and later allocation causes no faults. The example starts two child JVMs with -Xms256m and compares their resident size.

Java

import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;

public class PreTouch {
    public static void main(String[] args) throws Exception {
        if (args.length == 1) {   // child: print own VmRSS in KiB
            for (String l : Files.readAllLines(Path.of("/proc/self/status")))
                if (l.startsWith("VmRSS:")) System.out.println(l.split("\\s+")[1]);
            return;
        }
        long plain = child(List.of());
        long touched = child(List.of("-XX:+AlwaysPreTouch"));
        System.out.println("plain RSS < 256 MiB: " + (plain < 256 * 1024));
        System.out.println("pre-touched RSS >= 256 MiB: " + (touched >= 256 * 1024));
    }

    static long child(List<String> flags) throws Exception {
        String java = ProcessHandle.current().info().command().orElseThrow();
        var cmd = new java.util.ArrayList<>(List.of(java, "-Xms256m", "-Xmx256m", "-XX:+UseSerialGC"));
        cmd.addAll(flags);
        cmd.addAll(List.of("-cp", System.getProperty("java.class.path"), "PreTouch", "child"));
        Process p = new ProcessBuilder(cmd).redirectError(ProcessBuilder.Redirect.DISCARD).start();
        String out = new String(p.getInputStream().readAllBytes()).trim();
        p.waitFor();
        return Long.parseLong(out);
    }
}

Output

plain RSS < 256 MiB: true
pre-touched RSS >= 256 MiB: true

Pre-touching suits long-running low-latency services with -Xms equal to -Xmx, where a slower start is acceptable and faults during trading are not. G1 pre-touches with several threads; G1 Garbage Collector covers heap sizing. The flag also makes the full heap resident from the start, which matters in containers with memory limits.

Example 4: memory flag defaults

-XX:+PrintFlagsFinal prints the value of every flag after ergonomics. The three memory flags of this lesson are off by default on JDK 25.

Java

import java.util.List;
import java.util.regex.Pattern;

public class MemoryFlags {
    public static void main(String[] args) throws Exception {
        String java = ProcessHandle.current().info().command().orElseThrow();
        Process p = new ProcessBuilder(java, "-XX:+PrintFlagsFinal", "-version")
            .redirectError(ProcessBuilder.Redirect.DISCARD).start();
        List<String> lines = new String(p.getInputStream().readAllBytes()).lines().toList();
        p.waitFor();
        for (String flag : List.of("AlwaysPreTouch", "UseLargePages", "UseTransparentHugePages")) {
            Pattern re = Pattern.compile("\\s" + flag + "\\s+=\\s+(\\S+)");
            String value = lines.stream().map(re::matcher).filter(m -> m.find())
                .map(m -> m.group(1)).findFirst().orElseThrow();
            System.out.println(flag + " = " + value);
        }
    }
}

Output

AlwaysPreTouch = false
UseLargePages = false
UseTransparentHugePages = false

UseLargePages uses explicit huge pages from hugetlbfs, which an administrator reserves in advance (vm.nr_hugepages). UseTransparentHugePages asks the kernel for transparent huge pages instead, described next.

Example 5: transparent huge pages

A huge page maps 2 MiB with one page table entry instead of 512 entries for 4 KiB pages. That means 512 times fewer faults to populate memory and fewer TLB misses. Transparent huge pages (THP) are huge pages the kernel creates on its own. The mode in /sys/kernel/mm/transparent_hugepage/enabled is always, madvise (only regions that ask with madvise(MADV_HUGEPAGE)), or never. -XX:+UseTransparentHugePages makes the JVM call madvise on the heap. AnonHugePages in /proc/self/smaps_rollup shows how much memory huge pages back.

Java

import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;

public class Thp {
    public static void main(String[] args) throws Exception {
        if (args.length == 1) {   // child: print AnonHugePages in KiB
            for (String l : Files.readAllLines(Path.of("/proc/self/smaps_rollup")))
                if (l.startsWith("AnonHugePages:")) System.out.println(l.split("\\s+")[1]);
            return;
        }
        String mode = Files.readString(Path.of("/sys/kernel/mm/transparent_hugepage/enabled"));
        boolean usable = !mode.contains("[never]");
        System.out.println("THP mode is always, madvise, or never: "
            + (mode.contains("[always]") || mode.contains("[madvise]") || mode.contains("[never]")));

        long off = child("-XX:-UseTransparentHugePages");
        long on = child("-XX:+UseTransparentHugePages");
        System.out.println("huge pages with the flag (when THP is enabled): " + (!usable || on > off));
    }

    static long child(String flag) throws Exception {
        String java = ProcessHandle.current().info().command().orElseThrow();
        Process p = new ProcessBuilder(java, "-Xms256m", "-Xmx256m", "-XX:+UseSerialGC",
                "-XX:+AlwaysPreTouch", flag, "-cp", System.getProperty("java.class.path"), "Thp", "child")
            .redirectError(ProcessBuilder.Redirect.DISCARD).start();
        String out = new String(p.getInputStream().readAllBytes()).trim();
        p.waitFor();
        return Long.parseLong(out);
    }
}

Output

THP mode is always, madvise, or never: true
huge pages with the flag (when THP is enabled): true

The kernel can also collapse pages into huge pages in the background (khugepaged) and must compact memory to find free 2 MiB blocks. In always mode that work can add pauses, so latency-sensitive deployments usually set madvise and opt in per process with the JVM flag.

Example 6: mmap from the kernel side

FileChannel.map calls mmap, which adds a file-backed mapping to the address space (ByteBuffer, Direct Buffers, and Memory-Mapped Files covers the Java API). The mapping appears in /proc/self/maps with the file name. Pages are filled on first access by page faults: minor when the file page is already in the page cache, major when it must be read from disk.

Java

import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;

public class MmapView {
    public static void main(String[] args) throws Exception {
        Path f = Files.createTempFile("mapped", ".dat");
        int size = 8 * 1024 * 1024;
        try (FileChannel ch = FileChannel.open(f, StandardOpenOption.READ, StandardOpenOption.WRITE)) {
            MappedByteBuffer mb = ch.map(FileChannel.MapMode.READ_WRITE, 0, size);
            String name = f.getFileName().toString();
            boolean listed = Files.readAllLines(Path.of("/proc/self/maps")).stream()
                .anyMatch(l -> l.endsWith(name) && l.split("\\s+")[1].startsWith("rw-s"));
            System.out.println("mapping listed as rw-s (shared): " + listed);
            long sum = 0;
            for (int i = 0; i < size; i += 4096) { mb.put(i, (byte) 2); sum += mb.get(i); }
            System.out.println("pages written: " + sum / 2);
        }
        System.out.println("file length: " + Files.size(f));
        Files.delete(f);
    }
}

Output

mapping listed as rw-s (shared): true
pages written: 2048
file length: 8388608

The s in rw-s marks a shared mapping: writes go to the page cache and reach the file. A private mapping (MapMode.PRIVATE, shown as rw-p) uses copy-on-write: the first write to a page copies it, and the file stays unchanged. Anonymous memory such as the Java heap is also an mmap, with no file behind it.

Example 7: swapping and a /proc memory report

Swapping moves anonymous pages to a swap device when memory is short. A later access to a swapped-out page is a major fault with disk latency, which for a JVM shows up during GC, since a collection touches many heap pages. VmSwap in /proc/self/status shows how much of the process is swapped out, and /proc/sys/vm/swappiness (0 to 200) sets how readily the kernel swaps anonymous memory instead of dropping page cache. The example combines the earlier readers into one report that a service can log.

Java

import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Map;
import java.util.TreeMap;

public class MemReport {
    static Map<String, Long> report() throws Exception {
        Map<String, Long> m = new TreeMap<>();
        for (String l : Files.readAllLines(Path.of("/proc/self/status"))) {
            String[] p = l.split(":\\s+");
            if (p[0].equals("VmRSS") || p[0].equals("VmSwap") || p[0].equals("VmSize"))
                m.put(p[0] + "_kB", Long.parseLong(p[1].split(" ")[0]));
        }
        String s = Files.readString(Path.of("/proc/self/stat"));
        String[] f = s.substring(s.lastIndexOf(')') + 2).split(" ");
        m.put("minflt", Long.parseLong(f[7]));
        m.put("majflt", Long.parseLong(f[9]));
        return m;
    }

    public static void main(String[] args) throws Exception {
        Map<String, Long> r = report();
        System.out.println("fields: " + r.keySet());
        System.out.println("all non-negative: " + r.values().stream().allMatch(v -> v >= 0));
        int swappiness = Integer.parseInt(Files.readString(Path.of("/proc/sys/vm/swappiness")).trim());
        System.out.println("swappiness in 0..200: " + (swappiness >= 0 && swappiness <= 200));
    }
}

Output

fields: [VmRSS_kB, VmSize_kB, VmSwap_kB, majflt, minflt]
all non-negative: true
swappiness in 0..200: true

Low-latency hosts often run without swap or with swappiness near 0, and some lock memory with mlockall from native code. A rising majflt or non-zero VmSwap on a JVM explains long GC pauses that GC logs alone do not.

perf stat and perf record

perf is the Linux profiler. It is not installed on the machine that checks these examples, so the output below is representative. perf stat counts events for a whole run:

Text

$ perf stat -e task-clock,context-switches,page-faults,cycles,instructions java -Xms256m MinorFaults

 Performance counter stats for 'java -Xms256m MinorFaults':

            412.31 msec task-clock                #    1.734 CPUs utilized
               957      context-switches          #    2.321 K/sec
            39,812      page-faults               #   96.557 K/sec
     1,262,447,903      cycles                    #    3.062 GHz
     1,904,112,563      instructions              #    1.51  insn per cycle

perf record -g samples call stacks, and perf report shows where time goes, including kernel functions such as page fault handling:

Text

$ perf record -g -p <pid> -- sleep 10 && perf report --stdio
    18.40%  java  [kernel.kallsyms]  [k] clear_page_erms
     9.12%  java  [kernel.kallsyms]  [k] handle_mm_fault
     6.75%  java  perf-12345.map     [.] Interpreted ...

JIT-compiled Java frames appear as unknown addresses unless the JVM writes a symbol map (for example with -XX:+UnlockDiagnosticVMOptions -XX:+DumpPerfMapAtExit, or a tool such as async-profiler). JFR, async-profiler, and Flame Graphs covers Java-level profiling.

Interview points

Reserving virtual memory is free; physical memory is assigned on first touch through a page fault.
Minor faults are served from memory; major faults need disk I/O (file pages, swap).
-XX:+AlwaysPreTouch moves heap faults to startup and makes the full heap resident at once.
Huge pages cut faults and TLB misses by 512 times per 2 MiB; THP in always mode can add compaction and khugepaged pauses, so madvise plus -XX:+UseTransparentHugePages is the common setting.
UseLargePages uses pre-reserved hugetlbfs pages; UseTransparentHugePages uses THP.
mmap mappings show in /proc/<pid>/maps; s is shared, p is private copy-on-write.
Swapped heap pages turn GC into disk I/O; check VmSwap and majflt.
perf stat counts events; perf record samples stacks; JIT frames need a perf map.

Summary

Pages are 4 KiB by default; page tables map them; the TLB caches translations.
/proc/self/stat fields 10 and 12 count minor and major faults.
VmSize is virtual size, VmRSS resident size, VmSwap swapped size.
AlwaysPreTouch, UseLargePages, and UseTransparentHugePages default to false on JDK 25.
THP mode is read from /sys/kernel/mm/transparent_hugepage/enabled; AnonHugePages shows usage.
vm.swappiness sets how readily the kernel swaps anonymous memory.