Escape Analysis, Loop Optimizations, and Generated Assembly

This lesson covers the optimizations C2 applies after inlining: removing allocations and locks for objects that stay inside a compiled method, and transforming loops. It builds on Tiered Compilation, Inlining, and Intrinsics. After it, a reader can predict whether a short-lived object costs a heap allocation, write loops that C2 vectorizes, and read a C2 assembly listing.

Allocation is measured with com.sun.management.ThreadMXBean.getThreadAllocatedBytes, as in the previous lesson, after a warmup that lets C2 compile the measured code. Assembly and JIT logs are representative excerpts.

Example 1: scalar replacement

Escape analysis gives each allocation one of three states. NoEscape: the object is used only inside the compiled method, including inlined callees. ArgEscape: it is passed to a callee that was not inlined but does not store it. GlobalEscape: it is stored in a static field, a heap object, or returned, so other code can reach it. A NoEscape object is scalar replaced: its fields become local variables (often CPU registers) and no heap allocation happens.

import java.lang.management.ManagementFactory;

public class Main {
    record Point(double x, double y) {
        Point plus(Point o) { return new Point(x + o.x, y + o.y); }
        double length2() { return x * x + y * y; }
    }

    static double run(int n) {
        double t = 0;
        for (int i = 0; i < n; i++) {
            Point p = new Point(i, 1).plus(new Point(1, i));   // three Points per iteration
            t += p.length2();
        }
        return t;
    }

    public static void main(String[] args) {
        var threads = (com.sun.management.ThreadMXBean) ManagementFactory.getThreadMXBean();
        long id = Thread.currentThread().threadId();
        for (int i = 0; i < 20_000; i++) run(1_000);
        long before = threads.getThreadAllocatedBytes(id);
        double r = run(100_000);
        long bytes = threads.getThreadAllocatedBytes(id) - before;
        System.out.println("result: " + r);
        System.out.println("heap bytes per iteration: " + Math.round(bytes / 100_000.0));
    }
}

result: 6.666766667E14
heap bytes per iteration: 0

The same program with -XX:-DoEscapeAnalysis:

import java.lang.management.ManagementFactory;

public class Main {
    record Point(double x, double y) {
        Point plus(Point o) { return new Point(x + o.x, y + o.y); }
        double length2() { return x * x + y * y; }
    }

    static double run(int n) {
        double t = 0;
        for (int i = 0; i < n; i++) {
            Point p = new Point(i, 1).plus(new Point(1, i));
            t += p.length2();
        }
        return t;
    }

    public static void main(String[] args) {
        var threads = (com.sun.management.ThreadMXBean) ManagementFactory.getThreadMXBean();
        long id = Thread.currentThread().threadId();
        for (int i = 0; i < 20_000; i++) run(1_000);
        long before = threads.getThreadAllocatedBytes(id);
        double r = run(100_000);
        long bytes = threads.getThreadAllocatedBytes(id) - before;
        System.out.println("result: " + r);
        System.out.println("heap bytes per iteration: " + Math.round(bytes / 100_000.0));
    }
}

result: 6.666766667E14
heap bytes per iteration: 96

Without escape analysis each iteration allocates three 32-byte objects. With it, plus and length2 are inlined, all three Points are NoEscape, and the loop runs on plain double values. Value-style code (small records, immutable results) is therefore often free after warmup, but only in C2-compiled code: the interpreter and C1 always allocate.

Example 2: what defeats scalar replacement

Scalar replacement needs every use of the object to be visible and every field access to be resolvable at compile time. The program measures four methods, one per compiled loop.

import java.lang.management.ManagementFactory;

public class Main {
    record Point(double x, double y) {}
    static Point last;

    static double global(int i) { Point p = new Point(i, 1); last = p; return p.x(); }
    static double sometimes(int i) {
        Point p = new Point(i, 1);
        if ((i & 127) == 0) last = p;          // escapes on 1 call in 128
        return p.x();
    }
    static double constIndex(int i) { int[] a = new int[4]; a[0] = i; a[3] = 1; return a[0] + a[3]; }
    static double varIndex(int i) { int[] a = new int[4]; a[i & 3] = i; return a[0] + a[1]; }

    static final com.sun.management.ThreadMXBean THREADS =
        (com.sun.management.ThreadMXBean) ManagementFactory.getThreadMXBean();

    static double run(int which, int n) {
        double s = 0;
        for (int i = 0; i < n; i++) {
            s += switch (which) {                 // direct static calls, all inlined
                case 0 -> global(i);
                case 1 -> sometimes(i);
                case 2 -> constIndex(i);
                default -> varIndex(i);
            };
        }
        return s;
    }

    static long bytesPerCall(int which) {
        for (int r = 0; r < 20_000; r++) run(which, 1_000);
        long id = Thread.currentThread().threadId();
        long before = THREADS.getThreadAllocatedBytes(id);
        run(which, 100_000);
        return Math.round((THREADS.getThreadAllocatedBytes(id) - before) / 100_000.0);
    }

    public static void main(String[] args) {
        System.out.println("stored in a static field: " + bytesPerCall(0));
        System.out.println("escapes on a rare path:   " + bytesPerCall(1));
        System.out.println("array, constant indexes:  " + bytesPerCall(2));
        System.out.println("array, variable index:    " + bytesPerCall(3));
    }
}

stored in a static field: 32
escapes on a rare path:   32
array, constant indexes:  0
array, variable index:    32

Other blockers: a callee that is not inlined (the object becomes ArgEscape or worse, so it is allocated, although locks on it can still be removed), arrays longer than EliminateAllocationArraySizeLimit (64 elements), and the object flowing into a merge point (p = cond ? a : b) in some shapes. A branch that never ran during profiling is different from a rare one: it becomes an uncommon trap, and the escape on it does not count. Graal, a separate JIT compiler, implements partial escape analysis, which allocates only on the escaping path.

Example 3: lock elimination and lock coarsening

A lock on a NoEscape object cannot be contended, because no other thread can reach the object, so C2 removes the lock and unlock. StringBuffer synchronizes every method, which makes it the standard case. Lock coarsening is the related optimization for shared objects: adjacent synchronized blocks on the same object are merged into one block, which keeps the result correct (each block was atomic, and the merged block is too) and cuts the number of lock operations.

import java.lang.management.ManagementFactory;

public class Main {
    static final StringBuffer shared = new StringBuffer();

    static int local(int i) {
        StringBuffer b = new StringBuffer();     // NoEscape: locks removed, then the object
        b.append('a').append('b');
        return b.length() + i;
    }

    static long run(int n) {
        long s = 0;
        for (int i = 0; i < n; i++) s += local(i);
        return s;
    }

    static void coarsened(int i) {
        shared.append('x');                      // three adjacent locks on one object
        shared.append(i % 10);
        shared.append(';');
    }

    public static void main(String[] args) {
        var threads = (com.sun.management.ThreadMXBean) ManagementFactory.getThreadMXBean();
        long id = Thread.currentThread().threadId();
        for (int i = 0; i < 20_000; i++) run(1_000);
        long before = threads.getThreadAllocatedBytes(id);
        long s = run(100_000);
        long bytes = threads.getThreadAllocatedBytes(id) - before;
        System.out.println("local StringBuffer heap bytes per call: " + Math.round(bytes / 100_000.0));

        for (int i = 0; i < 30; i++) coarsened(i);
        System.out.println("shared: " + shared.substring(0, 12) + "... length " + shared.length());
        if (s == 0) System.out.println("unreachable");
    }
}

local StringBuffer heap bytes per call: 0
shared: x0;x1;x2;x3;... length 90

With -XX:-DoEscapeAnalysis the same local method allocates the buffer and its byte array on every call and keeps both locks. Coarsening is limited so that a merged block does not hold the lock for a long time; C2 does not merge across loop iterations without bound. Neither optimization is a reason to pick StringBuffer: StringBuilder has no locks to remove, and its behavior does not depend on the JIT.

Example 4: loop unrolling and range check elimination

Unrolling repeats a loop body several times per iteration, so the loop counter and branch run less often; LoopUnrollLimit (60) bounds the size of the unrolled body. Every array access in Java checks 0 <= i < length. For a counted loop (an int index with a known bound, from the earlier lesson's Example 7), C2 proves the range once before the loop, an optimization called loop predication, and drops the per-access checks. An index read from another array cannot be proven, so its check stays.

public class Main {
    static long direct(int[] a) {
        long s = 0;
        for (int i = 0; i < a.length; i++) s += a[i];        // checks removed, body unrolled
        return s;
    }

    static long indirect(int[] a, int[] idx) {
        long s = 0;
        for (int i = 0; i < idx.length; i++) s += a[idx[i]];  // a[...] keeps its check
        return s;
    }

    public static void main(String[] args) {
        int[] a = new int[1_024];
        int[] idx = new int[1_024];
        for (int i = 0; i < a.length; i++) {
            a[i] = i;
            idx[i] = (i * 7) % a.length;
        }
        long d = 0, x = 0;
        for (int r = 0; r < 20_000; r++) {
            d += direct(a);
            x += indirect(a, idx);
        }
        System.out.println("direct: " + d + ", indirect: " + x);
        try {
            idx[5] = 5_000;
            indirect(a, idx);
        } catch (ArrayIndexOutOfBoundsException e) {
            System.out.println("still checked: " + e.getMessage());
        }
    }
}

direct: 10475520000, indirect: 10475520000
still checked: Index 5000 out of bounds for length 1024

Range check elimination never changes behavior: if the bound cannot be proven, the check stays, and if a predicate fails at run time, the code deoptimizes and throws the exception at the right index. Loops over long indexes, or with a bound that changes inside the loop, are harder for C2 to treat as counted loops.

Example 5: loop-invariant hoisting

An expression whose inputs do not change inside the loop is computed once before it. This is the same transformation that makes a loop on a non-volatile flag spin forever in Visibility and Reordering: the field read is invariant as far as one thread can tell, so C2 moves it out.

public class Main {
    static double rate = 1.0825;          // plain field: hoistable
    static int scaleBits = 4;

    static double total(double[] prices) {
        double t = 0;
        for (double p : prices) {
            t += p * rate * (1 << scaleBits);   // rate and 1 << scaleBits read once
        }
        return t;
    }

    public static void main(String[] args) {
        double[] prices = new double[1_000];
        for (int i = 0; i < prices.length; i++) prices[i] = 100 + i % 4;
        double sum = 0;
        for (int r = 0; r < 10_000; r++) sum += total(prices);
        System.out.printf("total: %.2f%n", sum);
    }
}

total: 17579800000.00

Hoisting is legal because no code in the loop writes rate or scaleBits, and the memory model lets a thread ignore other threads' writes to plain fields. A volatile field or a call that C2 could not inline (which might write the field) blocks it. Writing the hoist by hand (double k = rate * 16; before the loop) makes the intent explicit and does not depend on inlining.

Example 6: auto-vectorization

SIMD instructions (single instruction, multiple data) process several array elements per instruction: an AVX2 register holds 8 ints. C2's SuperWord pass packs adjacent iterations of an unrolled loop into such instructions. It needs a counted loop, simple element-wise operations on primitive arrays, no calls that were not inlined, and no dependency between iterations. The result is identical with -XX:-UseSuperWord; only speed differs.

public class Main {
    static void add(int[] a, int[] b, int[] c) {
        for (int i = 0; i < c.length; i++) c[i] = a[i] + b[i];     // vectorizable
    }

    static void prefix(int[] c) {
        for (int i = 1; i < c.length; i++) c[i] += c[i - 1];      // depends on the previous element
    }

    public static void main(String[] args) {
        int[] a = new int[4_096], b = new int[4_096], c = new int[4_096];
        for (int i = 0; i < a.length; i++) { a[i] = i; b[i] = 2 * i; }
        for (int r = 0; r < 20_000; r++) add(a, b, c);
        System.out.println("c[4095] = " + c[4_095]);
        prefix(c);
        System.out.println("prefix c[4095] = " + c[4_095]);
    }
}

c[4095] = 12285
prefix c[4095] = 25159680

add vectorizes; prefix does not, because each element needs the one before it. A float or double sum into one variable is not vectorized, because reordering floating-point additions changes the rounded result and Java requires the sequential order. The Vector API (Example 8) gives explicit control when auto-vectorization does not apply.

Example 7: reading assembly with -XX:+PrintAssembly

-XX:+UnlockDiagnosticVMOptions -XX:+PrintAssembly prints the machine code of every compiled method. It needs the hsdis disassembler plugin (hsdis-amd64.so) in the JDK's lib directory or on the library path; OpenJDK does not ship it, and without it the JVM logs "Loading hsdis library failed" and prints raw bytes. -XX:CompileCommand=print,Main::add limits output to one method. The command, for Example 6's program:

java -XX:+UnlockDiagnosticVMOptions -XX:CompileCommand=print,Main::add Main

Illustration of the shape of the C2 main loop of add on an AVX2 machine. It was written for this lesson, not captured from a run (hsdis is not installed here), so register choices and offsets in real output differ:

  ;; B12: main vectorized loop, 8 ints per instruction, unrolled 4 times
  0x...a0:   vmovdqu ymm0, ymmword ptr [rsi + r10*4 + 0x10]
  0x...a7:   vpaddd  ymm0, ymm0, ymmword ptr [rdx + r10*4 + 0x10]
  0x...ae:   vmovdqu ymmword ptr [rcx + r10*4 + 0x10], ymm0
  0x...b5:   vmovdqu ymm1, ymmword ptr [rsi + r10*4 + 0x30]
  ...
  0x...e8:   add     r10d, 0x20
  0x...ec:   cmp     r10d, r11d
  0x...ef:   jl      0x...a0
  ;; post loop handles the remaining elements one at a time

What to look for: ymm/zmm registers and vp... instructions mean vectorization; the step (0x20 = 32 elements) shows unrolling; no cmp/jae pair to an exception stub inside the loop means the range checks are gone; call instructions inside a hot loop mean something was not inlined. + 0x10 is the array header size (16 bytes). JITWatch is an open-source viewer for the XML log from -XX:+UnlockDiagnosticVMOptions -XX:+LogCompilation -XX:+PrintAssembly: it shows source, bytecode, and assembly side by side and lists inlining failures and eliminated allocations. In practice, PrintInlining and allocation measurements answer most questions; assembly is for the last step.

Example 8: the Vector API

The Vector API (jdk.incubator.vector) expresses SIMD operations directly, with a species that matches the CPU's register width. It has been an incubator module since Java 16 and is still incubating in Java 25, so it needs --add-modules jdk.incubator.vector at both compile and run time, and its API may change. The checker passes JVM options when running but not javac options, so this example is marked ignore.

import jdk.incubator.vector.IntVector;
import jdk.incubator.vector.VectorSpecies;

public class Main {
    static final VectorSpecies<Integer> S = IntVector.SPECIES_PREFERRED;

    static void add(int[] a, int[] b, int[] c) {
        int i = 0;
        for (; i < S.loopBound(c.length); i += S.length()) {
            IntVector va = IntVector.fromArray(S, a, i);
            IntVector vb = IntVector.fromArray(S, b, i);
            va.add(vb).intoArray(c, i);
        }
        for (; i < c.length; i++) c[i] = a[i] + b[i];      // tail
    }

    public static void main(String[] args) {
        int[] a = new int[1_000], b = new int[1_000], c = new int[1_000];
        for (int i = 0; i < a.length; i++) { a[i] = i; b[i] = 2 * i; }
        add(a, b, c);
        System.out.println("c[999] = " + c[999]);
    }
}

c[999] = 2997

SPECIES_PREFERRED is 8 int lanes on AVX2 and 16 on AVX-512. loopBound rounds the length down to a multiple of the lane count, and the scalar tail handles the rest. C2 compiles each vector operation to one instruction when the CPU supports it; in the interpreter, or on an unsupported CPU, the operations run as slow Java fallbacks.

Interview points

• Escape states: NoEscape (scalar replaced), ArgEscape (allocated, locks removable), GlobalEscape (allocated).
• Scalar replacement depends on inlining and happens only in C2-compiled code.
• C2 does not do partial escape analysis: an object that escapes on any profiled path is allocated on every call.
• Arrays are scalar replaced only with constant indexes and at most 64 elements.
• Lock elimination removes locks on non-escaping objects; coarsening merges adjacent locks on one object.
• Counted loops get unrolling, range check elimination by loop predication, and SuperWord vectorization.
• Loop-invariant hoisting is the same transformation behind the non-volatile stop-flag bug.
• PrintAssembly needs hsdis; JITWatch reads LogCompilation output.
• The Vector API is incubating in Java 25 and needs --add-modules jdk.incubator.vector.

Summary

• C2 removes allocations and locks for objects that do not escape the compiled code.
• Allocated bytes per call, measured after warmup, show whether scalar replacement happened.
• Loops with an int index, a fixed bound, and simple bodies are unrolled, bounds-check free, and vectorized.
• Assembly listings confirm vectorization, unrolling, and missing checks; inlining logs come first.