Tiered Compilation, Inlining, and Intrinsics

This lesson explains what the JIT does with a method once it is hot: which tier compiles it, what profile data it uses, which calls it inlines, and which library methods it replaces with hand-written machine code. It builds on Safepoints, Warmup, and Deoptimization, which covers the interpreter, C1, C2, warmup, OSR, and deoptimization. After it, a reader can read an inlining log and explain why a call site is fast or slow.

All examples run on JDK 25, x64, with default flags unless the fence names a jvm= flag. JIT logs are representative excerpts: ids, times, and the order of lines vary by run. Runnable examples print checks instead of log lines, so their output is exact.

Example 1: the inlining limits from code

The JIT's limits are JVM flags. HotSpotDiagnosticMXBean (in com.sun.management) reads any flag's current value and where it came from, which is the same data -XX:+PrintFlagsFinal prints.

import com.sun.management.HotSpotDiagnosticMXBean;
import java.lang.management.ManagementFactory;

public class Main {
    public static void main(String[] args) {
        HotSpotDiagnosticMXBean hs = ManagementFactory.getPlatformMXBean(HotSpotDiagnosticMXBean.class);
        String[] flags = {
            "MaxInlineSize", "FreqInlineSize", "MaxInlineLevel", "MaxRecursiveInlineLevel",
            "MaxTrivialSize", "InlineSmallCode", "TypeProfileWidth",
            "Tier3InvocationThreshold", "Tier4InvocationThreshold"
        };
        for (String f : flags) {
            var opt = hs.getVMOption(f);
            System.out.println(f + " = " + opt.getValue() + " (" + opt.getOrigin() + ")");
        }
    }
}

MaxInlineSize = 35 (DEFAULT)
FreqInlineSize = 325 (DEFAULT)
MaxInlineLevel = 15 (DEFAULT)
MaxRecursiveInlineLevel = 1 (DEFAULT)
MaxTrivialSize = 6 (DEFAULT)
InlineSmallCode = 2500 (DEFAULT)
TypeProfileWidth = 2 (DEFAULT)
Tier3InvocationThreshold = 200 (DEFAULT)
Tier4InvocationThreshold = 5000 (DEFAULT)

The sizes are bytecode sizes, the byte count javap -c shows, not source lines.

Large methods are not slow in themselves, but a hot method over 325 bytes is not inlined into its callers, and everything inlining would have enabled is lost with it.

Example 2: tiers and the profile they collect

Tiers 0 (interpreter), 2, and 3 (C1 with profiling) fill a per-method profile: invocation and loop counters, how often each branch went each way, and which receiver classes each virtual call saw. C2 at tier 4 reads that profile. Tier 1 is C1 without profiling, used for trivial methods such as record accessors, because a profile would tell C2 nothing new about them.

public class Main {
    record Order(long price, int qty) {}

    static long notional(Order o) {
        if (o.qty() > 0) return o.price() * o.qty();
        return 0;
    }

    public static void main(String[] args) {
        Order[] book = new Order[1_000];
        for (int i = 0; i < book.length; i++) book[i] = new Order(100 + i % 10, 1 + i % 5);
        long total = 0;
        for (int round = 0; round < 50; round++) {
            for (Order o : book) total += notional(o);
        }
        System.out.println("total notional: " + total);
    }
}

total notional: 15775000

Excerpt with -XX:+PrintCompilation (the fourth column is the tier):

     26   15       1       Main$Order::qty (5 bytes)
     26   16       1       Main$Order::price (5 bytes)
     26   17       3       Main::notional (20 bytes)
     26   18       4       Main::notional (20 bytes)
     27   17       3       Main::notional (20 bytes)   made not entrant: not used

The accessors go straight to tier 1. notional goes 0, 3, 4. When the C2 queue is long, the policy compiles at tier 2 (cheaper counters only) first and moves to tier 3 later. With -XX:TieredStopAtLevel=1 every method, including notional, stops at tier 1: startup is faster and peak speed is lower. Since the profile records only what ran, the branch qty() > 0 was always taken here, so C2 compiles the return 0 path as an uncommon trap.

Example 3: reading -XX:+PrintInlining

-XX:+PrintInlining is a diagnostic flag, so it needs -XX:+UnlockDiagnosticVMOptions before it. It prints the inlining tree of each compiled method to standard output, which mixes it with program output; that is why the program's own output below is checked without the flag.

public class Main {
    static int small(int x) {
        return x * 31 + 7;
    }

    static int large(int x) {
        int a = x;
        a = a * 31 + 1; a = a * 31 + 2; a = a * 31 + 3; a = a * 31 + 4; a = a * 31 + 5;
        a = a * 31 + 6; a = a * 31 + 7; a = a * 31 + 8; a = a * 31 + 9; a = a * 31 + 10;
        return a;
    }

    public static void main(String[] args) {
        int s = 0;
        for (int i = 0; i < 100_000; i++) {
            s += small(i);
            s += large(i);
        }
        System.out.println("checksum: " + s);
    }
}

checksum: 656190848

Representative excerpt with -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining:

                              @ 12   Main::small (8 bytes)   inline
                              @ 19   Main::large (79 bytes)   failed to inline: callee is too large

Each line is one call site: @ 8 is the bytecode index of the call in the caller, then the callee and its bytecode size, then the decision. Indentation shows nesting. Common decisions:

Here main runs once and is compiled by OSR, so its call sites look rare to the policy: the 8-byte method is inlined because it is under 35 bytes, and the 79-byte method is refused. The same method called from a hot method would be inlined, since 79 is under 325.

Example 4: inlining makes later optimizations possible

Inlining removes the call cost, but its main value is that the caller and callee become one unit that C2 optimizes together. One such optimization removes a heap allocation whose object never leaves the compiled unit; it is covered in Escape Analysis, Loop Optimizations, and Generated Assembly. Here it serves only as a visible effect: allocated bytes per call, read with com.sun.management.ThreadMXBean.getThreadAllocatedBytes.

import java.lang.management.ManagementFactory;

public class Main {
    record Point(double x, double y) {}

    static Point make(int i) {
        return new Point(i, 1);
    }

    static double sum(int n) {
        double t = 0;
        for (int i = 0; i < n; i++) {
            Point p = make(i);
            t += p.x() + p.y();
        }
        return t;
    }

    public static void main(String[] args) {
        var threads = (com.sun.management.ThreadMXBean) ManagementFactory.getThreadMXBean();
        long id = Thread.currentThread().threadId();
        for (int i = 0; i < 20_000; i++) sum(1_000);          // warm up until C2 compiles sum
        long before = threads.getThreadAllocatedBytes(id);
        double r = sum(100_000);
        long bytes = threads.getThreadAllocatedBytes(id) - before;
        System.out.println("result: " + r);
        System.out.println("heap bytes per call: " + Math.round(bytes / 100_000.0));
    }
}

result: 5.00005E9
heap bytes per call: 0

The same program with inlining switched off (-XX:-Inline):

import java.lang.management.ManagementFactory;

public class Main {
    record Point(double x, double y) {}

    static Point make(int i) {
        return new Point(i, 1);
    }

    static double sum(int n) {
        double t = 0;
        for (int i = 0; i < n; i++) {
            Point p = make(i);
            t += p.x() + p.y();
        }
        return t;
    }

    public static void main(String[] args) {
        var threads = (com.sun.management.ThreadMXBean) ManagementFactory.getThreadMXBean();
        long id = Thread.currentThread().threadId();
        for (int i = 0; i < 20_000; i++) sum(1_000);
        long before = threads.getThreadAllocatedBytes(id);
        double r = sum(100_000);
        long bytes = threads.getThreadAllocatedBytes(id) - before;
        System.out.println("result: " + r);
        System.out.println("heap bytes per call: " + Math.round(bytes / 100_000.0));
    }
}

result: 5.00005E9
heap bytes per call: 32

With inlining, make is part of sum, C2 sees that the Point is used only for two field reads, and the allocation disappears. Without inlining, make returns a real object, so every call allocates 32 bytes (a 12-byte header and two 8-byte fields, padded to a multiple of 8). The result is identical in both runs; only the cost differs.

Example 5: monomorphic, bimorphic, and megamorphic call sites

An interface call has no fixed target. The profile records up to TypeProfileWidth (2) receiver classes per call site. With one class seen (monomorphic), C2 inlines that class's method behind a cheap class check. With two (bimorphic), it inlines both behind two checks. With more (megamorphic), it emits a real virtual call through the class's method table, unless one class covers at least TypeProfileMajorReceiverPercent (90) percent of calls. The interpreter and C1 use inline caches for the same purpose: a call site remembers the last receiver class and its target, and falls back to a table lookup when a new class arrives. Each lambda below is its own class.

import java.lang.management.ManagementFactory;

public class Main {
    record Point(double x, double y) {}
    interface Maker { Point make(int i); }

    static final com.sun.management.ThreadMXBean THREADS =
        (com.sun.management.ThreadMXBean) ManagementFactory.getThreadMXBean();

    static double sum(Maker m, int n) {
        double t = 0;
        for (int i = 0; i < n; i++) {
            Point p = m.make(i);                 // the call site being profiled
            t += p.x() + p.y();
        }
        return t;
    }

    static long bytesPerCall(Maker m) {
        for (int i = 0; i < 20_000; i++) sum(m, 1_000);
        long id = Thread.currentThread().threadId();
        long before = THREADS.getThreadAllocatedBytes(id);
        sum(m, 100_000);
        return Math.round((THREADS.getThreadAllocatedBytes(id) - before) / 100_000.0);
    }

    public static void main(String[] args) {
        Maker a = i -> new Point(i, 1);
        Maker b = i -> new Point(1, i);
        Maker c = i -> new Point(i, i);
        System.out.println("monomorphic, bytes per call: " + bytesPerCall(a));
        bytesPerCall(b);                          // the site has now seen a and b
        System.out.println("bimorphic, bytes per call: " + bytesPerCall(a));
        bytesPerCall(c);                          // a third class
        System.out.println("megamorphic, bytes per call: " + bytesPerCall(a));
    }
}

monomorphic, bytes per call: 0
bimorphic, bytes per call: 0
megamorphic, bytes per call: 32

Representative PrintInlining lines for the call at @ 14 in the bimorphic and megamorphic phases:

  @ 14   Main$$Lambda/0x...::make (5 bytes)   inline (hot)   \-> TypeProfile (263000/263409 counts) = Main$$Lambda+0x...
  @ 14   Main$Maker::make (0 bytes)   failed to inline: virtual call

After the third class arrives, the compiled sum hits a failed type check, deoptimizes, and is recompiled with a virtual call that C2 cannot see through, so the Point is allocated even when only a is passed again. A hot call site that sees many implementations (handlers, strategies, visitors) is a common cause of a slow path in a code base. Splitting the site per type, or making one class dominate it, restores inlining.

Example 6: class hierarchy analysis

When only one loaded class implements an abstract method, C2 can bind the call without any profile, using the loaded class hierarchy. The compiled code records this as a dependency, and loading a second subclass deoptimizes it (shown in the earlier lesson's Example 6). -XX:-UseTypeProfile turns off the type profile so that only CHA remains.

import java.lang.management.ManagementFactory;

public class Main {
    record Point(double x, double y) {}
    static abstract class Maker { abstract Point make(int i); }
    static final class Only extends Maker {
        Point make(int i) { return new Point(i, 1); }
    }

    static double sum(Maker m, int n) {
        double t = 0;
        for (int i = 0; i < n; i++) {
            Point p = m.make(i);
            t += p.x() + p.y();
        }
        return t;
    }

    public static void main(String[] args) {
        var threads = (com.sun.management.ThreadMXBean) ManagementFactory.getThreadMXBean();
        Maker m = new Only();
        for (int i = 0; i < 20_000; i++) sum(m, 1_000);
        long id = Thread.currentThread().threadId();
        long before = threads.getThreadAllocatedBytes(id);
        sum(m, 100_000);
        System.out.println("heap bytes per call: "
            + Math.round((threads.getThreadAllocatedBytes(id) - before) / 100_000.0));
    }
}

heap bytes per call: 0

Representative PrintInlining line: @ 14 Main$Only::make (11 bytes) inline (hot). final classes and methods, private methods, and static methods need no CHA because they have exactly one target. Marking a class final does not make it faster when CHA already proves a single implementation, but it removes the risk of a deoptimization when another subclass is loaded later.

Example 7: intrinsics

An intrinsic is a method whose bytecode the JIT ignores and replaces with machine code it generates itself. The method is marked @IntrinsicCandidate in the JDK source. Examples: System.arraycopy (vectorized copy), Math.sqrt and Math.fma (single CPU instructions), Integer.bitCount (popcnt), Long.numberOfTrailingZeros (tzcnt), and String.equals (compares many bytes at once).

import java.util.Arrays;

public class Main {
    public static void main(String[] args) {
        int[] src = new int[1_000];
        Arrays.setAll(src, i -> i);
        int[] dst = new int[1_000];
        long bits = 0;
        boolean allEqual = true;
        for (int round = 0; round < 50_000; round++) {
            System.arraycopy(src, 0, dst, 0, src.length);
            bits += Integer.bitCount(round) + Long.numberOfTrailingZeros(round | 1024);
            allEqual &= "EURUSD".equals(new String(new char[] {'E', 'U', 'R', 'U', 'S', 'D'}));
        }
        System.out.println("copy matches: " + Arrays.equals(src, dst));
        System.out.println("bit total: " + bits);
        System.out.println("strings equal: " + allEqual);
        System.out.println("sqrt(2): " + Math.sqrt(2));
        System.out.println("fma(0.1, 10, -1): " + Math.fma(0.1, 10, -1));
    }
}

copy matches: true
bit total: 432466
strings equal: true
sqrt(2): 1.4142135623730951
fma(0.1, 10, -1): 5.551115123125783E-17

Representative PrintInlining lines from JDK methods compiled during such a run:

  @ 22   java.lang.System::arraycopy (0 bytes)   intrinsic
  @ 22   java.lang.Integer::numberOfTrailingZeros (80 bytes)   (intrinsic)

Math.fma(0.1, 10, -1) rounds once, so it shows the representation error of 0.1 that 0.1 * 10 - 1 hides. Intrinsics produce the same results as the Java code they replace, so they never change behavior, only speed. A hand-written loop that copies an array is not an intrinsic; replacing it with System.arraycopy or Arrays.copyOf lets the JIT use its copy stub. -XX:+UnlockDiagnosticVMOptions -XX:+PrintIntrinsics lists the intrinsics used in each compilation.

Example 8: the code cache

Compiled code lives in the code cache, outside the Java heap. With tiered compilation and a reserved size of at least 240 MB, the cache is segmented (-XX:+SegmentedCodeCache, on by default since Java 9) into three heaps: non-method code (interpreter, stubs, adapters), profiled methods (tiers 2 and 3), and non-profiled methods (tiers 1 and 4). ReservedCodeCacheSize is 240 MB by default with tiered compilation. When the cache fills, the JVM prints "CodeCache is full. Compiler has been disabled." and new hot code stays interpreted, which shows as a slow service with no GC or CPU explanation. Run with -XX:ReservedCodeCacheSize=256m so the sizes are fixed.

import java.lang.management.ManagementFactory;
import java.lang.management.MemoryPoolMXBean;
import java.lang.management.MemoryType;
import java.util.ArrayList;
import java.util.List;

public class Main {
    public static void main(String[] args) {
        List<String> names = new ArrayList<>();
        long reserved = 0;
        boolean inUse = false;
        for (MemoryPoolMXBean pool : ManagementFactory.getMemoryPoolMXBeans()) {
            if (pool.getType() == MemoryType.NON_HEAP && pool.getName().startsWith("CodeHeap")) {
                names.add(pool.getName());
                reserved += pool.getUsage().getMax();
                inUse |= pool.getUsage().getUsed() > 0;
            }
        }
        names.sort(null);
        System.out.println("code heaps: " + names);
        System.out.println("total reserved MB: " + reserved / (1024 * 1024));
        System.out.println("some code compiled: " + inUse);
    }
}

code heaps: [CodeHeap 'non-nmethods', CodeHeap 'non-profiled nmethods', CodeHeap 'profiled nmethods']
total reserved MB: 256
some code compiled: true

Below 240 MB the cache is one pool named CodeCache. -XX:+PrintCodeCache prints usage at exit, jcmd <pid> Compiler.codecache prints it on a running process, and JFR records jdk.CodeCacheFull events. Services that load much generated code (many lambdas, proxies, scripting engines) may need a larger reserved size.

Interview points

• Tiers: 0 interpreter, 1 C1 without profiling (trivial methods), 2 and 3 C1 with profiling, 4 C2 using the profile.
• Inlining limits: 35 bytes for any site (MaxInlineSize), 325 for hot sites (FreqInlineSize, x64), 15 levels (MaxInlineLevel), compiled callees under 2,500 bytes of machine code (InlineSmallCode).
• -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining shows each call site's decision and its reason.
• Inlining matters mostly because it enables later optimizations such as allocation removal.
• One or two receiver classes: inlined behind type checks. Three or more: a virtual call, unless one class takes 90 percent.
• CHA binds a call when one loaded class implements it; loading another deoptimizes dependent code.
• Intrinsics replace methods like System.arraycopy, Math.sqrt, Integer.bitCount, and String.equals with JIT-generated code.
• A full code cache disables the compiler; size it with ReservedCodeCacheSize.

Summary

• C2 compiles from the profile gathered in tiers 0, 2, and 3; tier 1 handles trivial methods.
• Inlining decisions depend on bytecode size, call frequency, depth, and the receiver types seen.
• Megamorphic call sites block inlining and everything that depends on it.
• Intrinsics give hand-written machine code for selected JDK methods with identical results.
• The code cache is segmented by tier when it is 240 MB or larger and must not fill up.