Processes, Threads, and Scheduling

This lesson shows what the Linux kernel sees when a JVM runs: one process, many threads, a scheduler that decides which thread runs on which core, and system calls that cross into the kernel. After it, a reader can inspect a running JVM through /proc, pin work to cores, and explain why a latency-sensitive thread gets preempted.

Examples print checks instead of pids, counts, or times, because those differ on every run and machine.

Example 1: processes and threads on Linux

A process owns an address space; a thread is a flow of execution that shares the address space of its process. The Linux kernel represents both as tasks: every thread has its own id and its own directory under /proc/<pid>/task/. A platform Java thread (new Thread) is a native thread with a 1:1 mapping to a kernel task, so starting Java threads adds entries there.

import java.nio.file.Files;
import java.nio.file.Path;
import java.util.concurrent.CountDownLatch;
import java.util.stream.Stream;

public class TasksDemo {
    static long tasks() throws Exception {
        try (Stream<Path> s = Files.list(Path.of("/proc/self/task"))) {
            return s.count();
        }
    }

    public static void main(String[] args) throws Exception {
        long before = tasks();
        CountDownLatch release = new CountDownLatch(1);
        Thread[] ts = new Thread[5];
        for (int i = 0; i < ts.length; i++) {
            ts[i] = new Thread(() -> {
                try { release.await(); } catch (InterruptedException e) { }
            });
            ts[i].start();
        }
        long during = tasks();
        release.countDown();
        for (Thread t : ts) t.join();
        System.out.println("JVM has several tasks at start: " + (before > 5));
        System.out.println("5 Java threads added 5 kernel tasks: " + (during - before == 5));
    }
}

JVM has several tasks at start: true
5 Java threads added 5 kernel tasks: true

A JVM starts with many tasks before main creates any: the main thread, GC threads, JIT compiler threads, and others. Each started Thread adds exactly one task. Virtual threads are different: many of them share a few carrier threads, so they do not add tasks one for one (see Virtual Threads in Depth).

Example 2: ProcessHandle and child processes

ProcessHandle (Java 9) describes a process: its pid, parent, children, and info() with the command and start time where the OS reports them. ProcessBuilder starts a child process, a new process whose parent is the JVM.

import java.util.List;

public class ProcessHandleDemo {
    public static void main(String[] args) throws Exception {
        ProcessHandle self = ProcessHandle.current();
        System.out.println("pid > 0: " + (self.pid() > 0));
        System.out.println("command ends with java: "
            + self.info().command().map(c -> c.endsWith("/java")).orElse(false));

        Process p = new ProcessBuilder("sleep", "5").start();
        List<ProcessHandle> kids = self.children().toList();
        System.out.println("child listed: " + kids.stream().anyMatch(h -> h.pid() == p.pid()));
        System.out.println("child's parent is us: "
            + p.toHandle().parent().map(h -> h.pid() == self.pid()).orElse(false));

        p.destroy();
        p.onExit().get();
        System.out.println("child alive after destroy: " + p.isAlive());
    }
}

pid > 0: true
command ends with java: true
child listed: true
child's parent is us: true
child alive after destroy: false

destroy() sends SIGTERM on Linux and destroyForcibly() sends SIGKILL. onExit() returns a CompletableFuture that completes when the process ends, so the example waits on it before checking isAlive().

Example 3: scheduling and the CPU count

The scheduler is the kernel component that picks which runnable task runs on each core. From Linux 2.6.23 to 6.5 the default was CFS (Completely Fair Scheduler), which runs the task with the least virtual runtime. Since 6.6 it is EEVDF (Earliest Eligible Virtual Deadline First), which keeps the fairness of CFS and adds a virtual deadline per task so that tasks with short time slices get the CPU sooner. Both give each runnable task a share of CPU weighted by its nice value.

The JVM sizes thread pools from Runtime.availableProcessors(), which reads the task's affinity mask and container CPU limits. The nproc command reads the same affinity mask.

import java.io.IOException;

public class CpuCount {
    static String run(String... cmd) throws IOException, InterruptedException {
        Process p = new ProcessBuilder(cmd).redirectErrorStream(true).start();
        String out = new String(p.getInputStream().readAllBytes()).trim();
        p.waitFor();
        return out;
    }

    public static void main(String[] args) throws Exception {
        int jvm = Runtime.getRuntime().availableProcessors();
        int nproc = Integer.parseInt(run("nproc"));
        System.out.println("availableProcessors >= 1: " + (jvm >= 1));
        System.out.println("matches nproc: " + (jvm == nproc));
        System.out.println("pinned child sees 1 CPU: " + run("taskset", "-c", "0", "nproc"));
    }
}

availableProcessors >= 1: true
matches nproc: true
pinned child sees 1 CPU: 1

taskset -c 0 cmd starts cmd with an affinity mask of core 0 only, so nproc inside it prints 1. A JVM started the same way reports availableProcessors() == 1 and sizes its GC and ForkJoinPool threads to match.

Example 4: voluntary and involuntary context switches

A context switch is the kernel stopping one task on a core and running another (CPU Coherence and False Sharing covers its cache cost). It is voluntary when the task blocks (sleep, lock wait, I/O) and involuntary when the scheduler preempts a task that still wants to run. /proc/thread-self/status counts both for the calling thread.

import java.nio.file.Files;
import java.nio.file.Path;

public class ContextSwitches {
    static long field(String name) throws Exception {
        for (String line : Files.readAllLines(Path.of("/proc/thread-self/status"))) {
            if (line.startsWith(name + ":")) return Long.parseLong(line.split("\\s+")[1]);
        }
        throw new IllegalStateException(name);
    }

    public static void main(String[] args) throws Exception {
        long v0 = field("voluntary_ctxt_switches");
        for (int i = 0; i < 20; i++) Thread.sleep(1);
        long v1 = field("voluntary_ctxt_switches");
        System.out.println("20 sleeps caused >= 20 voluntary switches: " + (v1 - v0 >= 20));

        long n0 = field("nonvoluntary_ctxt_switches");
        long x = 0;
        for (long i = 0; i < 200_000_000L; i++) x += i ^ (x >>> 3);
        long n1 = field("nonvoluntary_ctxt_switches");
        System.out.println("involuntary count never decreases: " + (n1 >= n0));
        System.out.println("loop result used: " + (x != 1));
    }
}

20 sleeps caused >= 20 voluntary switches: true
involuntary count never decreases: true
loop result used: true

Each sleep blocks, so the thread gives up the core at least once per call. The involuntary count of a busy loop depends on how many other tasks compete for the core: on an idle machine it can stay at zero, on a loaded one it grows by one at each preemption. A low-latency thread with a rising involuntary count shares its core with other work.

Example 5: CPU affinity from Java

The JDK has no API to set affinity. Two common approaches are starting the whole JVM under taskset, or calling sched_setaffinity through a native library. The current mask is visible in /proc/self/status as Cpus_allowed_list.

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;

public class Affinity {
    static String allowed(String statusText) {
        return statusText.lines().filter(l -> l.startsWith("Cpus_allowed_list:"))
            .map(l -> l.split("\\s+")[1]).findFirst().orElseThrow();
    }

    static String run(String... cmd) throws IOException, InterruptedException {
        Process p = new ProcessBuilder(cmd).start();
        String out = new String(p.getInputStream().readAllBytes());
        p.waitFor();
        return out;
    }

    public static void main(String[] args) throws Exception {
        String mine = allowed(Files.readString(Path.of("/proc/self/status")));
        System.out.println("own mask is non-empty: " + !mine.isEmpty());
        System.out.println("child under taskset -c 0: "
            + allowed(run("taskset", "-c", "0", "cat", "/proc/self/status")));
        System.out.println("child under taskset -c 0,1: "
            + allowed(run("taskset", "-c", "0,1", "cat", "/proc/self/status")));
    }
}

own mask is non-empty: true
child under taskset -c 0: 0
child under taskset -c 0,1: 0-1

Affinity alone does not keep other tasks off a core. The isolcpus= kernel boot parameter removes the listed cores from the scheduler's load balancing, so only tasks pinned there run on them. Trading systems combine isolcpus (often with nohz_full= to stop the timer tick and rcu_nocbs=) with pinning of each hot thread to its own isolated core. A pinned thread on an isolated core shows an involuntary switch count that stays near zero.

Example 6: Java thread priorities on Linux

Thread.setPriority takes 1 to 10. On Linux the default -XX:ThreadPriorityPolicy=0 ignores it: every Java thread keeps the nice value of the process. The nice value is field 19 of /proc/thread-self/stat.

import java.nio.file.Files;
import java.nio.file.Path;
import java.util.concurrent.atomic.AtomicReference;

public class Priorities {
    static String nice() throws Exception {
        String stat = Files.readString(Path.of("/proc/thread-self/stat"));
        String[] f = stat.substring(stat.lastIndexOf(')') + 2).split(" ");
        return f[16];   // field 19 overall; fields after "comm" start at 3
    }

    public static void main(String[] args) throws Exception {
        AtomicReference<String> low = new AtomicReference<>(), high = new AtomicReference<>();
        Thread a = new Thread(() -> { try { low.set(nice()); } catch (Exception e) { } });
        Thread b = new Thread(() -> { try { high.set(nice()); } catch (Exception e) { } });
        a.setPriority(Thread.MIN_PRIORITY);
        b.setPriority(Thread.MAX_PRIORITY);
        a.start(); b.start(); a.join(); b.join();
        System.out.println("Java priorities 1 and 10: " + a.getPriority() + ", " + b.getPriority());
        System.out.println("same kernel nice value: " + low.get().equals(high.get()));
    }
}

Java priorities 1 and 10: 1, 10
same kernel nice value: true

The Java priority is stored on the Thread object and nothing else changes. -XX:ThreadPriorityPolicy=1 maps priorities to nice values, but raising priority (a negative nice) needs root or CAP_SYS_NICE. Real-time policies such as SCHED_FIFO are set from outside the JVM with chrt.

Example 7: system calls and strace

A system call is a switch from user mode into the kernel to ask it for a service: read, write, openat, futex (lock waits), epoll_wait, clock_gettime. Each call costs a mode switch and often a copy, from about a hundred nanoseconds to much more when it blocks. Some calls, such as clock_gettime, are served by the vDSO, a page of kernel code mapped into the process, without entering the kernel. Buffered I/O reduces the number of calls; the program below writes the same bytes with and without a buffer and counts the write calls it makes through strace.

import java.io.BufferedOutputStream;
import java.io.FileOutputStream;
import java.io.OutputStream;
import java.nio.file.Files;
import java.nio.file.Path;

public class Syscalls {
    public static void main(String[] args) throws Exception {
        Path f = Files.createTempFile("sys", ".bin");
        try (OutputStream raw = new FileOutputStream(f.toFile())) {
            for (int i = 0; i < 1000; i++) raw.write(i);           // 1000 write calls
        }
        try (OutputStream buf = new BufferedOutputStream(new FileOutputStream(f.toFile()), 8192)) {
            for (int i = 0; i < 1000; i++) buf.write(i);           // 1 write call at close
        }
        System.out.println("file size: " + Files.size(f));
        Files.delete(f);
    }
}

file size: 1000

FileOutputStream.write(int) issues one write system call per byte. BufferedOutputStream collects bytes in an 8 KiB array and calls write once when it flushes. strace -f -c -e trace=write java Syscalls counts the calls per thread tree. Representative summary (not captured by the checker; counts include the JVM's own writes):

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
100.00    0.004113           4      1003           write
------ ----------- ----------- --------- --------- ----------------
100.00    0.004113           4      1003           total

strace stops the traced process at every call, so it slows it down many times over and is a tool for counting and diagnosis, not for timing.

Interview points

• Linux schedules tasks; a process is a group of tasks sharing an address space. A platform Java thread is one task; virtual threads run on a few carrier tasks.
• /proc/<pid>/task/ lists a process's threads; /proc/thread-self/status has per-thread context switch counts.
• Voluntary switches come from blocking; involuntary ones from preemption. Rising involuntary counts on a hot thread mean core contention.
• CFS picks the task with the least virtual runtime; EEVDF (kernel 6.6+) adds virtual deadlines for latency.
• The JDK has no affinity API: use taskset, numactl, or a native call. Combine with isolcpus/nohz_full to keep other work off the core.
• availableProcessors() follows the affinity mask and container limits.
• Thread.setPriority has no effect on Linux with the default ThreadPriorityPolicy=0.
• Syscalls cost a mode switch; batching (buffers, writev, larger reads) cuts their number. strace counts them but adds heavy overhead.

Summary

• Processes own address spaces; threads share them; Linux calls both tasks.
• ProcessHandle gives pid, parent, children, and process info; ProcessBuilder starts children.
• The default scheduler is EEVDF since Linux 6.6, CFS before it.
• /proc/thread-self/status reports voluntary_ctxt_switches and nonvoluntary_ctxt_switches.
• taskset -c sets affinity at launch; Cpus_allowed_list shows it; isolcpus reserves cores.
• Java thread priorities do not change the kernel nice value by default.
• Unbuffered FileOutputStream.write(int) makes one system call per byte; a buffer makes one per flush.