The Runtime & Async Executors

A Rust program is the machine code. There's no VM, no class loader, no bytecode, no GC, and no JIT. What remains is small enough to describe completely:

1. The standard library's runtime support: a few steps before main, panic handling, threads, and exit.
2. An async executor, if you use async. The standard library doesn't include one, so you pick one (almost always Tokio). The executor is the closest Rust has to "the runtime", and it's where most senior-level runtime questions live.

Part 1: The standard library's runtime

1. What happens before main

On Linux, the operating system loads the binary and the C library, and the C library calls Rust's entry point. The standard library then does a handful of things and calls your main. Tracing the system calls of a hello-world program (strace ./app) shows all of it:

poll([{fd=0, events=0}, {fd=1, events=0}, {fd=2, events=0}], 3, 0) = 0 (Timeout)
rt_sigaction(SIGPIPE, {sa_handler=SIG_IGN, ...}, ...) = 0
sigaltstack({ss_sp=0x7f04907c2000, ss_flags=0, ss_size=11952}, NULL) = 0
rt_sigaction(SIGSEGV, {sa_handler=0x560d55d69960, ... SA_ONSTACK|SA_SIGINFO ...}, NULL, 8) = 0
rt_sigaction(SIGBUS, {sa_handler=0x560d55d69960, ... SA_ONSTACK|SA_SIGINFO ...}, NULL, 8) = 0

There's no interpreter or runtime to start. A stripped release build of hello world is about 350 KB and runs in about 2 ms (time ./app reports real 0m0.002s), most of which is the kernel loading the program.

2. main can return a Result or an ExitCode

main can return (), Result<(), E> where E: Debug, or std::process::ExitCode. The runtime turns the return value into the process exit status, through the Termination trait. Returning Err prints Error: plus the error's Debug output to stderr and exits with status 1.

use std::process::ExitCode;

fn check_config(path: &str) -> Result<(), String> {
    if path.ends_with(".toml") { Ok(()) } else { Err(format!("{path}: expected a .toml file")) }
}

fn main() -> ExitCode {
    let files = ["app.toml", "db.yaml", "cache.toml"];
    let failures: Vec<String> = files.iter().filter_map(|f| check_config(f).err()).collect();
    for f in &failures {
        println!("invalid: {f}");
    }
    if failures.is_empty() {
        ExitCode::SUCCESS
    } else {
        ExitCode::from(2) // like System.exit(2), but destructors still run
    }
}

invalid: db.yaml: expected a .toml file

Returning an ExitCode from main is better than calling std::process::exit, because returning lets every value's Drop run (flushing files, closing connections) before the process ends.

3. process::exit skips destructors

std::process::exit(code) ends the process immediately. Destructors of values on any thread's stack don't run, like System.exit skipping finally blocks, but without shutdown hooks.

struct TempFile(&'static str);

impl Drop for TempFile {
    fn drop(&mut self) {
        println!("deleting {}", self.0);
    }
}

fn main() {
    {
        let _scratch = TempFile("scratch-1.tmp");
        println!("work in a scope");
    } // dropped here as usual

    let _report = TempFile("report.tmp");
    println!("calling process::exit(0)");
    std::process::exit(0); // `report.tmp` is never deleted
}

work in a scope
deleting scratch-1.tmp
calling process::exit(0)

Use exit only where no cleanup matters, for example deep inside a CLI tool's argument handling. In a service, return from main instead.

4. SIGPIPE is ignored, so broken pipes are errors

A C program piped into head is killed by SIGPIPE when head exits. Rust ignores the signal, so the write returns an io::Error with kind BrokenPipe instead. println! turns that error into a panic:

./print-many-lines | head -1

line 0

thread 'main' (17662) panicked at library/std/src/io/stdio.rs:1165:9:
failed printing to stdout: Broken pipe (os error 32)

Command-line tools that write a lot of output should write through a locked, buffered handle and handle the error, so app | head exits quietly:

use std::io::{self, BufWriter, ErrorKind, Write};

fn run() -> io::Result<()> {
    let mut out = BufWriter::new(io::stdout().lock());
    for i in 0..3 {
        writeln!(out, "line {i}")?;
    }
    out.flush()
}

fn main() {
    if let Err(e) = run() {
        if e.kind() != ErrorKind::BrokenPipe {
            eprintln!("error: {e}");
            std::process::exit(1);
        }
    }
}

line 0
line 1
line 2

5. Threads are OS threads with 2 MiB stacks

std::thread::spawn creates one OS thread (1:1). The main thread's stack comes from the OS (usually 8 MiB on Linux, from ulimit -s); spawned threads get 2 MiB by default. The syscall trace of a spawn shows exactly that: mmap(NULL, 2101248, PROT_NONE, ...) is 2 MiB plus one guard page. You can change the size per thread with Builder::stack_size, or for all threads with the RUST_MIN_STACK environment variable.

use std::thread;

fn depth(n: u64) -> u64 {
    let frame = [n; 32]; // 256 bytes of stack per call
    if n == 0 { 0 } else { depth(n - 1) + std::hint::black_box(frame)[0] / n }
}

fn main() {
    // Deep recursion needs more than the default 2 MiB.
    let handle = thread::Builder::new()
        .name("deep-worker".into())
        .stack_size(256 * 1024 * 1024) // reserved, not committed: only touched pages use RAM
        .spawn(|| depth(100_000))
        .unwrap();
    println!("depth result: {}", handle.join().unwrap());
}

depth result: 100000

Stack memory is reserved, not committed: a thread that uses 20 KiB of its 2 MiB stack only costs 20 KiB of RAM. Still, thousands of OS threads mean thousands of kernel objects and context switches, which is why servers with many concurrent connections use async tasks instead (Part 2).

6. Stack overflow aborts the process

Every thread stack ends in a guard page that the process can't touch. Running into it triggers SIGSEGV, which the handler from example 1 recognizes as a stack overflow. It can't be caught: the process prints a message and aborts.

fn depth(n: u64) -> u64 {
    let buf = [n; 64];
    if n == 0 { 0 } else { std::hint::black_box(&buf); depth(n - 1) + buf[1] }
}

fn main() {
    println!("{}", depth(10_000_000));
}

thread 'main' (17642) has overflowed its stack
fatal runtime error: stack overflow, aborting
Aborted (exit status 134)

Rust doesn't guarantee tail-call elimination, so deep recursion over user-controlled input (nested JSON, a deep tree) is a denial-of-service risk. Use an explicit stack (a Vec) or a depth limit. serde_json, for example, refuses input nested deeper than 128 levels by default.

7. Binaries: what's inside and how they link

A Rust binary contains your code, every crate you use, and the parts of the standard library you call, all compiled together. By default on Linux it links dynamically only to the C library:

ldd target/release/app

	linux-vdso.so.1 (0x00007ff84c308000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ff84c271000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff84c000000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ff84c30a000)

For comparison, the unstripped hello world above is 3.9 MB because it includes debug info for the standard library; stripped, it's 350 KB. A web service with Tokio, axum, and serde is typically a few megabytes to a few tens of megabytes, with no runtime to install next to it.

8. No reflection, no class loading

Rust compiles everything ahead of time, and types don't exist at runtime beyond what you ask for. There is no built-in reflection or dynamic loading. The usual replacements:

use std::any::Any;

fn describe(value: &dyn Any) -> String {
    if let Some(n) = value.downcast_ref::<i32>() {
        format!("an i32: {n}")
    } else if let Some(s) = value.downcast_ref::<String>() {
        format!("a String of length {}", s.len())
    } else {
        "something else".to_string()
    }
}

fn main() {
    println!("{}", describe(&42));
    println!("{}", describe(&String::from("hello")));
    println!("{}", describe(&1.5));
}

an i32: 42
a String of length 5
something else

Rust libraries do this work with macros at compile time, which is a large part of why Rust services start instantly.

Part 2: The async runtime

9. Futures need an executor, and std doesn't have one

An async fn returns a future, a state machine that does nothing until something polls it (How Async Works). The standard library defines Future but doesn't include an executor to poll futures, and doesn't include async I/O. A library provides both. #[tokio::main] is a macro that builds a Tokio runtime and blocks on your main future:

fn main() {
    // What #[tokio::main] expands to, roughly:
    let runtime = tokio::runtime::Builder::new_multi_thread().enable_all().build().unwrap();
    runtime.block_on(async {
        let handle = tokio::spawn(async { 21 * 2 });
        println!("answer: {}", handle.await.unwrap());
    });
}

answer: 42

Libraries should be runtime-agnostic where possible, but in practice most of the ecosystem (axum, reqwest, sqlx, tonic) is built on Tokio.

10. Runtime flavors and worker threads

The multi-thread runtime starts one worker thread per CPU core by default, plus a separate pool for blocking work. The current-thread runtime runs every task on the thread that calls block_on.

use tokio::runtime::{Builder, Handle};

async fn where_am_i(task: u32) -> String {
    format!("task {task} on {}", std::thread::current().name().unwrap_or("?").split('-').take(2).collect::<Vec<_>>().join("-"))
}

fn main() {
    let multi = Builder::new_multi_thread().worker_threads(4).thread_name("tokio-worker").enable_all().build().unwrap();
    multi.block_on(async {
        println!("workers: {}", Handle::current().metrics().num_workers());
        let h = tokio::spawn(where_am_i(1));
        println!("{}", h.await.unwrap());
    });

    let single = Builder::new_current_thread().enable_all().build().unwrap();
    single.block_on(async {
        println!("workers: {}", Handle::current().metrics().num_workers());
        let h = tokio::spawn(where_am_i(2));
        println!("{}", h.await.unwrap());
    });
}

workers: 4
task 1 on tokio-worker
workers: 1
task 2 on main

In containers, check what "one per core" means: Rust's available_parallelism (which Tokio uses) respects CPU affinity and cgroup CPU quotas on Linux. If a service is limited to 2 CPUs on a 64-core host, it gets 2 workers. Set worker_threads explicitly (or the TOKIO_WORKER_THREADS environment variable) when you want a different number.

11. Work stealing: how tasks are scheduled

Tokio's multi-thread scheduler uses work stealing:

graph TD
    G[Global queue<br/>tasks spawned from outside the runtime] --> W1
    G --> W2
    subgraph W1[Worker 1]
        L1[LIFO slot] --> Q1[Local queue<br/>256 tasks]
    end
    subgraph W2[Worker 2]
        L2[LIFO slot] --> Q2[Local queue<br/>256 tasks]
    end
    Q1 -. steals half when idle .-> Q2
    Q2 -. steals half when idle .-> Q1

• Each worker has a local run queue of 256 tasks. A task spawned or woken on a worker goes to that worker's queue. When it's full, half of it moves to the shared global queue.
• The most recently woken task goes into a LIFO slot and runs next. When task A sends a message to task B and then waits, B runs immediately on the same core, with A's data still in the cache. To keep fairness, the LIFO slot is used at most three times in a row.
• A worker checks the global queue periodically, so tasks spawned from outside the runtime aren't starved.
• An idle worker steals half of another worker's local queue before going to sleep.

The consequence for your code: a task can move between threads at every .await. That's why spawned futures must be Send, and why holding a std::sync::MutexGuard across .await doesn't compile in tokio::spawn (Tokio Patterns).

12. The reactor: how I/O wakes tasks

Async I/O doesn't use a thread per connection. Tokio registers each socket with the operating system's readiness API (epoll on Linux, kqueue on macOS, IOCP on Windows, through the mio crate). When a read would block, the task stores its Waker and returns Pending. When the OS reports the socket as readable, the reactor calls wake, and the task goes back into a run queue.

use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};

#[tokio::main(flavor = "current_thread")]
async fn main() -> std::io::Result<()> {
    let listener = TcpListener::bind("127.0.0.1:0").await?;
    let addr = listener.local_addr()?;

    // One thread, three concurrent connections: each read that would block
    // parks its task in the reactor instead of blocking the thread.
    let server = tokio::spawn(async move {
        for _ in 0..3 {
            let (mut socket, _) = listener.accept().await.unwrap();
            tokio::spawn(async move {
                let mut buf = [0u8; 64];
                let n = socket.read(&mut buf).await.unwrap();
                socket.write_all(&buf[..n].to_ascii_uppercase()).await.unwrap();
            });
        }
    });

    let mut clients = Vec::new();
    for name in ["ada", "grace", "linus"] {
        clients.push(tokio::spawn(async move {
            let mut conn = TcpStream::connect(addr).await.unwrap();
            conn.write_all(name.as_bytes()).await.unwrap();
            let mut reply = String::new();
            conn.read_to_string(&mut reply).await.unwrap();
            reply
        }));
    }
    for c in clients {
        println!("{}", c.await.unwrap());
    }
    server.await.unwrap();
    Ok(())
}

ADA
GRACE
LINUS

The scheduler alternates between running tasks and polling the reactor: by default it checks for I/O and timer events every 61 task polls (the event_interval), and whenever it has nothing else to run.

13. Timers: a hierarchical timer wheel

tokio::time::sleep doesn't create a thread or an OS timer per call. Tokio keeps all timers in a hierarchical timer wheel with millisecond resolution, which makes creating and cancelling a timer O(1). Consequences: a sleep never ends early, it may end up to about a millisecond late (more under load), and a million pending timeouts are cheap.

use std::time::{Duration, Instant};
use tokio::time::{sleep, timeout};

#[tokio::main]
async fn main() {
    let start = Instant::now();
    sleep(Duration::from_millis(20)).await;
    println!("slept at least 20 ms: {}", start.elapsed() >= Duration::from_millis(20));

    // Ten thousand concurrent timers, one thread's worth of work.
    let start = Instant::now();
    let handles: Vec<_> = (0..10_000).map(|i| tokio::spawn(sleep(Duration::from_millis(10 + i % 10)))).collect();
    for h in handles {
        h.await.unwrap();
    }
    println!("10k timers done well under a second: {}", start.elapsed() < Duration::from_secs(1));

    let slow = timeout(Duration::from_millis(5), sleep(Duration::from_secs(10))).await;
    println!("timeout fired: {}", slow.is_err());
}

slept at least 20 ms: true
10k timers done well under a second: true
timeout fired: true

For tests, #[tokio::test(start_paused = true)] (with Tokio's test-util feature) freezes the clock and advances it instantly whenever all tasks are waiting on timers, so a test with an hour of simulated timeouts runs in milliseconds.

14. Cooperative scheduling: tasks must yield

Tokio cannot preempt a task: a task runs until it returns Pending. A task that never awaits anything that's pending holds its worker thread forever. To limit the damage, Tokio gives each task a budget of 128 operations per poll. Once it's spent, Tokio's own resources (channels, sockets, timers) return Pending even if they're ready, forcing the task to yield.

use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use tokio::sync::mpsc;

#[tokio::main(flavor = "current_thread")]
async fn main() {
    let (tx, mut rx) = mpsc::unbounded_channel();
    for i in 0..1_000u32 {
        tx.send(i).unwrap(); // 1000 messages ready: recv() never has to wait
    }
    drop(tx);

    let other_ran = Arc::new(AtomicBool::new(false));
    let flag = Arc::clone(&other_ran);
    tokio::spawn(async move { flag.store(true, Ordering::SeqCst) });

    let mut received = 0;
    let mut received_when_other_ran = None;
    while let Some(_msg) = rx.recv().await {
        received += 1;
        if received_when_other_ran.is_none() && other_ran.load(Ordering::SeqCst) {
            received_when_other_ran = Some(received);
        }
    }
    println!("received {received} messages");
    println!("the other task ran after {:?} messages", received_when_other_ran);
}

received 1000 messages
the other task ran after Some(129) messages

Even on a single thread, the other task got to run after the receiving loop had used up its budget of 128 operations. The budget only covers Tokio's own resources. A loop of pure computation, or a call to blocking code, has no .await points to interrupt, and it stalls every task on that worker. That's the most common async production problem, covered in detail in Async Pitfalls in Production.

15. Yielding by hand in long computations

When a task must do a long CPU-bound loop on the runtime, it can yield periodically with tokio::task::yield_now(). tokio::task::consume_budget() is the finer tool: it counts one unit against the budget and only yields when the budget is spent.

use std::time::Instant;
use tokio::task;

async fn checksum(data: &[u8]) -> u64 {
    let mut sum = 0u64;
    for chunk in data.chunks(4096) {
        sum = chunk.iter().fold(sum, |s, &b| s.wrapping_mul(31).wrapping_add(b as u64));
        task::consume_budget().await; // lets other tasks in every 128 chunks
    }
    sum
}

#[tokio::main(flavor = "current_thread")]
async fn main() {
    let data = vec![7u8; 8 * 1024 * 1024];
    let started = Instant::now();
    let ticker = tokio::spawn(async move {
        let mut ticks = 0;
        while started.elapsed().as_millis() < 1_000 && ticks < 3 {
            ticks += 1;
            task::yield_now().await;
        }
        ticks
    });
    let sum = checksum(&data).await;
    println!("checksum computed: {}", sum != 0);
    println!("ticker kept running meanwhile: {}", ticker.await.unwrap() == 3);
}

checksum computed: true
ticker kept running meanwhile: true

For anything longer than about 10–100 microseconds between .await points, move the work off the runtime with spawn_blocking or rayon (next example) instead.

16. The blocking pool

spawn_blocking runs a closure on a separate pool of threads, so blocking calls (file I/O, a synchronous database driver, heavy computation) don't stall the async workers. Its defaults matter in production:

use std::time::{Duration, Instant};

fn hash_password(pw: &str) -> u64 {
    std::thread::sleep(Duration::from_millis(50)); // stands in for a slow, blocking hash
    pw.bytes().fold(5381u64, |h, b| h.wrapping_mul(33) ^ b as u64)
}

#[tokio::main(worker_threads = 2)]
async fn main() {
    let start = Instant::now();
    let handles: Vec<_> = (0..8)
        .map(|i| tokio::task::spawn_blocking(move || hash_password(&format!("user-{i}"))))
        .collect();
    let mut hashes = Vec::new();
    for h in handles {
        hashes.push(h.await.unwrap());
    }
    // 8 blocking jobs ran on the blocking pool in parallel, not on the 2 workers.
    println!("{} hashes in under 400 ms: {}", hashes.len(), start.elapsed() < Duration::from_millis(400));
}

8 hashes in under 400 ms: true

tokio::fs uses the blocking pool internally, because most operating systems have no truly async file API. For CPU-heavy work, prefer a bounded pool (rayon, or a Semaphore in front of spawn_blocking): 512 threads all hashing passwords is not what you want.

17. One runtime, entered once

A runtime is meant to be created once per process (or once per thread for current-thread designs). Blocking on a runtime from inside async code panics, because it would block a worker thread that other tasks depend on:

#[tokio::main]
async fn main() {
    let inner = tokio::runtime::Runtime::new().unwrap();
    inner.block_on(async { println!("never reached") });
}

thread 'main' panicked at ~/.cargo/registry/src/.../tokio-1.53.1/src/runtime/scheduler/multi_thread/mod.rs:91:9:
Cannot start a runtime from within a runtime. This happens because a function (like `block_on`) attempted to block the current thread while the thread is being used to drive asynchronous tasks.

To call async code from synchronous code inside the runtime, use Handle::current().block_on inside spawn_blocking, or tokio::task::block_in_place (multi-thread runtime only). To call it from a plain thread outside the runtime, pass that thread a Handle and use handle.block_on.

18. Shutdown: dropping the runtime cancels tasks

When the runtime is dropped (at the end of #[tokio::main]), every task still running is cancelled at its next .await: its future is dropped, so its destructors run, but code after that .await doesn't. Blocking tasks can't be cancelled; the runtime waits for them, or gives up after shutdown_timeout.

use std::time::Duration;

struct Connection(&'static str);

impl Drop for Connection {
    fn drop(&mut self) {
        println!("closed {}", self.0);
    }
}

fn main() {
    let runtime = tokio::runtime::Runtime::new().unwrap();
    runtime.block_on(async {
        tokio::spawn(async {
            let _conn = Connection("background poller");
            loop {
                tokio::time::sleep(Duration::from_millis(10)).await;
            }
        });
        tokio::time::sleep(Duration::from_millis(30)).await;
        println!("main future done");
    });
    runtime.shutdown_timeout(Duration::from_secs(1));
    println!("runtime shut down");
}

main future done
closed background poller
runtime shut down

This is why graceful shutdown needs to be explicit: signal tasks (with a CancellationToken or a watch channel), then await their handles before main returns. Tokio Patterns shows a complete example.

19. Runtime metrics

Tokio exposes counters about its own state, the async equivalent of JMX thread-pool metrics. The stable ones cover workers, tasks, and the global queue; more (poll counts, steal counts, busy time per worker) are available with the tokio_unstable flag.

use std::time::Duration;
use tokio::runtime::Handle;

#[tokio::main(worker_threads = 2)]
async fn main() {
    let metrics = Handle::current().metrics();
    let tasks: Vec<_> = (0..5).map(|_| tokio::spawn(tokio::time::sleep(Duration::from_millis(50)))).collect();

    println!("workers: {}", metrics.num_workers());
    println!("alive tasks: {}", metrics.num_alive_tasks());
    for t in tasks {
        t.await.unwrap();
    }
    println!("alive tasks after join: {}", metrics.num_alive_tasks());
}

workers: 2
alive tasks: 5
alive tasks after join: 0

For live debugging, tokio-console shows every task, how long it has been polled, how often it was woken, and which tasks have been busy for too long without yielding. It requires building with --cfg tokio_unstable and the console-subscriber crate.

20. Tokio's concurrency model at a glance

The biggest mental shift: with Tokio, blocking inside a task is a bug and the type system can't always catch it. The next section starts with exactly that.

Summary

• Rust has no VM. Before main, the standard library checks the standard streams, ignores SIGPIPE, and installs a stack-overflow handler; that's the whole startup.
• main can return Result or ExitCode. process::exit skips destructors.
• Threads are OS threads with 2 MiB default stacks and guard pages. Stack overflow aborts; avoid unbounded recursion on untrusted input.
• Binaries are self-contained (static with musl), start in about a millisecond, and have no reflection: macros generate the needed code at compile time.
• Async needs an executor. Tokio's multi-thread runtime uses one worker per core, local queues with a LIFO slot, work stealing, a reactor over epoll/kqueue/IOCP, and a timer wheel.
• Tasks are scheduled cooperatively. Tokio's budget of 128 operations helps, but CPU-heavy loops and blocking calls must move to spawn_blocking, rayon, or be split with yield_now/consume_budget.
• Dropping the runtime cancels tasks at their next .await; blocking tasks can't be cancelled. Plan shutdown explicitly, and watch the runtime with metrics and tokio-console.

This completes the Under the Hood section. Next: Panics & Failure Handling, the start of the Production Rust section.