Blocking and CPU-Bound Work

Async tasks share a small number of threads. A task gives its thread back only when it reaches an .await that is not ready. Code that runs for a long time without an .await keeps the thread, and every other task on that thread waits. This lesson shows that effect with printed orderings, then covers the tools that move such work off the async threads.

Terms used below:

• Blocking: a call that keeps its thread busy or waiting and does not return to the runtime. std::thread::sleep, a std::sync::mpsc::Receiver::recv, a synchronous file read, and a long loop are all blocking.
• Worker thread: a runtime thread that polls tasks. A current_thread runtime has one; a multi-thread runtime has several (Your First Tokio Program shows both).
• Blocking pool: a separate set of threads, created on demand, where blocking code is allowed to run.

Example 1: A blocking task stalls its neighbour

Two tasks run on a current_thread runtime, so both are polled by the same thread. Task A waits up to 200 ms for a message that task B sends. A uses the standard library channel, which blocks the thread while it waits.

use std::sync::mpsc;
use std::time::Duration;

#[tokio::main(flavor = "current_thread")]
async fn main() {
    let (tx, rx) = mpsc::channel::<()>();

    let a = tokio::spawn(async move {
        println!("A: start");
        let got = rx.recv_timeout(Duration::from_millis(200)).is_ok();
        println!("A: signal received = {got}");
    });
    let b = tokio::spawn(async move {
        println!("B: running");
        let _ = tx.send(());
    });

    a.await.unwrap();
    b.await.unwrap();
}

A: start
A: signal received = false
B: running

A started first and never reached an .await, so the only worker thread stayed inside recv_timeout for the full 200 ms. B did not run until A finished, which is why the message never arrived in time. The result does not depend on the length of the wait: while A holds the thread, B cannot be polled at all.

Example 2: The cooperative version interleaves

The next program has the same two-task shape, but A calls tokio::task::yield_now().await between its two steps. yield_now returns the thread to the runtime and puts the task at the back of the queue (Spawning Tasks introduces it).

#[tokio::main(flavor = "current_thread")]
async fn main() {
    let a = tokio::spawn(async {
        println!("A: step 1");
        tokio::task::yield_now().await;
        println!("A: step 2");
    });
    let b = tokio::spawn(async {
        println!("B: runs between the steps");
    });

    a.await.unwrap();
    b.await.unwrap();
}

A: step 1
B: runs between the steps
A: step 2

The runtime polled A, A yielded, the runtime polled B to completion, and then it polled A again. Every .await on something that is not ready is such a switch point. Blocking code has no switch points.

Example 3: spawn_blocking moves the wait off the worker

tokio::task::spawn_blocking takes a closure, runs it on a thread of the blocking pool, and returns a JoinHandle that can be awaited like the handle of an ordinary task. The runtime thread stays free to poll other tasks. Here A is Example 1 rewritten with spawn_blocking.

use std::sync::mpsc;
use std::time::Duration;

#[tokio::main(flavor = "current_thread")]
async fn main() {
    let (tx, rx) = mpsc::channel::<()>();

    let a = tokio::task::spawn_blocking(move || {
        let got = rx.recv_timeout(Duration::from_secs(5)).is_ok();
        println!("A: signal received = {got}");
    });
    let b = tokio::spawn(async move {
        println!("B: running");
        let _ = tx.send(());
    });

    a.await.unwrap();
    b.await.unwrap();
}

B: running
A: signal received = true

A waits on a blocking-pool thread, B runs on the worker thread and sends the message, and A receives it. The 5-second timeout is only an upper bound; the program finishes as soon as B has sent. The closure passed to spawn_blocking must be 'static and Send, and it cannot use .await, for the same reasons as a closure passed to std::thread::spawn.

Example 4: Results and panics come back through the handle

The JoinHandle from spawn_blocking resolves to Result<T, JoinError>. T is the closure's return value. If the closure panics, the error reports it and the runtime keeps running.

use tokio::task::spawn_blocking;

fn slow_square(n: u64) -> u64 {
    // stands in for a blocking or CPU-heavy call
    n * n
}

#[tokio::main]
async fn main() {
    let value = spawn_blocking(|| slow_square(12)).await.unwrap();
    println!("value = {value}");

    let failed = spawn_blocking(|| -> u64 { panic!("bad input") }).await;
    match failed {
        Ok(_) => println!("no error"),
        Err(e) => println!("panicked = {}", e.is_panic()),
    }
}

value = 144
panicked = true

The panic message goes to standard error, so it does not appear in the output above. JoinError::is_panic is explained in Cancellation, abort, and Panics.

Example 5: The blocking pool has a thread limit

Tokio creates blocking-pool threads when spawn_blocking is called and stops creating them at a limit. Extra jobs wait in a queue. The source sets the default limit to 512 threads (max_blocking_threads: 512 in runtime/builder.rs). An idle blocking thread exits after 10 seconds by default (KEEP_ALIVE in runtime/blocking/pool.rs).

The program below builds a runtime with Builder::new_current_thread().max_blocking_threads(1). Builder configures a runtime before it starts; Runtime Flavors and the Builder covers every option. With one blocking thread, job 2 cannot start until job 1 returns, so job 1 waits for a message that only job 2 can send and gives up after 300 ms.

use std::sync::mpsc;
use std::time::Duration;

fn main() {
    let rt = tokio::runtime::Builder::new_current_thread()
        .max_blocking_threads(1)
        .build()
        .unwrap();

    rt.block_on(async {
        let (tx, rx) = mpsc::channel::<()>();

        let job1 = tokio::task::spawn_blocking(move || {
            println!("job 1 started");
            let got = rx.recv_timeout(Duration::from_millis(300)).is_ok();
            println!("job 1 saw signal = {got}");
        });
        let job2 = tokio::task::spawn_blocking(move || {
            println!("job 2 started");
            let _ = tx.send(());
        });

        job1.await.unwrap();
        job2.await.unwrap();
    });
}

job 1 started
job 1 saw signal = false
job 2 started

Blocking jobs that wait on each other can deadlock a full pool. CPU-heavy jobs need a bound too: spawn_blocking would start up to 512 threads, far more than the number of cores. The spawn_blocking documentation recommends a semaphore or a dedicated CPU pool such as rayon for many CPU-bound jobs (Semaphore, Notify, Barrier, and OnceCell covers Semaphore).

Example 6: abort cannot stop a running blocking job

JoinHandle::abort cancels an async task at its next .await. A blocking closure has no .await, so abort has no effect once the closure has started. A job still waiting in the queue can be cancelled.

The program keeps the pool at one thread. The first job starts and waits for a release message. The second job is queued behind it. The main task aborts both handles, then releases the first job.

use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{mpsc, Arc};
use std::time::Duration;

fn main() {
    let rt = tokio::runtime::Builder::new_current_thread()
        .max_blocking_threads(1)
        .build()
        .unwrap();

    rt.block_on(async {
        let (release_tx, release_rx) = mpsc::channel::<()>();
        let started = Arc::new(AtomicBool::new(false));
        let started_in_job = started.clone();

        let running = tokio::task::spawn_blocking(move || {
            started_in_job.store(true, Ordering::SeqCst);
            release_rx.recv_timeout(Duration::from_secs(5)).unwrap();
            "running job finished"
        });
        let queued = tokio::task::spawn_blocking(|| "queued job finished");

        while !started.load(Ordering::SeqCst) {
            tokio::task::yield_now().await;
        }

        running.abort();
        queued.abort();
        release_tx.send(()).unwrap();

        match running.await {
            Ok(text) => println!("{text}"),
            Err(e) => println!("running: cancelled = {}", e.is_cancelled()),
        }
        match queued.await {
            Ok(text) => println!("{text}"),
            Err(e) => println!("queued: cancelled = {}", e.is_cancelled()),
        }
    });
}

running job finished
queued: cancelled = true

The running job kept going and returned its value. The queued job never started and reports is_cancelled() == true. AtomicBool is a shared flag that several places can read and write without a lock; the loop waits for the job to set it. The documentation of spawn_blocking states the same rule: abort takes effect only if the task has not started. It also says that dropping a runtime waits for started blocking jobs, and Runtime::shutdown_timeout stops the wait without stopping the jobs.

Example 7: block_in_place on the multi-thread runtime

tokio::task::block_in_place runs a closure on the current worker thread and allows blocking inside it. Before the closure runs, the runtime hands the worker's other queued tasks to another worker, so they are not stalled. It works only on the multi-thread runtime.

The program is Example 1 on two workers. A blocks inside block_in_place, and B still runs. The first print of Example 1 is removed because, with two workers, the order of the two task starts is not fixed.

use std::sync::mpsc;
use std::time::Duration;

#[tokio::main(flavor = "multi_thread", worker_threads = 2)]
async fn main() {
    let (tx, rx) = mpsc::channel::<()>();

    let a = tokio::spawn(async move {
        let got = tokio::task::block_in_place(|| {
            rx.recv_timeout(Duration::from_secs(5)).is_ok()
        });
        println!("A: signal received = {got}");
    });
    let b = tokio::spawn(async move {
        println!("B: running");
        let _ = tx.send(());
    });

    a.await.unwrap();
    b.await.unwrap();
}

B: running
A: signal received = true

The closure passed to block_in_place runs on the worker thread itself, so it can borrow local variables and does not need to be 'static. The task that calls it is suspended for the duration. Other code in the same task, for example the other branches of a join!, does not make progress during the call. spawn_blocking avoids that, because the calling task awaits a handle instead of blocking. Code inside block_in_place cannot be cancelled.

Example 8: block_in_place panics on a current-thread runtime

A current_thread runtime has no second worker to receive the queued tasks. The call panics instead of stalling silently.

#[tokio::main(flavor = "current_thread")]
async fn main() {
    tokio::task::block_in_place(|| {});
}

thread 'main' panicked at src/main.rs:3:5:
can call blocking only when running on the multi-threaded runtime

Example 9: Handle::block_on runs a future from blocking code

Blocking code sometimes needs to call an async fn. tokio::runtime::Handle is a cloneable reference to a runtime. Handle::current() returns the handle of the runtime the caller is in. Handle::block_on(future) runs the future to completion on the calling thread and returns its output.

use std::time::Duration;
use tokio::runtime::Handle;

async fn fetch_answer() -> u32 {
    tokio::time::sleep(Duration::from_millis(5)).await;
    42
}

#[tokio::main]
async fn main() {
    let handle = Handle::current();

    let answer = tokio::task::spawn_blocking(move || {
        // This closure runs on a blocking-pool thread, not a worker.
        handle.block_on(fetch_answer())
    })
    .await
    .unwrap();

    println!("answer = {answer}");
}

answer = 42

The same call works from a thread created with std::thread::spawn that received a cloned Handle.

Example 10: block_on inside async code panics

Handle::block_on blocks the calling thread until the future finishes. A worker thread that is polling tasks cannot do that, and Tokio rejects the call.

use tokio::runtime::Handle;

#[tokio::main(flavor = "current_thread")]
async fn main() {
    let handle = Handle::current();
    handle.block_on(async { println!("never printed") });
}

thread 'main' panicked at src/main.rs:6:12:
Cannot start a runtime from within a runtime. This happens because a function (like `block_on`) attempted to block the current thread while the thread is being used to drive asynchronous tasks.

Inside async code, use .await. Use block_on only on threads that are not runtime workers: the main thread of a program that owns a runtime, a std::thread, or a spawn_blocking closure. Inside block_in_place, Handle::current().block_on(...) is also allowed (the documentation shows this pattern).

Example 11: A dedicated thread for long-lived work

A blocking-pool thread is occupied for as long as the closure runs. The spawn_blocking documentation describes the pool as intended for bounded work that finishes on its own and recommends std::thread::spawn for background workers and loops that run for the life of the program.

An async task can wait for a std::thread result without blocking a worker by wrapping join in spawn_blocking:

use std::thread;

#[tokio::main]
async fn main() {
    let worker = thread::spawn(|| {
        // long-running or blocking work owned by a dedicated thread
        (1..=10u64).product::<u64>()
    });

    let result = tokio::task::spawn_blocking(move || worker.join().unwrap())
        .await
        .unwrap();

    println!("10! = {result}");
}

10! = 3628800

JoinHandle::join of the standard library blocks, so it belongs on a blocking thread. Channels that connect a dedicated thread to async code are covered in oneshot, broadcast, and watch and mpsc Channels.

Example 12: rayon for data-parallel CPU work

rayon is a crate with its own thread pool sized to the number of cores. Its parallel iterators split a computation across that pool. A rayon call blocks the calling thread until the result is ready, so an async program calls it inside spawn_blocking. This keeps CPU-bound work off the workers and limits it to the size of rayon's pool.

use rayon::prelude::*;

fn sum_of_squares(limit: u64) -> u64 {
    (1..=limit).into_par_iter().map(|n| n * n).sum()
}

#[tokio::main]
async fn main() {
    let total = tokio::task::spawn_blocking(|| sum_of_squares(1_000))
        .await
        .unwrap();
    println!("sum of squares = {total}");
}

sum of squares = 333833500

The Tokio documentation suggests sending the result back with a oneshot channel when a rayon task is started with rayon::spawn. Wrapping the blocking rayon call in spawn_blocking, as here, needs no channel, at the cost of one blocking-pool thread that waits while rayon works.

Example 13: Splitting a long loop with yield_now

Sometimes the work must stay in async code, for example because it uses async state. A long loop can hand the thread back between chunks with yield_now().await. The program below runs a worker that processes three chunks and a second task that prints one line. The single worker thread runs the second task after the first chunk, when the worker yields.

use tokio::task::yield_now;

#[tokio::main(flavor = "current_thread")]
async fn main() {
    let worker = tokio::spawn(async {
        let data: Vec<u32> = (1..=6).collect();
        let mut total = 0;
        for (i, chunk) in data.chunks(2).enumerate() {
            total += chunk.iter().sum::<u32>();
            println!("worker: chunk {i} done, total {total}");
            yield_now().await;
        }
        total
    });
    let ticker = tokio::spawn(async {
        println!("ticker: got a turn");
    });

    println!("result = {}", worker.await.unwrap());
    ticker.await.unwrap();
}

worker: chunk 0 done, total 3
ticker: got a turn
worker: chunk 1 done, total 10
worker: chunk 2 done, total 21
result = 21

Without the yield_now calls the worker would print all three chunks before the ticker ran, as in Example 1. Each yield costs one trip through the scheduler, so the yield is placed between chunks of work, not between single items. For a chunk that takes long even by itself, or for work that needs no async state, spawn_blocking is the better fit.

Example 14: The cooperative budget

Tokio also forces some yields itself. Each task gets a budget of 128 units each time it is polled (Budget::initial() in task/coop/mod.rs returns Some(128)). Operations on Tokio resources spend one unit when they make progress. The source calls poll_proceed in channel receives (sync/mpsc/chan.rs), semaphore acquires (sync/batch_semaphore.rs), oneshot (sync/oneshot.rs), awaiting a JoinHandle (runtime/task/join.rs), I/O readiness (runtime/io/registration.rs), and Sleep (time/sleep.rs). When the budget reaches zero, the next such operation returns Pending even though it could have completed, and the task is woken again and put at the back of the queue. This stops one task that always finds data ready from starving the others.

Plain Rust code spends no budget. tokio::task::consume_budget().await spends one unit and yields only when the budget is exhausted, so it can be placed in a loop that uses no Tokio resource. The program counts how many iterations the loop finishes before another task gets to run.

use std::sync::atomic::{AtomicU32, Ordering};
use std::sync::Arc;

#[tokio::main(flavor = "current_thread")]
async fn main() {
    let iterations = Arc::new(AtomicU32::new(0));
    let counter = iterations.clone();

    let looper = tokio::spawn(async move {
        for _ in 0..300 {
            counter.fetch_add(1, Ordering::SeqCst);
            tokio::task::consume_budget().await;
        }
    });
    let observer = tokio::spawn(async move {
        println!("observer ran after {} iterations", iterations.load(Ordering::SeqCst));
    });

    looper.await.unwrap();
    observer.await.unwrap();
}

observer ran after 129 iterations

The loop counted 129 iterations, not 128: the first 128 calls to consume_budget spent the whole budget, and the 129th call, made after the 129th increment, was the one that yielded. AtomicU32 is a shared integer that can be updated without a lock; Arc shares it between the two tasks. The budget is a safety net. It cannot help code that never touches a Tokio resource and never calls consume_budget, such as Example 1.

Example 15: Recognizing accidental blocking

Blocking calls look like ordinary function calls, and nothing in the type system marks them. This table lists common sources and the async-friendly replacement.

Signs of accidental blocking: unrelated tasks pause at the same moment, timers fire late, and throughput falls when load rises although the CPU is not saturated. The remedy is to find the call without an .await in the busy task and move it to one of the tools in this lesson.

Some blocking-style methods exist for use on blocking threads only, for example Mutex::blocking_lock, mpsc::Sender::blocking_send, and mpsc::Receiver::blocking_recv. They wait synchronously and are meant to be called from spawn_blocking closures or plain threads, not from async code. The lessons on Shared State: Mutex and RwLock and mpsc Channels cover them.

Choosing a tool

Interview points

• A task gives up its thread only at an .await that returns Pending. Blocking code and long loops without .await hold the worker, so other tasks on it wait.
• On a current_thread runtime one blocked task stops every other task. On a multi-thread runtime it removes one worker, and the stall becomes total when the number of blocked tasks reaches the number of workers.
• spawn_blocking runs a 'static + Send closure on the blocking pool (up to 512 threads by default, 10 seconds keep-alive) and returns a JoinHandle.
• abort does not stop a blocking closure that has started; it can cancel one still in the queue.
• block_in_place hands the worker's queued tasks to another worker and blocks in place. It panics on current_thread and suspends the rest of the calling task.
• Handle::block_on panics when called on a runtime worker; call it from spawn_blocking, block_in_place, or a plain thread.
• Long-lived blocking work fits a dedicated std::thread, because it would hold a pool thread for its whole life.
• rayon is entered through spawn_blocking; it bounds CPU parallelism by its own pool size.
• The cooperative budget is 128 units per poll; Tokio resource operations spend it, and plain code does not.

Summary

• Blocking means holding a thread without returning to the runtime. It delays every task on that thread.
• Example 1 and Example 2 show the difference: blocking gives A then B with the message missed, cooperative code gives A, B, A.
• spawn_blocking(f) returns JoinHandle<T>; panics arrive as JoinError with is_panic().
• Builder::max_blocking_threads limits the pool; extra jobs queue.
• block_in_place works on the multi-thread runtime only.
• Handle::current() and Handle::block_on connect blocking threads to async code.
• yield_now returns the thread between chunks; consume_budget yields only when the 128-unit budget is spent.