Guide contents

Lesson 89 of 94

Testing, Fuzzing & Benchmarking

Testing covered Rust's built-in test framework: #[test], assertions, fakes, integration tests, and doc tests. That's enough for most code. Senior engineers also own the testing strategy: which bugs example-based tests miss, how to find inputs nobody thought of, how to prove a change made something faster, and how to keep a large test suite fast and trustworthy.

This lesson covers the tools for that:

Goal Rust
Generate inputs and check properties proptest
Find crashes with random or mutated inputs cargo fuzz (libFuzzer), arbitrary
Microbenchmarks criterion, divan
Snapshot tests insta
Mocks mockall (or hand-written fakes)
Coverage cargo llvm-cov
Mutation testing cargo mutants
Faster, isolated test runs cargo nextest

Part 1: Property-based testing

1. A property instead of examples

An example-based test checks a handful of inputs you chose. A property says something that must hold for every input, and the framework generates hundreds of inputs to try to break it. proptest is the standard Rust library for this.

Here the property is that decoding an encoded string gives back the original:

Rustcargo test
/// Run-length encoding: "aaab" -> [('a', 3), ('b', 1)].
pub fn encode(s: &str) -> Vec<(char, usize)> {
    let mut out: Vec<(char, usize)> = Vec::new();
    for c in s.chars() {
        match out.last_mut() {
            Some((last, n)) if *last == c => *n += 1,
            _ => out.push((c, 1)),
        }
    }
    out
}

pub fn decode(runs: &[(char, usize)]) -> String {
    runs.iter().flat_map(|&(c, n)| std::iter::repeat_n(c, n)).collect()
}

fn main() {}

#[cfg(test)]
mod tests {
    use super::*;
    use proptest::prelude::*;

    #[test]
    fn example() {
        assert_eq!(encode("aaab"), vec![('a', 3), ('b', 1)]);
    }

    proptest! {
        #[test]
        fn decode_inverts_encode(s in ".*") {
            prop_assert_eq!(decode(&encode(&s)), s);
        }

        #[test]
        fn runs_never_repeat_a_char(s in "[abc]{0,50}") {
            let runs = encode(&s);
            for pair in runs.windows(2) {
                prop_assert_ne!(pair[0].0, pair[1].0);
            }
        }
    }
}
Output
running 3 tests
test tests::decode_inverts_encode ... ok
test tests::example ... ok
test tests::runs_never_repeat_a_char ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

Each proptest! test runs 256 generated cases by default. ".*" is a strategy: a regular expression that generates arbitrary strings, including empty ones and non-ASCII text. The second property uses a narrow alphabet on purpose, so that runs of repeated characters are common.

2. Shrinking: from a random failure to a minimal one

The real power of property testing is what happens when a property fails. proptest doesn't just report the random input that broke it: it shrinks that input step by step until it finds the simplest one that still fails. This helper, meant to shorten strings for log lines, has a bug that example tests with ASCII input never find:

Rustcargo test · tests fail
/// Shortens `s` to at most `max` bytes for a log line.
pub fn truncate(s: &str, max: usize) -> &str {
    if s.len() <= max { s } else { &s[..max] } // BUG: can cut a UTF-8 character in half
}

fn main() {}

#[cfg(test)]
mod tests {
    use super::*;
    use proptest::prelude::*;

    #[test]
    fn ascii_examples_pass() {
        assert_eq!(truncate("hello", 3), "hel");
        assert_eq!(truncate("hi", 10), "hi");
    }

    proptest! {
        #![proptest_config(ProptestConfig { failure_persistence: None, ..ProptestConfig::default() })]

        #[test]
        fn truncate_is_a_prefix_within_max(s in any::<String>(), max in 0usize..20) {
            let t = truncate(&s, max);
            prop_assert!(t.len() <= max);
            prop_assert!(s.starts_with(t));
        }
    }
}

The generated inputs are random, so the exact report differs between runs, but it always ends like this:

Text
test tests::truncate_is_a_prefix_within_max ... FAILED
...
Test failed: byte index 1 is not a char boundary; it is inside 'Σ' (bytes 0..2) of `Σ`.
minimal failing input: s = "Σ", max = 1

Proptest first found some long random string that panicked, then shrunk it to a single two-byte character and max = 1. That minimal case explains the bug by itself. The fix backs up to the nearest character boundary:

Rustcargo test
pub fn truncate(s: &str, max: usize) -> &str {
    if s.len() <= max {
        return s;
    }
    let mut end = max;
    while !s.is_char_boundary(end) {
        end -= 1; // back up to the start of the character
    }
    &s[..end]
}

fn main() {}

#[cfg(test)]
mod tests {
    use super::*;
    use proptest::prelude::*;

    proptest! {
        #[test]
        fn truncate_is_a_prefix_within_max(s in any::<String>(), max in 0usize..20) {
            let t = truncate(&s, max);
            prop_assert!(t.len() <= max);
            prop_assert!(s.starts_with(t));
        }
    }
}
Output
running 1 test
test tests::truncate_is_a_prefix_within_max ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s

By default, proptest saves failing inputs in a proptest-regressions/ directory next to the source file and replays them first on every run. Commit that directory: it turns every bug proptest ever found into a permanent regression test. (The example above turns persistence off only because it's a demonstration.)

3. Strategies: generating the inputs you need

A strategy describes a space of values. Proptest has strategies for primitives, ranges, regexes, collections, and combinators to build domain types from them.

Rustcargo test
use proptest::prelude::*;

#[derive(Debug, Clone, PartialEq)]
enum Currency {
    Eur,
    Usd,
    Inr,
}

#[derive(Debug, Clone)]
struct Order {
    sku: String,
    quantity: u32,
    unit_price_cents: u64,
    currency: Currency,
}

impl Order {
    fn total_cents(&self) -> Option<u64> {
        self.unit_price_cents.checked_mul(self.quantity as u64)
    }
}

fn currency() -> impl Strategy<Value = Currency> {
    prop_oneof![Just(Currency::Eur), Just(Currency::Usd), Just(Currency::Inr)]
}

fn order() -> impl Strategy<Value = Order> {
    ("[A-Z]{3}-[0-9]{4}", 1u32..1_000, 0u64..1_000_000, currency())
        .prop_map(|(sku, quantity, unit_price_cents, currency)| Order { sku, quantity, unit_price_cents, currency })
}

fn main() {}

proptest! {
    #[test]
    fn totals_never_overflow_for_realistic_orders(o in order()) {
        prop_assert!(o.total_cents().is_some());
        prop_assert_eq!(o.sku.len(), 8);
    }

    #[test]
    fn a_basket_total_is_the_sum_of_its_orders(orders in prop::collection::vec(order(), 0..20)) {
        let total: u64 = orders.iter().map(|o| o.total_cents().unwrap()).sum();
        let by_currency: u64 = [Currency::Eur, Currency::Usd, Currency::Inr]
            .iter()
            .map(|c| orders.iter().filter(|o| &o.currency == c).map(|o| o.total_cents().unwrap()).sum::<u64>())
            .sum();
        prop_assert_eq!(total, by_currency);
    }
}
Output
running 2 tests
test a_basket_total_is_the_sum_of_its_orders ... ok
test totals_never_overflow_for_realistic_orders ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s
Strategy Generates
any::<T>() any value of T (for types implementing Arbitrary)
0..100u32, 1.0..2.0f64 numbers in a range
"[a-z]{1,8}" strings matching a regex
prop::collection::vec(s, 0..20) vectors whose elements come from s (also hash_map, btree_set, ...)
(s1, s2, s3) tuples
s.prop_map(f) values transformed by f
s.prop_filter("why", pred) values satisfying pred (use sparingly: rejected values are wasted)
prop_oneof![a, b, c] a choice between strategies
Just(v) always v

With the proptest-derive crate, #[derive(Arbitrary)] generates a strategy for your own structs and enums.

4. Kinds of properties worth testing

Coming up with properties is the skill. A handful of patterns cover most code:

Pattern Property Example
Round trip decode(encode(x)) == x serialization, compression, parsing and printing
Invariant output always satisfies a rule sorted output, balanced tree, total never negative
Oracle / model fast implementation agrees with a simple one a custom cache vs HashMap, SIMD vs scalar code
Idempotence f(f(x)) == f(x) normalization, formatting, deduplication
Commutativity / order independence result doesn't depend on input order merging, aggregation, set operations
No crash the function returns for every input parsers, validators given untrusted input
Metamorphic a known change to the input causes a known change to the output adding an item raises the total by its price

5. Model-based testing: an oracle for a data structure

For data structures and stateful components, generate a sequence of operations, apply it both to your implementation and to a simple model known to be correct, and check they agree after every step. Here a small sorted-vector set is checked against BTreeSet:

Rustcargo test
use proptest::prelude::*;
use std::collections::BTreeSet;

/// A set stored as a sorted Vec: fast iteration, compact memory.
#[derive(Default)]
pub struct SortedVecSet {
    items: Vec<i32>,
}

impl SortedVecSet {
    pub fn insert(&mut self, x: i32) -> bool {
        match self.items.binary_search(&x) {
            Ok(_) => false,
            Err(pos) => {
                self.items.insert(pos, x);
                true
            }
        }
    }
    pub fn remove(&mut self, x: i32) -> bool {
        match self.items.binary_search(&x) {
            Ok(pos) => {
                self.items.remove(pos);
                true
            }
            Err(_) => false,
        }
    }
    pub fn contains(&self, x: i32) -> bool {
        self.items.binary_search(&x).is_ok()
    }
}

#[derive(Debug, Clone)]
enum Op {
    Insert(i32),
    Remove(i32),
    Contains(i32),
}

fn op() -> impl Strategy<Value = Op> {
    // Small values, so that inserts, removes, and lookups hit the same elements often.
    let v = -10i32..10;
    prop_oneof![v.clone().prop_map(Op::Insert), v.clone().prop_map(Op::Remove), v.prop_map(Op::Contains)]
}

fn main() {}

proptest! {
    #[test]
    fn behaves_like_btreeset(ops in prop::collection::vec(op(), 0..100)) {
        let mut real = SortedVecSet::default();
        let mut model = BTreeSet::new();
        for op in ops {
            match op {
                Op::Insert(x) => prop_assert_eq!(real.insert(x), model.insert(x)),
                Op::Remove(x) => prop_assert_eq!(real.remove(x), model.remove(&x)),
                Op::Contains(x) => prop_assert_eq!(real.contains(x), model.contains(&x)),
            }
            prop_assert!(real.items.iter().copied().eq(model.iter().copied()));
        }
    }
}
Output
running 1 test
test behaves_like_btreeset ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.04s

When such a test fails, shrinking reduces the operation list to the shortest sequence that reproduces the bug, which reads like a minimal bug report. The same technique scales up to caches, connection pools, and state machines.

6. Configuring runs

Rustcargo test
use proptest::prelude::*;

fn slugify(title: &str) -> String {
    let mut slug = String::new();
    for word in title.split(|c: char| !c.is_ascii_alphanumeric()).filter(|w| !w.is_empty()) {
        if !slug.is_empty() {
            slug.push('-');
        }
        slug.push_str(&word.to_ascii_lowercase());
    }
    slug
}

fn main() {}

proptest! {
    // More cases for an important property; the default is 256.
    #![proptest_config(ProptestConfig::with_cases(2_000))]

    #[test]
    fn slugify_is_idempotent(title in "\\PC{0,40}") {
        let once = slugify(&title);
        prop_assert_eq!(slugify(&once), once.clone());
        prop_assert!(once.chars().all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-'));
    }
}
Output
running 1 test
test slugify_is_idempotent ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.07s

\PC means "any printable Unicode character". The PROPTEST_CASES environment variable overrides the number of cases without changing code, which is handy for a nightly CI job that runs 100,000 cases.

Part 2: Fuzzing

A property test generates inputs from a strategy you wrote. A fuzzer starts from a few inputs (or none) and mutates bytes, keeping mutations that reach new code paths. Given minutes or hours, it finds inputs no human would write: malformed length fields, deeply nested structures, invalid UTF-8 in exactly the wrong place. Fuzzing is standard practice for anything that parses untrusted data, and essential for unsafe code.

cargo fuzz runs LLVM's libFuzzer. It needs a nightly toolchain because it compiles with sanitizer instrumentation.

Terminal
cargo install cargo-fuzz
cargo fuzz init                 # creates fuzz/ with its own Cargo.toml
cargo fuzz add parse_header     # creates fuzz/fuzz_targets/parse_header.rs

A fuzz target is a function that receives the fuzzer's bytes:

Rust
// fuzz/fuzz_targets/parse_header.rs
#![no_main]

use libfuzzer_sys::fuzz_target;

fuzz_target!(|data: &[u8]| {
    // Any panic, crash, or sanitizer error is a finding.
    let _ = my_crate::parse_header(data);
});

The function under test, a parser for a tiny binary format (a 1-byte version, a 2-byte length, then that many bytes of name), with a bug:

Rust
pub fn parse_header(data: &[u8]) -> Option<(u8, &str)> {
    let version = *data.first()?;
    let len = u16::from_be_bytes([*data.get(1)?, *data.get(2)?]) as usize;
    let name = &data[3..3 + len]; // BUG: trusts the length field
    std::str::from_utf8(name).ok().map(|n| (version, n))
}

fn main() {
    println!("{:?}", parse_header(&[1, 0, 3, b'a', b'b', b'c']));
}
Output
Some((1, "abc"))

8. Running the fuzzer and reproducing a crash

Terminal
cargo +nightly fuzz run parse_header -- -max_total_time=60

The fuzzer found the bug within its first few executions (the log lines are shortened here):

Text
#2	INITED cov: 24 ft: 25 corp: 1/1b exec/s: 0 rss: 32Mb
#4	NEW    cov: 25 ft: 26 corp: 2/3b lim: 4 exec/s: 0 rss: 32Mb L: 2/2 MS: 2 ChangeBit-CopyPart-

thread '<unnamed>' (11939) panicked at src/lib.rs:4:21:
range end index 2607 out of range for slice of length 3
...
SUMMARY: libFuzzer: deadly signal
────────────────────────────────────────────────────────────────────────────────
Failing input:
	fuzz/artifacts/parse_header/crash-9bf838bfcbb8144b5f8b9b93e1719bc35cbd789b
Output of `std::fmt::Debug`:
	[176, 10, 44]
Reproduce with:
	cargo fuzz run parse_header fuzz/artifacts/parse_header/crash-9bf838bfcbb8144b5f8b9b93e1719bc35cbd789b
Minimize test case with:
	cargo fuzz tmin parse_header fuzz/artifacts/parse_header/crash-9bf838bfcbb8144b5f8b9b93e1719bc35cbd789b
────────────────────────────────────────────────────────────────────────────────

The input [176, 10, 44] is three bytes whose length field claims 2,604 bytes of name (0x0A2C). cov is the number of code edges reached so far; NEW lines are inputs that reached new code and were added to the corpus. The workflow after a finding:

  1. Reproduce it with the saved artifact (the command above), and minimize it with cargo fuzz tmin.
  2. Turn it into an ordinary unit test, so the bug stays fixed without the fuzzer:
Rustcargo test
pub fn parse_header(data: &[u8]) -> Option<(u8, &str)> {
    let version = *data.first()?;
    let len = u16::from_be_bytes([*data.get(1)?, *data.get(2)?]) as usize;
    let name = data.get(3..3 + len)?; // fixed: a short buffer is None, not a panic
    std::str::from_utf8(name).ok().map(|n| (version, n))
}

fn main() {}

#[test]
fn fuzz_crash_length_field_longer_than_data() {
    assert_eq!(parse_header(&[176, 10, 44]), None);
}

#[test]
fn valid_header_still_parses() {
    assert_eq!(parse_header(&[1, 0, 3, b'a', b'b', b'c']), Some((1, "abc")));
}
Output
running 2 tests
test fuzz_crash_length_field_longer_than_data ... ok
test valid_header_still_parses ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
  1. Run the fuzzer again until it stops finding crashes, and keep the fuzz/corpus/ directory: it's the fuzzer's memory of interesting inputs.

cargo fuzz builds with AddressSanitizer by default, so in unsafe code it also reports out-of-bounds reads and use-after-free that wouldn't panic in a normal build.

9. Structured fuzzing with arbitrary

Raw bytes are ideal for parsers. For APIs that take structured values, the arbitrary crate turns fuzzer bytes into typed values, so the fuzzer explores meaningful inputs instead of mostly rejected ones:

Rust
#![no_main]

use arbitrary::Arbitrary;
use libfuzzer_sys::fuzz_target;

#[derive(Debug, Arbitrary)]
enum Op {
    Insert(i32),
    Remove(i32),
    Contains(i32),
}

fuzz_target!(|ops: Vec<Op>| {
    let mut real = my_crate::SortedVecSet::default();
    let mut model = std::collections::BTreeSet::new();
    for op in ops {
        match op {
            Op::Insert(x) => assert_eq!(real.insert(x), model.insert(x)),
            Op::Remove(x) => assert_eq!(real.remove(x), model.remove(&x)),
            Op::Contains(x) => assert_eq!(real.contains(x), model.contains(&x)),
        }
    }
});

This is the model-based test from example 5, driven by a coverage-guided fuzzer instead of random generation. The two approaches complement each other:

Property testing (proptest) Fuzzing (cargo fuzz)
Runs in every cargo test dedicated jobs (minutes to hours)
Input generation random from strategies you write mutation guided by code coverage
Finds logic bugs in stated properties crashes, panics, hangs, memory errors, sanitizer findings
Minimal failing case shrinking cargo fuzz tmin
Toolchain stable nightly
Best for business logic, data structures parsers, decoders, unsafe code, anything facing untrusted input

In CI, run each fuzz target for a bounded time (cargo fuzz run parse_header -- -max_total_time=300) on a schedule, and keep the corpus between runs so the fuzzer builds on previous coverage. Services such as OSS-Fuzz run fuzzers continuously for open-source projects.

Part 3: Benchmarking

10. Why timing loops lie

A benchmark written as "call it a million times in a loop and time it" is wrong more often than right. The optimizer may delete or precompute the work (How the Rust Compiler Works), the first iterations run with cold caches, CPU frequency changes during the run, and one measurement can't tell a real 5% improvement from noise. criterion warms up, runs many samples, applies statistics, and compares against earlier runs.

11. A criterion benchmark

Benchmarks live in benches/ and use their own harness:

TOMLCargo.toml
[dev-dependencies]
criterion = "0.7"

[[bench]]
name = "parsing"
harness = false

The code under test, in src/lib.rs:

Rust
/// Sums the numbers in a comma-separated line, allocating a String per field first.
pub fn sum_collect(line: &str) -> u64 {
    let parts: Vec<String> = line.split(',').map(|s| s.trim().to_string()).collect();
    parts.iter().map(|p| p.parse::<u64>().unwrap_or(0)).sum()
}

/// Same result, no allocation.
pub fn sum_streaming(line: &str) -> u64 {
    line.split(',').map(|s| s.trim().parse::<u64>().unwrap_or(0)).sum()
}

fn main() {
    let line = "1, 2, 3, 4";
    println!("{} {}", sum_collect(line), sum_streaming(line));
}
Output
10 10

The benchmark compares both at two input sizes:

Rust
// benches/parsing.rs
use criterion::{BenchmarkId, Criterion, criterion_group, criterion_main};
use my_crate::{sum_collect, sum_streaming};
use std::hint::black_box;

fn bench_parsing(c: &mut Criterion) {
    let mut group = c.benchmark_group("sum_line");
    for n in [10, 1_000] {
        let line: String = (0..n).map(|i| i.to_string()).collect::<Vec<_>>().join(", ");
        group.bench_with_input(BenchmarkId::new("collect", n), &line, |b, line| b.iter(|| sum_collect(black_box(line))));
        group.bench_with_input(BenchmarkId::new("streaming", n), &line, |b, line| b.iter(|| sum_streaming(black_box(line))));
    }
    group.finish();
}

criterion_group!(benches, bench_parsing);
criterion_main!(benches);

cargo bench --bench parsing prints (numbers are from one run on a cloud VM; yours will differ):

Text
sum_line/collect/10     time:   [449.63 ns 450.90 ns 452.19 ns]
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  2 (2.00%) high severe
sum_line/streaming/10   time:   [160.48 ns 161.16 ns 161.87 ns]
sum_line/collect/1000   time:   [55.493 µs 57.030 µs 58.848 µs]
sum_line/streaming/1000 time:   [21.141 µs 21.240 µs 21.343 µs]

The three numbers are a confidence interval: the lower bound, the best estimate, and the upper bound of the mean time per call. The streaming version is about 2.7 times faster at both sizes, which points at the per-field String allocations. HTML reports with plots are written to target/criterion/.

12. Baselines: proving a change helped

Criterion remembers the last run and reports the change automatically. For a before/after comparison across branches, save a named baseline:

Terminal
git switch main
cargo bench --bench parsing -- --save-baseline main
git switch my-optimization
cargo bench --bench parsing -- --baseline main

After changing sum_collect to collect &str slices instead of Strings, the comparison reports:

Text
sum_line/collect/1000   time:   [21.053 µs 21.096 µs 21.139 µs]
                        change: [−62.156% −61.975% −61.805%] (p = 0.00 < 0.05)
                        Performance has improved.

Now the other side of the story. Here is the same benchmark run twice against its own baseline without any code change, on a shared cloud machine:

Text
sum_line/collect/1000   time:   [54.968 µs 55.440 µs 56.025 µs]
                        change: [+6.8835% +12.137% +17.157%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 22 outliers among 100 measurements (22.00%)
  22 (22.00%) high severe

A "statistically significant 12% regression" from noise: another tenant on the same host was busy. The 22% severe outliers are the warning sign. Lessons:

  • Trust large, repeatable differences; re-run before believing small ones.
  • Run comparisons on a quiet, dedicated machine, with CPU frequency scaling fixed if you can.
  • In shared CI, prefer benchmarks that count instructions instead of time (Callgrind-based harnesses such as iai-callgrind), which are stable across noisy machines, and keep wall-clock benchmarks for manual investigation.

13. Benchmark design

Do Why
Wrap inputs and outputs in black_box stops constant folding and dead-code elimination
Build inputs outside b.iter(...), or use iter_batched measure the operation, not the setup
Benchmark several input sizes exposes accidental O(n²) behavior
Use realistic data (real log lines, real payloads) branch prediction and caches depend on data
Benchmark release builds only (cargo bench does) debug builds are 10–100× slower and distort ratios
Keep benchmarks in the repository and run them before merging performance work regressions creep in one "harmless" change at a time

For async code, criterion can drive a Tokio runtime with b.to_async(&runtime).iter(|| async { ... }) (the async_tokio feature). divan is a newer harness with a simpler API and built-in allocation counting.

Part 4: More tools for the test suite

14. Snapshot tests with insta

Some outputs are too big to assert field by field: a rendered report, a pretty-printed AST, a JSON response, an error message. A snapshot test stores the expected output and fails when it changes, showing a diff. With inline snapshots, the expected value lives in the test itself:

Rustcargo test
#[derive(Debug)]
struct Invoice {
    customer: String,
    lines: Vec<(String, u32, u64)>,
}

fn render(inv: &Invoice) -> String {
    let mut out = format!("INVOICE for {}\n", inv.customer);
    let mut total = 0;
    for (item, qty, cents) in &inv.lines {
        let line_total = *qty as u64 * cents;
        total += line_total;
        out.push_str(&format!("{item:<10} {qty:>3} x {:>7.2} = {:>8.2}\n", *cents as f64 / 100.0, line_total as f64 / 100.0));
    }
    out.push_str(&format!("TOTAL {:>29.2}\n", total as f64 / 100.0));
    out
}

fn main() {}

#[cfg(test)]
mod tests {
    use super::*;

    fn sample() -> Invoice {
        Invoice { customer: "ACME".into(), lines: vec![("widget".into(), 3, 250), ("gadget".into(), 1, 1999)] }
    }

    #[test]
    fn invoice_text() {
        insta::assert_snapshot!(render(&sample()), @r"
        INVOICE for ACME
        widget       3 x    2.50 =     7.50
        gadget       1 x   19.99 =    19.99
        TOTAL                         27.49
        ");
    }

    #[test]
    fn invoice_debug() {
        insta::assert_debug_snapshot!(sample().lines[0], @r#"
        (
            "widget",
            3,
            250,
        )
        "#);
    }
}
Output
running 2 tests
test tests::invoice_debug ... ok
test tests::invoice_text ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

The workflow is what makes it productive: when output changes on purpose, run cargo insta review (from cargo install cargo-insta), look at each diff, and accept it; insta rewrites the snapshot in the source file. Snapshots without @ are stored in snapshots/*.snap files next to the tests. Use assert_json_snapshot! or assert_yaml_snapshot! (with insta's json/yaml features) for API responses, and redactions for timestamps and ids that change every run.

15. Mocks with mockall

Rust code usually tests collaborators with hand-written fakes behind a trait (Testing shows this). When you need expectations (called exactly once, with these arguments, returning this), mockall generates a mock from the trait:

Rustcargo test
use mockall::automock;

#[derive(Debug, PartialEq)]
pub struct Rate(pub f64);

#[automock]
pub trait RateProvider {
    fn rate(&self, from: &str, to: &str) -> Option<Rate>;
}

pub fn convert(provider: &dyn RateProvider, cents: u64, from: &str, to: &str) -> Result<u64, String> {
    if from == to {
        return Ok(cents); // must not call the provider
    }
    let Rate(r) = provider.rate(from, to).ok_or_else(|| format!("no rate {from}->{to}"))?;
    Ok((cents as f64 * r).round() as u64)
}

fn main() {}

#[cfg(test)]
mod tests {
    use super::*;
    use mockall::predicate::eq;

    #[test]
    fn converts_with_the_provider_rate() {
        let mut provider = MockRateProvider::new();
        provider.expect_rate().with(eq("EUR"), eq("USD")).times(1).returning(|_, _| Some(Rate(1.1)));
        assert_eq!(convert(&provider, 1000, "EUR", "USD"), Ok(1100));
    }

    #[test]
    fn same_currency_skips_the_provider() {
        let mut provider = MockRateProvider::new();
        provider.expect_rate().never();
        assert_eq!(convert(&provider, 1000, "EUR", "EUR"), Ok(1000));
    }

    #[test]
    fn missing_rate_is_an_error() {
        let mut provider = MockRateProvider::new();
        provider.expect_rate().returning(|_, _| None);
        assert_eq!(convert(&provider, 1000, "EUR", "JPY"), Err("no rate EUR->JPY".to_string()));
    }
}
Output
running 3 tests
test tests::converts_with_the_provider_rate ... ok
test tests::missing_rate_is_an_error ... ok
test tests::same_currency_skips_the_provider ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Expectations are verified when the mock is dropped at the end of the test. Keep mocks at real boundaries (HTTP clients, payment providers, clocks): mocking everything couples tests to implementation details.

16. Coverage with cargo llvm-cov

cargo llvm-cov uses the compiler's own instrumentation (source-based coverage), so line and region coverage are exact, including for generics and macros.

Terminal
cargo install cargo-llvm-cov
cargo llvm-cov                      # summary table in the terminal
cargo llvm-cov --html --open        # browsable report
cargo llvm-cov --lcov --output-path lcov.info   # for Codecov, Coveralls, or SonarQube
cargo llvm-cov nextest              # with cargo-nextest as the runner

Coverage shows what's not tested; a high number doesn't show that the tests check anything. That's what the next tool is for.

17. Mutation testing with cargo mutants

cargo mutants changes your code in small ways (replaces a function body with Default::default(), flips < to <=, deletes a !), runs the tests for each change, and reports mutants that survived: changes no test noticed.

Terminal
cargo install cargo-mutants
cargo mutants                       # all mutants (slow on big crates)
cargo mutants --file src/pricing.rs # focus on one module
cargo mutants --in-diff pr.diff     # only code changed in a pull request

A surviving mutant such as replace apply_discount -> u64 with 0 means no test checks the discount's effect on a total. It's the best available measure of test quality rather than quantity.

18. cargo nextest: a faster, stricter test runner

cargo nextest runs each test in its own process, in parallel across all test binaries. That's faster for large workspaces, isolates tests that touch global state, and adds features the built-in runner lacks.

Terminal
cargo install cargo-nextest --locked
cargo nextest run                        # instead of cargo test
cargo nextest run --retries 2            # retry, and report tests as flaky if they pass on retry
cargo nextest run --partition count:1/4  # split the suite across 4 CI machines

Timeouts per test (slow-timeout in .config/nextest.toml) catch hanging tests, and XML test reports plug into CI dashboards. Doc tests still need cargo test --doc.

19. A test strategy for a Rust service

Layer Tool Runs Catches
Unit tests #[test], fakes, mockall every build logic errors in functions and types
Property tests proptest every build edge cases in parsers, algorithms, data structures
Snapshot tests insta every build unintended changes to rendered output, errors, API responses
Integration tests tests/, testcontainers for real databases every PR wiring, SQL, serialization, configuration
Concurrency tests loom, Miri, tokio::test(start_paused) every PR (Miri: nightly job) races, orderings, UB, timeouts
Fuzzing cargo fuzz scheduled jobs crashes on untrusted input
Benchmarks criterion, instruction-count harnesses performance PRs, nightly regressions in hot paths
Coverage and mutants cargo llvm-cov, cargo mutants weekly or on demand untested code, weak assertions
Load tests oha, k6, goose before releases throughput, tail latency, memory under load

Summary

  • Property-based tests state what must hold for all inputs; proptest generates inputs from strategies and shrinks failures to minimal cases. Commit proptest-regressions/.
  • Useful properties: round trips, invariants, model/oracle comparisons, idempotence, order independence, and "never crashes".
  • Fuzzing with cargo fuzz searches for crashing inputs guided by coverage; use it for parsers, untrusted input, and unsafe code, with arbitrary for structured inputs.
  • Benchmark with criterion: black_box, several input sizes, saved baselines. Distrust small differences on noisy machines; prefer instruction counts in CI.
  • insta snapshot tests cover large outputs with a fast review workflow; mockall provides expectations when hand-written fakes aren't enough.
  • cargo llvm-cov shows what isn't tested, cargo mutants shows what isn't checked, and cargo nextest makes big suites faster and flaky tests visible.

Next: Compile Times & Build Engineering: why Rust builds are slow, how to measure them, and how to make them fast.