Lesson 52 of 94
Reading & Writing Files
File I/O in Rust is split between two modules: std::fs works with whole files and the filesystem, and std::io defines the traits for reading and writing streams of bytes (Read, BufRead, Write, Seek). Every operation that can fail returns io::Result<T>, which is Result<T, io::Error>. The error handling from the previous section applies directly, and ? does the propagating.
| Task | Rust |
|---|---|
| Read a whole file as text | fs::read_to_string(path) |
| Write text to a file | fs::write(path, text) |
| Read a whole file as bytes | fs::read(path) |
| Read line by line | BufReader::new(file).lines() or read_line |
| Write formatted output | write! / writeln! / write_all |
| Buffered writing | BufWriter |
| Append to a file | OpenOptions::new().append(true) |
| Close a file | automatic: a file closes when it goes out of scope |
| Error type | io::Error |
| File not found | io::ErrorKind::NotFound |
| Byte streams | the Read / Write traits |
| Seeking | the Seek trait |
The examples create files with relative names such as notes.txt. Run with cargo run, they appear in the project directory, next to Cargo.toml.
Part 1: Whole files
1. Writing and reading a whole file
fs::write creates the file, or replaces its contents if it exists. fs::read_to_string reads it back into a String. main returns io::Result<()>, so ? can pass any error up, and the program exits with an error message if one happens.
use std::fs;
use std::io;
fn main() -> io::Result<()> {
fs::write("greeting.txt", "Hello from Rust!\nSecond line\n")?;
let text = fs::read_to_string("greeting.txt")?;
print!("{text}");
println!("{} bytes, {} lines", text.len(), text.lines().count());
Ok(())
}Hello from Rust!
Second line
29 bytes, 2 linesBoth functions accept anything path-like: a &str, a String, or a Path (the next lesson covers paths).
2. When the file isn't there
A failed read returns an io::Error. Its kind() tells you what went wrong, so you can handle one case, such as a missing file, and pass the rest on. It's an ordinary value.
use std::fs;
use std::io::{self, ErrorKind};
fn load_notes(path: &str) -> io::Result<String> {
match fs::read_to_string(path) {
Ok(text) => Ok(text),
Err(e) if e.kind() == ErrorKind::NotFound => Ok(String::from("(no notes yet)")),
Err(e) => Err(e),
}
}
fn main() -> io::Result<()> {
println!("{:?}", load_notes("missing.txt")?);
fs::write("notes.txt", "buy milk")?;
println!("{:?}", load_notes("notes.txt")?);
let err = fs::read_to_string("missing.txt").unwrap_err();
println!("kind: {:?}", err.kind());
println!("message: {err}");
Ok(())
}"(no notes yet)"
"buy milk"
kind: NotFound
message: No such file or directory (os error 2)Other common kinds are PermissionDenied, AlreadyExists, InvalidData, and UnexpectedEof. Prefer trying the operation and handling the error over checking exists() first: another program can delete or create the file between your check and your read.
3. Error messages that name the file
Look at the message above: No such file or directory (os error 2). It doesn't say which file. In an application with many files, add that yourself. With anyhow (from thiserror & anyhow), with_context wraps the error with a message:
use anyhow::{Context, Result};
use std::fs;
fn read_config(path: &str) -> Result<String> {
fs::read_to_string(path).with_context(|| format!("could not read config file {path:?}"))
}
fn main() -> Result<()> {
let config = read_config("settings.toml")?;
println!("{config}");
Ok(())
}Error: could not read config file "settings.toml"
Caused by:
No such file or directory (os error 2)In a library, give your error enum a variant that stores the path next to the io::Error source, as Custom Error Types showed.
4. Bytes instead of text
fs::read returns the raw bytes as a Vec<u8>, and fs::write accepts bytes as well as text. Here, slice patterns recognize file formats by their first bytes (their "magic number").
use std::fs;
use std::io;
fn file_type(bytes: &[u8]) -> &'static str {
match bytes {
[0x89, b'P', b'N', b'G', ..] => "PNG image",
[b'%', b'P', b'D', b'F', ..] => "PDF document",
[b'P', b'K', 3, 4, ..] => "ZIP archive (also .jar and .docx)",
_ => "unknown",
}
}
fn main() -> io::Result<()> {
fs::write("logo.png", [0x89, b'P', b'N', b'G', 0x0D, 0x0A, 0x1A, 0x0A])?;
fs::write("report.pdf", b"%PDF-1.7 ...")?;
fs::write("data.bin", vec![0u8; 16])?;
for name in ["logo.png", "report.pdf", "data.bin"] {
let bytes = fs::read(name)?;
println!("{name:<10} {:>2} bytes {}", bytes.len(), file_type(&bytes));
}
Ok(())
}logo.png 8 bytes PNG image
report.pdf 12 bytes PDF document
data.bin 16 bytes unknown5. Text that isn't valid UTF-8
A Rust String is always valid UTF-8, so read_to_string refuses a file in another encoding instead of silently producing garbage. String::from_utf8_lossy converts anyway, replacing invalid bytes with �.
use std::fs;
use std::io;
fn main() -> io::Result<()> {
fs::write("latin1.txt", [b'c', b'a', b'f', 0xE9, b'\n'])?; // "café" in ISO-8859-1, not UTF-8
match fs::read_to_string("latin1.txt") {
Ok(text) => println!("text: {text}"),
Err(e) => println!("read_to_string failed: {e} ({:?})", e.kind()),
}
let bytes = fs::read("latin1.txt")?;
println!("lossy: {}", String::from_utf8_lossy(&bytes).trim_end());
Ok(())
}read_to_string failed: stream did not contain valid UTF-8 (InvalidData)
lossy: caf�To read other encodings properly, decode the bytes with a crate such as encoding_rs.
Part 2: Readers and writers
6. Reading line by line
Reading a whole file is fine for small files. For large ones, wrap the File in a BufReader, which reads big chunks from disk and hands out one line at a time. Memory use stays constant however big the file is. lines() needs the BufRead trait in scope, and it removes the \n (or \r\n) from each line.
use std::fs::{self, File};
use std::io::{self, BufRead, BufReader};
fn main() -> io::Result<()> {
fs::write("app.log", "INFO started\nWARN disk 81%\nINFO GET /orders\nERROR db timeout\nINFO GET /health\nERROR db timeout\n")?;
let reader = BufReader::new(File::open("app.log")?);
let (mut info, mut warn, mut error) = (0, 0, 0);
for (index, line) in reader.lines().enumerate() {
let line = line?; // each line can fail, for example on invalid UTF-8
if line.starts_with("ERROR") {
error += 1;
println!("line {}: {line}", index + 1);
} else if line.starts_with("WARN") {
warn += 1;
} else {
info += 1;
}
}
println!("info={info} warn={warn} error={error}");
Ok(())
}line 4: ERROR db timeout
line 6: ERROR db timeout
info=3 warn=1 error=27. Reusing one buffer with read_line
lines() allocates a new String for every line. read_line appends the next line to a buffer you own, so one allocation serves the whole file. It returns the number of bytes read, and 0 at the end of the file. The line keeps its \n, and the buffer must be cleared before the next call.
use std::fs::{self, File};
use std::io::{self, BufRead, BufReader};
fn main() -> io::Result<()> {
fs::write("numbers.txt", "10\n20\n 30 \nx\n40\n")?;
let mut reader = BufReader::new(File::open("numbers.txt")?);
let mut line = String::new();
let (mut sum, mut bad) = (0, 0);
while reader.read_line(&mut line)? > 0 {
match line.trim().parse::<i64>() {
Ok(n) => sum += n,
Err(_) => bad += 1,
}
line.clear(); // read_line appends, so empty the buffer each time
}
println!("sum={sum}, unparsable lines={bad}");
Ok(())
}sum=100, unparsable lines=18. Writing with the Write trait, and automatic closing
File::create opens a file for writing, creating it or emptying it. The write! and writeln! macros work like format! but write into anything that implements Write; write_all writes raw bytes. There's no close(): a file is closed when its variable goes out of scope.
use std::fs::{self, File};
use std::io::{self, Write};
fn main() -> io::Result<()> {
{
let mut file = File::create("invoice.txt")?;
writeln!(file, "INVOICE #{}", 1042)?;
for (item, cents) in [("keyboard", 4_999), ("mouse", 1_950)] {
writeln!(file, "{item:<10}{:>8.2}", cents as f64 / 100.0)?;
}
file.write_all(b"-- thank you --\n")?;
} // `file` goes out of scope here, which closes it
print!("{}", fs::read_to_string("invoice.txt")?);
Ok(())
}INVOICE #1042
keyboard 49.99
mouse 19.50
-- thank you --The methods come from the Write trait, so it must be imported. Forgetting is a common first error, and the compiler says exactly what's missing:
use std::fs::File;
fn main() -> std::io::Result<()> {
let mut file = File::create("hello.txt")?;
file.write_all(b"hello")?;
Ok(())
}error[E0599]: no method named `write_all` found for struct `File` in the current scope
--> src/main.rs:5:10
|
5 | file.write_all(b"hello")?;
| ^^^^^^^^^
|
--> /rustc/<version>/library/std/src/io/mod.rs:1875:7
|
= note: the method is available for `File` here
|
= help: items from traits can only be used if the trait is in scope
help: there is a method `write_all_at` with a similar name, but with different arguments
--> /rustc/<version>/library/std/src/os/unix/fs.rs:330:4
help: trait `Write` which provides `write_all` is implemented but not in scope; perhaps you want to import it
|
1 + use std::io::Write;
|9. BufWriter for many small writes
Every write on a File is a separate request to the operating system. BufWriter collects writes in memory and passes them on in large chunks; for many small writes, the difference is large. Call flush() when you're done: dropping a BufWriter flushes too, but it has no way to report an error at that point, so a failed write would go unnoticed.
use std::fs::{self, File};
use std::io::{self, BufWriter, Write};
fn main() -> io::Result<()> {
let mut out = BufWriter::new(File::create("squares.csv")?);
writeln!(out, "n,square")?;
for n in 1..=10_000u64 {
writeln!(out, "{n},{}", n * n)?;
}
out.flush()?; // report write errors instead of losing them
let text = fs::read_to_string("squares.csv")?;
println!("{} lines, {} bytes", text.lines().count(), text.len());
println!("last line: {}", text.lines().last().unwrap_or(""));
Ok(())
}10001 lines, 134285 bytes
last line: 10000,10000000010. Appending, and refusing to overwrite
OpenOptions chooses exactly how to open a file: append, create, create_new (fail if it exists), truncate, read, and write. File::open and File::create are shortcuts for the two most common combinations.
use std::fs::{self, OpenOptions};
use std::io::{self, ErrorKind, Write};
fn append_line(path: &str, line: &str) -> io::Result<()> {
let mut file = OpenOptions::new().create(true).append(true).open(path)?;
writeln!(file, "{line}")
}
fn main() -> io::Result<()> {
let _ = fs::remove_file("audit.log"); // start fresh; ignore "not found"
append_line("audit.log", "ana logged in")?;
append_line("audit.log", "ana exported report")?;
append_line("audit.log", "ana logged out")?;
print!("{}", fs::read_to_string("audit.log")?);
// create_new: fail if the file already exists.
match OpenOptions::new().write(true).create_new(true).open("audit.log") {
Ok(_) => println!("created"),
Err(e) if e.kind() == ErrorKind::AlreadyExists => println!("audit.log already exists, not overwriting"),
Err(e) => return Err(e),
}
Ok(())
}ana logged in
ana exported report
ana logged out
audit.log already exists, not overwritinglet _ = fs::remove_file(...) deliberately ignores the result: if there was no file to remove, that's fine.
11. Functions that accept any reader
A function that takes a file path can only read files. One that takes impl BufRead works with a file, standard input, a network connection, or bytes in memory. That makes it trivial to test: a &[u8] is a BufRead, so tests pass text directly.
use std::fs::{self, File};
use std::io::{self, BufRead, BufReader};
// Works with any buffered input: a file, stdin, a socket, or bytes in memory.
fn longest_line(input: impl BufRead) -> io::Result<Option<String>> {
let mut best: Option<String> = None;
for line in input.lines() {
let line = line?;
// Keep `line` if there's no best yet, or if it's longer than the best.
if best.as_ref().is_none_or(|b| line.len() > b.len()) {
best = Some(line);
}
}
Ok(best)
}
fn main() -> io::Result<()> {
// In a test: bytes in memory, no file needed.
println!("{:?}", longest_line("short\nthe longest line\nmid-sized\n".as_bytes())?);
println!("{:?}", longest_line("".as_bytes())?);
// In the program: a real file.
fs::write("poem.txt", "Roses are red\nViolets are blue\nRust is fast\n")?;
println!("{:?}", longest_line(BufReader::new(File::open("poem.txt")?))?);
Ok(())
}Some("the longest line")
None
Some("Violets are blue")std::io::Cursor is the other common in-memory reader: it wraps a Vec<u8> or String and also supports seeking.
12. Functions that accept any writer
The same idea works for output. A report function that takes &mut impl Write can write to a file, to the terminal, or into a Vec<u8>, which is how tests check output.
use std::fs::{self, File};
use std::io::{self, Write};
struct Sale {
region: &'static str,
cents: u64,
}
fn write_report(out: &mut impl Write, sales: &[Sale]) -> io::Result<()> {
writeln!(out, "{:<8}{:>10}", "region", "total")?;
for s in sales {
writeln!(out, "{:<8}{:>10.2}", s.region, s.cents as f64 / 100.0)?;
}
let total: u64 = sales.iter().map(|s| s.cents).sum();
writeln!(out, "{:<8}{:>10.2}", "ALL", total as f64 / 100.0)
}
fn main() -> io::Result<()> {
let sales = [Sale { region: "north", cents: 1_250_000 }, Sale { region: "south", cents: 980_050 }];
let mut buffer = Vec::new(); // a Vec<u8> is a Write: perfect for tests
write_report(&mut buffer, &sales)?;
println!("{} bytes written to memory", buffer.len());
write_report(&mut File::create("report.txt")?, &sales)?;
print!("{}", fs::read_to_string("report.txt")?);
write_report(&mut io::stdout(), &sales[..1]) // or straight to the terminal
}76 bytes written to memory
region total
north 12500.00
south 9800.50
ALL 22300.50
region total
north 12500.00
ALL 12500.0013. Reading in chunks
For binary data, or when lines don't matter, read into a fixed-size buffer with read. It returns how many bytes it put into the buffer, which may be fewer than the buffer holds, and 0 at the end of the input. io::copy runs this loop for you when all you want is to move bytes from a reader to a writer.
use std::fs::{self, File};
use std::io::{self, Read};
// The FNV-1a hash of everything the reader produces, and its length.
fn fnv1a(mut input: impl Read) -> io::Result<(u64, usize)> {
let mut hash: u64 = 0xcbf29ce484222325;
let mut buf = [0u8; 4096];
let mut total = 0;
loop {
let n = input.read(&mut buf)?;
if n == 0 {
break;
}
for &byte in &buf[..n] {
hash ^= byte as u64;
hash = hash.wrapping_mul(0x100000001b3);
}
total += n;
}
Ok((hash, total))
}
fn main() -> io::Result<()> {
let data: Vec<u8> = (0..100_000u32).map(|i| (i % 251) as u8).collect();
fs::write("big.bin", &data)?;
let (hash, size) = fnv1a(File::open("big.bin")?)?;
println!("{size} bytes, hash {hash:016x}");
let copied = io::copy(&mut File::open("big.bin")?, &mut File::create("big-copy.bin")?)?;
let (copy_hash, _) = fnv1a(File::open("big-copy.bin")?)?;
println!("copied {copied} bytes, same hash: {}", copy_hash == hash);
Ok(())
}100000 bytes, hash e796b565d8a90d28
copied 100000 bytes, same hash: true14. Jumping around with Seek
Files of fixed-size records can be read and updated in place. seek moves the position; read_exact fills the whole buffer or fails with UnexpectedEof.
use std::fs::{File, OpenOptions};
use std::io::{self, Read, Seek, SeekFrom, Write};
const RECORD: usize = 16; // 12 bytes of name + 4 bytes of score
fn encode(name: &str, score: u32) -> [u8; RECORD] {
let mut rec = [b' '; RECORD];
let len = name.len().min(12);
rec[..len].copy_from_slice(&name.as_bytes()[..len]);
rec[12..].copy_from_slice(&score.to_le_bytes());
rec
}
fn read_record(file: &mut File, index: u64) -> io::Result<(String, u32)> {
let mut rec = [0u8; RECORD];
file.seek(SeekFrom::Start(index * RECORD as u64))?;
file.read_exact(&mut rec)?;
let name = String::from_utf8_lossy(&rec[..12]).trim_end().to_string();
let score = u32::from_le_bytes([rec[12], rec[13], rec[14], rec[15]]);
Ok((name, score))
}
fn main() -> io::Result<()> {
let mut file = OpenOptions::new().read(true).write(true).create(true).truncate(true).open("scores.dat")?;
for (name, score) in [("ana", 120), ("ben", 95), ("cy", 310)] {
file.write_all(&encode(name, score))?;
}
println!("record 2: {:?}", read_record(&mut file, 2)?);
println!("record 0: {:?}", read_record(&mut file, 0)?);
// Overwrite record 1 in place.
file.seek(SeekFrom::Start(RECORD as u64))?;
file.write_all(&encode("ben", 99))?;
println!("record 1: {:?}", read_record(&mut file, 1)?);
println!("file size: {} bytes", file.metadata()?.len());
match read_record(&mut file, 7) {
Ok(r) => println!("record 7: {r:?}"),
Err(e) => println!("record 7: {:?}", e.kind()),
}
Ok(())
}record 2: ("cy", 310)
record 0: ("ana", 120)
record 1: ("ben", 99)
file size: 48 bytes
record 7: UnexpectedEofPart 3: Managing files
15. Copying, renaming, and removing
fs::copy, fs::rename, and fs::remove_file copy, move, and delete files. fs::metadata returns a file's size, type, and timestamps, and fs::exists checks whether a path exists (it can also fail, for example without permission to look).
use std::fs;
use std::io;
fn main() -> io::Result<()> {
fs::write("draft.md", "# Plan\n\n- ship it\n")?;
let bytes = fs::copy("draft.md", "draft-backup.md")?; // returns the number of bytes copied
println!("copied {bytes} bytes");
fs::rename("draft.md", "final.md")?; // rename or move, within one filesystem
println!("draft.md exists: {}", fs::exists("draft.md")?);
println!("final.md is {} bytes", fs::metadata("final.md")?.len());
fs::remove_file("draft-backup.md")?;
match fs::remove_file("draft-backup.md") {
Ok(()) => println!("removed"),
Err(e) => println!("second remove failed: {:?}", e.kind()),
}
Ok(())
}copied 18 bytes
draft.md exists: false
final.md is 18 bytes
second remove failed: NotFound16. Saving safely: write, then rename
If a program crashes halfway through fs::write, the file is left half-written, and the old contents are gone. The safe way to replace a file is to write the new contents to a temporary file next to it and then rename it over the original. On the same filesystem, the rename happens at once: other programs see either the old file or the new one, never a mix.
use std::fs::{self, File};
use std::io::{self, Write};
fn save_atomically(path: &str, contents: &str) -> io::Result<()> {
let tmp = format!("{path}.tmp");
let mut file = File::create(&tmp)?;
file.write_all(contents.as_bytes())?;
file.sync_all()?; // wait until the bytes have reached the disk
fs::rename(&tmp, path)
}
fn main() -> io::Result<()> {
save_atomically("state.json", r#"{"version":1,"items":[]}"#)?;
save_atomically("state.json", r#"{"version":2,"items":["a"]}"#)?;
println!("{}", fs::read_to_string("state.json")?);
println!("temp file left behind: {}", fs::exists("state.json.tmp")?);
Ok(())
}{"version":2,"items":["a"]}
temp file left behind: falsePart 4: Realistic examples
17. Realistic example: loading a CSV file with line numbers in errors
A loader for a simple CSV file. Its error type distinguishes I/O failures from bad data, and bad data reports the line number, which is what a user needs to fix the file. (For real CSV, with quoting and escaping, use the csv crate.)
use std::fmt;
use std::fs::{self, File};
use std::io::{self, BufRead, BufReader};
struct Order {
id: u32,
customer: String,
cents: u64,
}
#[derive(Debug)]
enum LoadError {
Io(io::Error),
Parse { line: usize, message: String },
}
impl fmt::Display for LoadError {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
LoadError::Io(e) => write!(f, "{e}"),
LoadError::Parse { line, message } => write!(f, "line {line}: {message}"),
}
}
}
impl From<io::Error> for LoadError {
fn from(e: io::Error) -> Self {
LoadError::Io(e)
}
}
fn parse_line(line: &str) -> Result<Order, String> {
let fields: Vec<&str> = line.split(',').map(str::trim).collect();
let [id, customer, amount] = fields[..] else {
return Err(format!("expected 3 fields, found {}", fields.len()));
};
let id = id.parse().map_err(|_| format!("bad id {id:?}"))?;
let amount: f64 = amount.parse().map_err(|_| format!("bad amount {amount:?}"))?;
Ok(Order { id, customer: customer.to_string(), cents: (amount * 100.0).round() as u64 })
}
fn load_orders(path: &str) -> Result<Vec<Order>, LoadError> {
let reader = BufReader::new(File::open(path)?);
let mut orders = Vec::new();
for (index, line) in reader.lines().enumerate().skip(1) {
let line = line?;
if line.trim().is_empty() {
continue;
}
let order = parse_line(&line).map_err(|message| LoadError::Parse { line: index + 1, message })?;
orders.push(order);
}
Ok(orders)
}
fn main() -> io::Result<()> {
fs::write("orders.csv", "id,customer,amount\n1,Acme,120.50\n2,Globex,99.99\n\n3,Initech,15\n")?;
fs::write("broken.csv", "id,customer,amount\n1,Acme,120.50\n2,Globex\n")?;
fs::write("typo.csv", "id,customer,amount\n1,Acme,12O.50\n")?;
for path in ["orders.csv", "broken.csv", "typo.csv", "nope.csv"] {
match load_orders(path) {
Ok(orders) => {
let total: u64 = orders.iter().map(|o| o.cents).sum();
let names: Vec<String> = orders.iter().map(|o| format!("#{} {}", o.id, o.customer)).collect();
println!("{path}: {} (total {}.{:02})", names.join(", "), total / 100, total % 100);
}
Err(e) => println!("{path}: {e}"),
}
}
Ok(())
}orders.csv: #1 Acme, #2 Globex, #3 Initech (total 235.49)
broken.csv: line 3: expected 3 fields, found 2
typo.csv: line 2: bad amount "12O.50"
nope.csv: No such file or directory (os error 2)18. Realistic example: an append-only journal
Some applications never rewrite their data file: they append one line per event and rebuild their state by replaying the file at startup. Appends are cheap, and the file is a complete history. This one keeps account balances.
use std::collections::BTreeMap;
use std::fs::{self, File, OpenOptions};
use std::io::{self, BufRead, BufReader, ErrorKind, Write};
struct Journal {
path: String,
}
impl Journal {
fn record(&self, event: &str) -> io::Result<()> {
let mut file = OpenOptions::new().create(true).append(true).open(&self.path)?;
writeln!(file, "{event}")
}
fn replay(&self) -> io::Result<BTreeMap<String, i64>> {
let mut balances = BTreeMap::new();
let file = match File::open(&self.path) {
Ok(f) => f,
Err(e) if e.kind() == ErrorKind::NotFound => return Ok(balances), // no journal yet
Err(e) => return Err(e),
};
for (index, line) in BufReader::new(file).lines().enumerate() {
let line = line?;
let bad = |why: &str| io::Error::new(ErrorKind::InvalidData, format!("line {}: {why}: {line:?}", index + 1));
let [op, account, amount] = line.split_whitespace().collect::<Vec<_>>()[..] else {
return Err(bad("expected 3 fields"));
};
let amount: i64 = amount.parse().map_err(|_| bad("bad amount"))?;
let balance = balances.entry(account.to_string()).or_insert(0);
match op {
"deposit" => *balance += amount,
"withdraw" => *balance -= amount,
_ => return Err(bad("unknown operation")),
}
}
Ok(balances)
}
}
fn main() -> io::Result<()> {
let _ = fs::remove_file("bank.journal");
let journal = Journal { path: String::from("bank.journal") };
println!("before: {:?}", journal.replay()?);
for event in ["deposit ana 500", "deposit ben 300", "withdraw ana 120", "deposit ana 40"] {
journal.record(event)?;
}
println!("after replay: {:?}", journal.replay()?);
journal.record("withdraw ben lots")?;
println!("after a bad line: {}", journal.replay().unwrap_err());
Ok(())
}before: {}
after replay: {"ana": 420, "ben": 300}
after a bad line: line 5: bad amount: "withdraw ben lots"io::Error::new creates an io::Error of your own, here with the InvalidData kind, so the function can keep returning io::Result.
19. Realistic example: the most common words in a large file
BufReader and a HashMap handle a file of any size: only the counts are kept in memory, never the whole text.
use std::collections::HashMap;
use std::fs::{self, File};
use std::io::{self, BufRead, BufReader};
fn top_words(path: &str, n: usize) -> io::Result<Vec<(String, usize)>> {
let mut counts: HashMap<String, usize> = HashMap::new();
for line in BufReader::new(File::open(path)?).lines() {
for word in line?.split(|c: char| !c.is_alphanumeric()).filter(|w| !w.is_empty()) {
*counts.entry(word.to_lowercase()).or_insert(0) += 1;
}
}
let mut sorted: Vec<(String, usize)> = counts.into_iter().collect();
sorted.sort_by(|a, b| b.1.cmp(&a.1).then_with(|| a.0.cmp(&b.0)));
sorted.truncate(n);
Ok(sorted)
}
fn main() -> io::Result<()> {
let text = "The quick brown fox jumps over the lazy dog.\nThe dog sleeps; the fox runs!\n";
fs::write("story.txt", text.repeat(1_000))?;
println!("{} bytes", fs::metadata("story.txt")?.len());
for (word, count) in top_words("story.txt", 4)? {
println!("{word:<6} {count}");
}
Ok(())
}75000 bytes
the 4000
dog 2000
fox 2000
brown 100020. Realistic example: splitting a log into one file per day
Reading one file and writing several: each day gets its own BufWriter, kept in a map until the input is done, then flushed.
use std::collections::BTreeMap;
use std::fs::{self, File};
use std::io::{self, BufRead, BufReader, BufWriter, Write};
fn split_by_day(input: &str) -> io::Result<BTreeMap<String, usize>> {
let mut writers: BTreeMap<String, BufWriter<File>> = BTreeMap::new();
let mut counts = BTreeMap::new();
for line in BufReader::new(File::open(input)?).lines() {
let line = line?;
let Some(day) = line.get(..10) else { continue }; // lines start with "2026-09-27"
if !writers.contains_key(day) {
let file = File::create(format!("log-{day}.txt"))?;
writers.insert(day.to_string(), BufWriter::new(file));
}
let message = line.get(11..).unwrap_or("");
writeln!(writers.get_mut(day).unwrap(), "{message}")?;
*counts.entry(day.to_string()).or_insert(0) += 1;
}
for writer in writers.values_mut() {
writer.flush()?;
}
Ok(counts)
}
fn main() -> io::Result<()> {
fs::write(
"server.log",
"2026-09-26 09:00 start\n2026-09-26 17:30 stop\n2026-09-27 08:55 start\n2026-09-27 12:01 deploy v2\n2026-09-27 18:00 stop\n",
)?;
for (day, lines) in split_by_day("server.log")? {
println!("log-{day}.txt: {lines} lines");
}
print!("{}", fs::read_to_string("log-2026-09-27.txt")?);
Ok(())
}log-2026-09-26.txt: 2 lines
log-2026-09-27.txt: 3 lines
08:55 start
12:01 deploy v2
18:00 stopSummary
fs::read_to_string,fs::read, andfs::writehandle whole files in one call.- Every I/O operation returns
io::Result. Match one.kind()to handle specific cases likeNotFound, and add the file name to error messages withwith_contextor your own error type. BufReader+lines()(orread_linewith a reused buffer) reads large files line by line;readwith a buffer reads binary data in chunks.File::create+writeln!writes text; importstd::io::Writefor the methods. Wrap files inBufWriterfor many small writes and callflush()at the end.OpenOptionscovers appending,create_new, and read-write access;Seekmoves around in a file.- Files close automatically when they go out of scope.
- Write functions against
impl BufRead,impl Read, andimpl Writerather than paths, so tests can use in-memory bytes. - Replace important files by writing a temporary file and renaming it.
Next: Paths & Directories.