Streams, Readers, and Files

This lesson covers blocking I/O in java.io and the java.nio.file API for files. After it, a reader can read and write bytes and text with the right charset, choose a buffer, work with files through Path and Files, and explain why Java serialization is avoided in new code. ByteBuffer and memory-mapped files are taught in ByteBuffer, Direct Buffers, and Memory-Mapped Files and are only linked here.

API	Purpose
`InputStream`, `OutputStream`	Sequences of bytes
`Reader`, `Writer`	Sequences of `char`, decoded from bytes with a `Charset`
`BufferedInputStream`, `BufferedOutputStream`, `BufferedReader`, `BufferedWriter`	Fewer calls to the underlying stream
`Path`, `Files`	File names and file operations
`RandomAccessFile`	Read and write at any offset
`FileChannel`	Channel-based file I/O, `transferTo`
`ObjectOutputStream`, `ObjectInputStream`	Java serialization

A blocking call is one that does not return until data is available, the data is written, or an error occurs. Every API in this lesson blocks. Non-blocking channels are the subject of NIO Channels and Selectors.

Every file example works inside a directory from Files.createTempDirectory and deletes it at the end. The programs print contents and counts, never paths.

Example 1: InputStream and OutputStream

An OutputStream accepts bytes and an InputStream produces them. read() returns one byte as an int from 0 to 255, or -1 at the end of the stream. read(byte[]) returns the number of bytes it put in the array, which can be fewer than the array length.

Java

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.Arrays;

public class ByteStreams {
    public static void main(String[] args) throws IOException {
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        out.write(1);
        out.write(new byte[] {2, 3, 4, 5, (byte) 200});
        byte[] data = out.toByteArray();
        System.out.println("written: " + Arrays.toString(data));

        try (InputStream in = new ByteArrayInputStream(data)) {
            System.out.println("first read(): " + in.read());
            byte[] chunk = new byte[4];
            int n;
            while ((n = in.read(chunk)) != -1) {
                System.out.println("read " + n + " bytes: " + Arrays.toString(Arrays.copyOf(chunk, n)));
            }
            System.out.println("at end: " + in.read());
        }
    }
}

Output

written: [1, 2, 3, 4, 5, -56]
first read(): 1
read 4 bytes: [2, 3, 4, 5]
read 1 bytes: [-56]
at end: -1

The byte 200 is stored as the signed byte value -56. read() returns an int so that all 256 byte values and the end marker -1 fit in one return type. The second read(chunk) returned 1, not 4: code that assumes a full array is a common bug. InputStream.readNBytes(n) and readAllBytes() loop until they have the requested count or reach the end.

Example 2: readers, writers, and charsets

A Reader produces char values and a Writer accepts them. InputStreamReader and OutputStreamWriter connect the two worlds by decoding and encoding with a Charset. The same text has a different byte length in different charsets.

Java

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.Reader;
import java.io.Writer;
import java.nio.charset.StandardCharsets;

public class Charsets {
    public static void main(String[] args) throws IOException {
        String text = "naïve €5";
        System.out.println("chars: " + text.length());

        ByteArrayOutputStream utf8 = new ByteArrayOutputStream();
        try (Writer w = new OutputStreamWriter(utf8, StandardCharsets.UTF_8)) {
            w.write(text);
        }
        System.out.println("UTF-8 bytes: " + utf8.size());

        ByteArrayOutputStream utf16 = new ByteArrayOutputStream();
        try (Writer w = new OutputStreamWriter(utf16, StandardCharsets.UTF_16BE)) {
            w.write(text);
        }
        System.out.println("UTF-16BE bytes: " + utf16.size());

        try (Reader r = new InputStreamReader(new ByteArrayInputStream(utf8.toByteArray()), StandardCharsets.UTF_8)) {
            StringBuilder sb = new StringBuilder();
            int c;
            while ((c = r.read()) != -1) sb.append((char) c);
            System.out.println("decoded with UTF-8: " + sb);
        }
        String wrong = new String(utf8.toByteArray(), StandardCharsets.ISO_8859_1);
        System.out.println("decoded with ISO-8859-1: " + wrong.length() + " chars, equal = " + wrong.equals(text));
    }
}

Output

chars: 8
UTF-8 bytes: 11
UTF-16BE bytes: 16
decoded with UTF-8: naïve €5
decoded with ISO-8859-1: 11 chars, equal = false

ï takes 2 bytes in UTF-8 and € takes 3, so 8 characters become 11 bytes. Decoding with the wrong charset does not fail: it produces different text, one character per byte. Since Java 18 the default charset is UTF-8, but passing the charset explicitly keeps the code correct when it reads data produced elsewhere.

Example 3: buffering

Each call on an unbuffered stream reaches the underlying sink, and for a file or socket that is one system call. A buffered stream collects writes in an array and passes them on in large blocks. This example counts the calls that reach the sink.

Java

import java.io.BufferedOutputStream;
import java.io.IOException;
import java.io.OutputStream;

public class Buffering {
    static class CountingSink extends OutputStream {
        int calls;
        long bytes;
        @Override public void write(int b) { calls++; bytes++; }
        @Override public void write(byte[] b, int off, int len) { calls++; bytes += len; }
    }

    public static void main(String[] args) throws IOException {
        CountingSink raw = new CountingSink();
        for (int i = 0; i < 100_000; i++) raw.write('x');
        System.out.println("unbuffered: calls=" + raw.calls + " bytes=" + raw.bytes);

        CountingSink sink = new CountingSink();
        try (OutputStream out = new BufferedOutputStream(sink, 8192)) {
            for (int i = 0; i < 100_000; i++) out.write('x');
        }
        System.out.println("buffered:   calls=" + sink.calls + " bytes=" + sink.bytes);

        CountingSink lost = new CountingSink();
        OutputStream notClosed = new BufferedOutputStream(lost, 8192);
        notClosed.write(new byte[100]);
        System.out.println("before flush: bytes=" + lost.bytes);
        notClosed.flush();
        System.out.println("after flush:  bytes=" + lost.bytes);
    }
}

Output

unbuffered: calls=100000 bytes=100000
buffered:   calls=13 bytes=100000
before flush: bytes=0
after flush:  bytes=100

100,000 single-byte writes become 13 block writes of up to 8192 bytes. The last part of the data stays in the buffer until flush or close, so a buffered stream that is never closed loses its tail. The JDK default buffer size for BufferedInputStream, BufferedReader, and BufferedWriter is 8192 (DEFAULT_BUFFER_SIZE and DEFAULT_CHAR_BUFFER_SIZE in the JDK source). BufferedOutputStream without a size argument starts smaller and grows up to 8192.

Example 4: Path and Files

Path is an immutable file name. Files holds static methods that act on paths: create, read, write, copy, move, delete, list, and read attributes. writeString and readAllLines open and close the file inside the call.

Java

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.nio.file.StandardOpenOption;
import java.util.Comparator;
import java.util.List;
import java.util.stream.Stream;

public class PathsAndFiles {
    public static void main(String[] args) throws IOException {
        Path dir = Files.createTempDirectory("io-demo");
        try {
            Path notes = dir.resolve("notes.txt");
            System.out.println("file name: " + notes.getFileName());
            System.out.println("exists before: " + Files.exists(notes));

            Files.writeString(notes, "alpha\nbeta\n", StandardCharsets.UTF_8);
            Files.writeString(notes, "gamma\n", StandardOpenOption.APPEND);
            System.out.println("size: " + Files.size(notes));
            List<String> lines = Files.readAllLines(notes);
            System.out.println("lines: " + lines);

            Path copy = Files.copy(notes, dir.resolve("copy.txt"));
            Files.move(copy, dir.resolve("moved.txt"), StandardCopyOption.ATOMIC_MOVE);
            Files.createDirectories(dir.resolve("a/b"));
            try (Stream<Path> s = Files.list(dir)) {
                System.out.println("entries: " + s.map(p -> p.getFileName().toString()).sorted().toList());
            }
        } finally {
            try (Stream<Path> walk = Files.walk(dir)) {
                walk.sorted(Comparator.reverseOrder()).forEach(p -> p.toFile().delete());
            }
        }
        System.out.println("deleted: " + !Files.exists(dir));
    }
}

Output

file name: notes.txt
exists before: false
size: 17
lines: [alpha, beta, gamma]
entries: [a, moved.txt, notes.txt]
deleted: true

resolve joins path parts without touching the disk. ATOMIC_MOVE asks the file system for a rename that other processes see as one step, a common way to publish a finished file. Files.list and Files.walk return streams that hold an open directory handle, so they go in try-with-resources. Deleting a tree means deleting children first, which the reverse-sorted walk does.

Example 5: Files.lines and try-with-resources

Files.lines returns a lazy Stream<String>: it reads lines as the stream pulls them, so a large file is not loaded into memory at once. The stream holds the file open until it is closed. Try-with-resources closes it even when the pipeline throws.

Java

import java.io.BufferedReader;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Map;
import java.util.TreeMap;
import java.util.stream.Collectors;
import java.util.stream.Stream;

public class LogScan {
    public static void main(String[] args) throws IOException {
        Path dir = Files.createTempDirectory("logs");
        Path log = dir.resolve("app.log");
        try {
            Files.write(log, java.util.List.of(
                "INFO start", "WARN slow disk", "INFO request ok",
                "ERROR timeout", "INFO request ok", "ERROR refused"));

            try (Stream<String> lines = Files.lines(log)) {
                Map<String, Long> byLevel = lines
                    .map(l -> l.substring(0, l.indexOf(' ')))
                    .collect(Collectors.groupingBy(l -> l, TreeMap::new, Collectors.counting()));
                System.out.println("by level: " + byLevel);
            }

            try (BufferedReader r = Files.newBufferedReader(log)) {
                String line;
                int n = 0;
                while ((line = r.readLine()) != null) {
                    n++;
                    if (line.startsWith("ERROR")) System.out.println("line " + n + ": " + line);
                }
            }
        } finally {
            Files.deleteIfExists(log);
            Files.deleteIfExists(dir);
        }
    }
}

Output

by level: {ERROR=2, INFO=3, WARN=1}
line 4: ERROR timeout
line 6: ERROR refused

The two styles read the same file. The stream suits filtering and grouping. The BufferedReader loop suits code that tracks state across lines, such as a line number. Both decode with UTF-8 unless a charset is passed. Resources declared in one try-with-resources header close in reverse order, and an exception from close is added as a suppressed exception when the body has already thrown (see Exceptions).

Example 6: RandomAccessFile with fixed-size records

RandomAccessFile reads and writes at any byte offset through seek. With fixed-size records, record i starts at i * RECORD_SIZE, so one record can be read or updated without reading the rest of the file.

Java

import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.file.Files;
import java.nio.file.Path;

public class Records {
    static final int RECORD = Long.BYTES + Integer.BYTES; // id + quantity = 12 bytes

    public static void main(String[] args) throws IOException {
        Path dir = Files.createTempDirectory("raf");
        Path file = dir.resolve("orders.dat");
        try (RandomAccessFile raf = new RandomAccessFile(file.toFile(), "rw")) {
            for (int i = 0; i < 5; i++) {
                raf.writeLong(1000 + i);
                raf.writeInt(10 * (i + 1));
            }
            System.out.println("length: " + raf.length());

            raf.seek(3L * RECORD + Long.BYTES);     // quantity of record 3
            raf.writeInt(999);

            for (int i = 0; i < 5; i++) {
                raf.seek((long) i * RECORD);
                System.out.println("record " + i + ": id=" + raf.readLong() + " qty=" + raf.readInt());
            }
            System.out.println("pointer: " + raf.getFilePointer());
        } finally {
            Files.deleteIfExists(file);
            Files.deleteIfExists(dir);
        }
    }
}

Output

length: 60
record 0: id=1000 qty=10
record 1: id=1001 qty=20
record 2: id=1002 qty=30
record 3: id=1003 qty=999
record 4: id=1004 qty=50
pointer: 60

writeLong and writeInt use big-endian byte order, the same as DataOutputStream. The mode string "rw" opens for reading and writing and creates the file; "rwd" and "rws" also write each update to the storage device before returning. RandomAccessFile is unbuffered, so each readInt is one system call. For bulk access, a FileChannel with a large ByteBuffer or a mapped file is faster.

Example 7: FileChannel basics and transferTo

FileChannel reads and writes ByteBuffers at a position and is covered in detail, together with mapping, in ByteBuffer, Direct Buffers, and Memory-Mapped Files. This example shows two things that lesson does not: getting a channel from a stream class, and transferTo, which copies between channels and lets the operating system move the bytes without passing them through a Java array.

Java

import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.channels.FileChannel;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;

public class ChannelCopy {
    public static void main(String[] args) throws IOException {
        Path dir = Files.createTempDirectory("chan");
        Path src = dir.resolve("src.bin");
        Path dst = dir.resolve("dst.bin");
        try {
            byte[] data = new byte[100_000];
            for (int i = 0; i < data.length; i++) data[i] = (byte) i;
            Files.write(src, data);

            try (RandomAccessFile raf = new RandomAccessFile(src.toFile(), "r");
                 FileChannel in = raf.getChannel();
                 FileChannel out = FileChannel.open(dst, StandardOpenOption.CREATE, StandardOpenOption.WRITE)) {
                System.out.println("source size: " + in.size());
                long done = 0;
                while (done < in.size()) {
                    done += in.transferTo(done, in.size() - done, out);
                }
                System.out.println("copied: " + done);
                System.out.println("source position unchanged: " + (in.position() == 0));
            }
            System.out.println("mismatch index: " + Files.mismatch(src, dst));
        } finally {
            Files.deleteIfExists(src);
            Files.deleteIfExists(dst);
            Files.deleteIfExists(dir);
        }
    }
}

Output

source size: 100000
copied: 100000
source position unchanged: true
mismatch index: -1

transferTo can copy fewer bytes than asked, so the loop continues until the count is reached. The positional form does not move the source channel's position. On Linux the JDK can hand the copy to a kernel system call so the bytes stay in kernel memory. Files.mismatch (Java 12) returns -1 when the two files are equal. Files.copy does the same job in one call; transferTo matters when the target is a socket channel.

Example 8: serialization and why it is avoided

Java serialization writes an object graph to bytes with ObjectOutputStream and rebuilds it with ObjectInputStream. Reading creates objects of whatever classes the bytes name, without running their constructors (records are the exception: they are rebuilt through the canonical constructor). An ObjectInputFilter limits which classes may be created.

Java

import java.io.*;
import java.util.ArrayList;
import java.util.List;

public class Serial {
    record Trade(String symbol, long qty) implements Serializable {
        Trade {
            if (qty <= 0) throw new IllegalArgumentException("qty must be positive");
        }
    }

    static byte[] write(Object o) throws IOException {
        ByteArrayOutputStream bytes = new ByteArrayOutputStream();
        try (ObjectOutputStream out = new ObjectOutputStream(bytes)) {
            out.writeObject(o);
        }
        return bytes.toByteArray();
    }

    public static void main(String[] args) throws Exception {
        byte[] bytes = write(new Trade("ACME", 100));
        System.out.println("serialized size: " + bytes.length + " bytes for 4 chars + 8 bytes of data");
        try (ObjectInputStream in = new ObjectInputStream(new ByteArrayInputStream(bytes))) {
            System.out.println("read back: " + in.readObject());
        }

        byte[] listBytes = write(new ArrayList<>(List.of(1, 2, 3)));
        try (ObjectInputStream in = new ObjectInputStream(new ByteArrayInputStream(listBytes))) {
            in.setObjectInputFilter(ObjectInputFilter.Config.createFilter("Serial$Trade;java.lang.*;!*"));
            in.readObject();
        } catch (InvalidClassException e) {
            System.out.println("rejected: " + e.getMessage());
        }
    }
}

Output

serialized size: 84 bytes for 4 chars + 8 bytes of data
read back: Trade[symbol=ACME, qty=100]
rejected: filter status: REJECTED

The reasons serialization is avoided in new code:

Problem	Consequence
Reading creates any class on the class path	Crafted input can run code through "gadget" classes; many CVEs come from this
Constructors of ordinary classes are skipped	Invariants checked in constructors are not enforced on read
Format tied to class names and `serialVersionUID`	Renaming or changing a class breaks old data
Size and speed	Class descriptors make small objects large; the format is Java-only

The filter allows only Trade and java.lang classes, so the ArrayList in the second stream is rejected before any object is created. Common replacements are explicit formats: JSON, Protocol Buffers, or a hand-written binary layout (see Binary Protocols and Encoding). When old code must read serialized data, a filter with an allow list and !* at the end rejects everything else.

Interview points

read() returns -1 at end of stream; read(byte[]) may return fewer bytes than requested. Loop, or use readNBytes.
Bytes become text only through a Charset. Wrong charset gives wrong text, not an exception.
Buffering reduces system calls; an unflushed buffered writer loses data. Close in try-with-resources.
Files.lines, Files.list, and Files.walk hold open handles and must be closed.
RandomAccessFile gives offset access for fixed-size records; it is unbuffered.
FileChannel.transferTo copies without a user-space array and can be partial.
Java serialization is a security risk on untrusted input; use explicit formats or an ObjectInputFilter allow list.

Summary

InputStream/OutputStream carry bytes; Reader/Writer carry characters decoded with a charset.
Default JDK buffer size for buffered readers and streams is 8192.
Path names a file; Files performs operations on it.
Files.lines is lazy and must be closed.
RandomAccessFile.seek moves to an offset; its writeInt/writeLong are big-endian.
FileChannel adds positional reads and writes and transferTo.
Serialization skips constructors of non-record classes and creates classes named by the input.