Design Google Docs

A collaborative editor lets several users edit the same document at once and see each other's changes within a fraction of a second. The central problem is concurrent edits: two users change the same text at the same time, and every replica must converge to the same document without losing intent. This article covers the sync algorithm, the server architecture, and storage.

Requirements

Functional

• Create, open, and edit documents; rich text formatting.
• Multiple users edit simultaneously; changes appear on all clients in near real time.
• Show collaborator cursors and selections (presence).
• Version history and restore.
• Sharing with permissions (view, comment, edit).
• Offline editing with sync on reconnect (optional).

Non-functional

• Edit propagation latency under a few hundred milliseconds.
• Convergence: all clients end in the same state after applying all edits.
• Durability: no acknowledged edit is lost.
• Availability and low latency for local typing (edits apply locally first).
• Scale: many documents, but few concurrent editors per document (tens, rarely hundreds).

Estimation

Assumptions: 100 million DAU (daily active users), 10 million concurrently connected editors at peak, an average of 3 editors per active document.

• Active documents: 10 x 10^6 / 3 ≈ 3.3 million concurrent documents.
• A typing editor produces about 1 operation per second (keystrokes batched), and about 30% of connected users are typing at any moment: 10 x 10^6 x 0.3 x 1 = 3 x 10^6 ops/s peak.
• Operation size about 100 bytes: 3 x 10^6 x 100 B = 300 MB/s of ingress; each op fans out to about 2 other editors, so about 600 MB/s egress.
• WebSocket connections: 10 million concurrent. At about 50,000 connections per server, this needs about 200 connection servers.
• Storage: a single document's operation log is typically kilobytes to megabytes (for example, 10,000 operations x 100 B = 1 MB). Snapshots plus logs for billions of documents, including revision history, add up to petabytes.

The workload is dominated by many small, independent per-document streams, which makes per-document partitioning natural.

What the numbers imply

• 3 x 10^6 ops/s is spread over about 1 million typing documents, each producing only 1 to 3 ops/s. Per-document load is trivial, so one single-threaded sequencer per document is enough. The cluster-wide log write rate, not per-document throughput, is the scaling problem: at roughly 5,000 appends/s per log shard, this needs about 600 shards, so the log store must be partitioned by doc_id.
• If the average active document holds about 50 KB in memory, 3.3 million documents need about 165 GB spread over 200 collaboration servers, under 1 GB each. Memory is not a constraint; connection count is.
• 600 MB/s egress is about 5 Gbit/s in total, roughly 25 Mbit/s per gateway. Bandwidth is small. Fan-out cost grows with editors squared per document (see hot documents below), which is why editors per document are capped.

API design

Document management uses REST:

POST /v1/docs                      -> {doc_id}
GET  /v1/docs/{id}                 -> latest snapshot + revision number
GET  /v1/docs/{id}/revisions?from= -> operations since revision
POST /v1/docs/{id}/share           {user, role}

Editing uses a persistent WebSocket connection (WebSockets, SSE, long polling):

client -> server: {type: "op", doc: id, base_rev: 41, op: [retain 5, insert "x"]}
server -> client: {type: "ack", rev: 42}
server -> others: {type: "op", rev: 42, op: [...], author: u}
client <-> server: {type: "presence", cursor: 17, selection: [17, 22]}

Data model

The operation log is the source of truth. A snapshot is taken every N operations so loading a document means reading the latest snapshot and replaying only the operations after it. Old revisions are reconstructable for version history.

High-level design

flowchart LR
    C1[Client A] <-->|WebSocket| GW[Connection gateway]
    C2[Client B] <-->|WebSocket| GW
    GW --> R[Router by doc_id]
    R --> CS[Collaboration server for doc]
    CS --> OL[(Operation log)]
    CS --> PS[Pub/sub for cross-server fan-out]
    CS --> Snap[Snapshot worker]
    Snap --> BS[(Snapshot store)]
    API[REST API] --> Meta[(Metadata and ACL)]
    GW --> API

Components

• Connection gateway terminates WebSockets and authenticates users.
• Router maps a document to one collaboration server using consistent hashing on doc_id, so all editors of a document reach the same process.
• Collaboration server holds the in-memory state of active documents, serializes operations into a single order, transforms them, appends them to the log, and broadcasts them.
• Snapshot worker compacts the log into snapshots asynchronously.

Edit flow:

sequenceDiagram
    participant A as Client A
    participant S as Collaboration server
    participant L as Operation log
    participant B as Client B
    A->>A: apply edit locally
    A->>S: op (base_rev 41)
    S->>S: transform against ops 42..n if any
    S->>L: append as rev 42
    L-->>S: durable
    S-->>A: ack rev 42
    S->>B: op rev 42
    B->>B: transform against pending local ops, apply

Deep dive

Concurrency control: operational transformation vs CRDTs

Locking a document is unusable for real-time editing, and last-writer-wins would discard text. Two families of algorithms merge concurrent edits.

Operational transformation (OT). Edits are expressed as operations such as insert(pos, text) and delete(pos, len). When an operation was created against an older document state, it is transformed against the operations that happened concurrently so that its position is adjusted. Example: the document is "abc". User A inserts "X" at position 1; concurrently user B deletes the character at position 2. Applying B's delete as-is after A's insert would remove the wrong character. Transforming B's delete against A's insert shifts it to position 3.

OT in Google Docs style uses a central server that assigns a total order. The client sends operations tagged with the revision they were based on. The server transforms the incoming operation against every operation committed after that revision, applies it, and broadcasts. Clients keep at most one operation in flight and buffer others, which keeps the client logic simple. The transformation functions must satisfy convergence properties, and designing them for rich operation sets is hard to get right.

CRDTs (sequence CRDTs). A CRDT gives every character a unique, ordered identifier so that inserts and deletes commute without transformation. Examples are RGA, Logoot, and Yjs/Automerge-style structures. Any replica can apply operations in any order and converge, so a central sequencer is not required, which suits peer-to-peer and offline use. The cost is metadata: each character carries an identifier, and deleted characters may remain as tombstones until garbage-collected.

A minimal transform for two concurrent inserts shows the mechanism. If both insert at the same position, a tie-break (here, a site ID) decides the order.

public class Main {
    record Insert(int pos, String text, int site) {}

    /** Transform op so it can be applied after other, where both were made on the same base document. */
    static Insert transform(Insert op, Insert other) {
        boolean shift = other.pos() < op.pos() || (other.pos() == op.pos() && other.site() < op.site());
        return new Insert(shift ? op.pos() + other.text().length() : op.pos(), op.text(), op.site());
    }

    static String apply(String doc, Insert op) {
        return new StringBuilder(doc).insert(op.pos(), op.text()).toString();
    }

    public static void main(String[] args) {
        var base = "abc";
        var a = new Insert(1, "X", 1);
        var b = new Insert(1, "Y", 2);

        // Replica 1 applies a, then b transformed against a.
        var r1 = apply(apply(base, a), transform(b, a));
        // Replica 2 applies b, then a transformed against b.
        var r2 = apply(apply(base, b), transform(a, b));
        System.out.println(r1 + " " + r2 + " converged=" + r1.equals(r2));
    }
}

aXYbc aXYbc converged=true

Both replicas end with the same text. The tie at position 1 is broken by the site id: the insert from the lower site id goes first. Java strings are immutable, so apply builds a new string with StringBuilder.insert. The sketch counts positions in UTF-16 code units, which equals characters for the ASCII text used here. A production editor must also define positions for characters outside the Basic Multilingual Plane.

Ordering, persistence, and durability

The collaboration server for a document is the single writer for that document's log, which gives a total order without distributed consensus per keystroke. The server appends the operation to the durable log before acknowledging it. Operations are batched in small groups to limit write rate. Snapshots are written every N operations or every few minutes.

If the collaboration server fails, the router reassigns the document to another server, which loads the latest snapshot and replays the log. Clients reconnect and resend unacknowledged operations along with their last known revision; the server transforms or deduplicates them by client-assigned operation ID. Coordination of ownership uses a lease in a coordination service (see leader election) so two servers never both sequence the same document; a fencing token on log appends protects against a stale owner.

Presence and cursors

Cursor positions are ephemeral and high frequency. They are not written to the log. The collaboration server keeps them in memory, broadcasts them with throttling, and expires them on disconnect. Cursor positions must be transformed through concurrent edits the same way operations are, so a remote cursor stays attached to the intended text.

Offline editing and version history

With OT, the client on reconnect sends its buffered operations based on the last synced revision, and the server transforms them against everything committed meanwhile. Long offline periods create large transform chains and are a reason some products limit offline editing. With CRDTs, offline replicas merge state directly. Version history is a replay of the log up to a chosen revision, or the nearest snapshot plus a partial replay; named versions are pointers to revisions.

Sizing and thresholds

• Snapshot policy. Snapshot every 1,000 operations or 10 minutes, whichever comes first. A load then replays at most about 1,000 ops x 100 B = 100 KB, which takes milliseconds. Keep the last few snapshots and move older log segments to cold storage after about 30 days, keeping named versions.
• Consistency model. Each document has a linearizable order (a single sequencer per document), and each client sees its own edits immediately (local apply). There is no ordering guarantee across documents, which is fine because there are no cross-document transactions.
• Ownership. The collaboration server holds a lease of about 10 seconds renewed every 3 seconds. Failover therefore costs up to one lease TTL plus a snapshot load. The alternative is a Raft group per document, which fails over faster but multiplies replicas and coordination for millions of mostly idle documents; a lease plus a replicated log is cheaper and sufficient.
• Hot documents. With n interactive editors, each op fans out to n - 1 clients. 1,000 editors at 1 op/s each is about 1 million messages/s for one document, which one server cannot handle. Cap interactive editors at around 100 and admit further users as read-only viewers served from a fan-out tier (pub/sub plus cache) that receives ops from the sequencer once.

Bottlenecks and scaling

• Hot documents. A document with hundreds of editors concentrates load on one server. Limit editors per document, throttle presence, and batch broadcasts.
• Connection count. Scale gateways horizontally; they are stateless apart from the connection. Cross-server fan-out (for example, viewers connected to different gateways) uses pub/sub keyed by document.
• Log growth. Snapshotting and compaction bound replay time. Old operations move to cold storage.
• Sharding. Metadata and logs are partitioned by doc_id; there are no cross-document transactions, so partitioning is straightforward (see sharding).

Failure modes

• Collaboration server crash. Clients keep typing locally and show an offline indicator. After the lease expires, another server loads snapshot plus log and clients resend unacknowledged ops with their operation IDs, which are deduplicated. No acknowledged edit is lost because the ack follows the durable append. Expect a pause of roughly 10 to 15 seconds.
• Log store unavailable or partitioned. The server cannot commit, so it stops acknowledging. Clients degrade to local-only editing and buffer ops (cap the buffer, for example at a few thousand ops, then make the document read-only until sync) rather than accept edits that cannot be made durable.
• At 10x scale. 100 million concurrent connections need about 2,000 gateways, and 30 million ops/s needs thousands of log shards. Shard and lease-coordinator load grows with the number of active documents, so the coordination service must be sharded too. Snapshot backlog lengthens replay times, so trigger snapshots by log length, not only by time.

Trade-offs and follow-up questions

• OT vs CRDT. Central sequencing with OT keeps metadata small; CRDTs remove the central dependency at a memory cost.
• Latency vs durability. Acknowledging after the log write adds a round trip; acknowledging before risks losing an acknowledged edit on failure.
• Follow-ups: How would you implement undo in a collaborative setting (undo only your own operations, transformed against later ones)? How do comments anchor to text ranges that shift? How do you enforce permission changes on live connections? How would you support 10,000 simultaneous viewers of a document (read-only fan-out via cache and pub/sub)?
• Two users type at the same position simultaneously? Both ops are transformed against each other and ordered by the sequencer (or by a site ID tie-break), so both insertions survive and all replicas show the same order.
• A client is offline for a day? With OT, the transform chain equals the number of ops committed meanwhile, which can be thousands, so cap it and fall back to a three-way merge or a conflict copy. With a CRDT, the replica merges state.