Design Google Drive

A cloud file storage service stores users' files, keeps them synchronized across many devices, and lets users share them. The main problems are efficient transfer of large files that change slightly, keeping metadata consistent across devices, detecting and resolving conflicting edits, and storing petabytes durably at low cost.

Requirements

Functional

• Upload and download files of any type (up to a few GB each).
• Sync files automatically across a user's devices.
• Share files and folders with other users with permissions (view, edit).
• Keep file version history and allow restore.
• Notify clients when files change.

Non-functional

• Reliability and durability: file data must never be lost or corrupted.
• Fast sync: small edits must not re-upload whole files.
• Strong consistency for metadata within a user's view: a client must not see a partial or mixed state of a folder. Availability may be sacrificed briefly for this.
• Scalability to hundreds of millions of users.
• Low bandwidth use, especially on mobile.

Out of scope: real-time collaborative editing (see Design Google Docs), full-text search of contents.

Estimation

Assumptions: 50 million registered users, 10 million DAU, 10 GB of quota each on average with 50% used, 2 file changes per active user per day, average file 500 KB.

• Total storage: 50M x 5 GB = 250 PB before replication overhead.

• Upload rate: 10M x 2 = 20M changes/day, about 230 changes/s average, roughly 700/s at peak.

• Upload bandwidth: 230 x 500 KB is about 115 MB/s average. Block-level sync with dedup reduces this considerably for edits of existing files.

• Metadata: 50M users x an assumed 200 files = 10 billion file records. At about 500 bytes each, that is 5 TB, which requires sharding but not exotic storage.

• Reads are similar in volume to writes on the sync path, because every change is downloaded by the user's other devices.

• Block count: 250 PB / 4 MB ≈ 6 x 10^10 blocks before dedup. At about 100 bytes per block index entry that is about 6 TB, so the block index is itself a sharded store.

• Metadata QPS: each sync involves roughly 1 init, 1 commit, and several /changes polls, so about 700 x 3 ≈ 2,000 metadata requests/s at peak for uploads, plus 10M devices polling. If every DAU device long-polls once per minute as a fallback, that is about 170,000 requests/s, which dominates the write path.

• Raw upload volume: 700/s × 500 KB peak ≈ 350 MB/s; with 3x replication or 1.5x erasure coding, disk ingest is 0.5 to 1 GB/s.

What each number implies: 250 PB rules out anything but object storage with erasure coding (1.5x overhead versus 3x saves about 375 PB of raw disk). 2,000 writes/s of metadata is modest, so sharding is for the 5 TB size and isolation, not write rate. Notifications and polling, not uploads, are the biggest request load, so the notification tier must be cheap (long-poll with a signal, no payload).

Conclusion: file bytes dominate storage and belong in blob storage. Metadata is small but requires consistency, so it belongs in a sharded transactional database.

API design

POST /v1/files/init                    { path, size, blockHashes[] } -> { fileId, missingBlocks[] }
PUT  /v1/blocks/{hash}                 (block bytes)                 -> 200
POST /v1/files/{fileId}/commit         { baseVersion, blockList[] }  -> { version } | 409 conflict
GET  /v1/files/{fileId}?version=N      -> file metadata with block list
GET  /v1/blocks/{hash}                 -> block bytes
GET  /v1/changes?cursor=C              -> list of changes after cursor C, new cursor
POST /v1/files/{fileId}/share          { userId, role }

baseVersion on commit makes the update an optimistic compare-and-set: it succeeds only if the file has not changed since the client last saw it. The cursor is a monotonically increasing per-user change sequence number.

Data model

Separate metadata from content.

Reasoning: metadata operations need transactions (rename plus version bump, move between folders), so a relational database with ACID guarantees is used, sharded by user so that one user's operations stay on one shard (see sharding and partitioning). Shared folders complicate this because permissions cross shards, so sharing is stored as a reference entry on each participant's shard. Block bytes go to object storage.

High-level design

graph TD
    CL["Client app (watcher, chunker, local DB)"] --> LB["Load balancer"]
    LB --> API["Metadata / API servers"]
    LB --> BS["Block servers"]
    API --> MDB[("Metadata DB (sharded)")]
    API --> CACHE["Metadata cache"]
    BS --> OBJ["Object storage (blocks)"]
    BS --> BDB[("Block index")]
    API --> MQ["Change queue"]
    MQ --> NS["Notification service"]
    NS -->|"long poll / push"| CL
    OBJ --> COLD["Cold storage tier"]

Responsibilities:

• Client. Watches the local folder, splits changed files into blocks, computes hashes, keeps a local database of the last synced state, and syncs.
• Block servers. Receive block uploads, verify hashes, compress and encrypt blocks, write to object storage, and record them in the block index.
• Metadata servers. Validate commits, update file and version records atomically, and append to the user's change log.
• Notification service. Tells other devices that a new change exists. Clients then call /changes.
• Object storage. Durable storage of blocks.

Upload flow

sequenceDiagram
    participant C as Client
    participant M as Metadata server
    participant B as Block server
    participant N as Notification service
    participant D as Other device
    C->>C: split file into blocks, hash each
    C->>M: init (path, block hashes)
    M-->>C: missing blocks only
    C->>B: upload missing blocks
    B-->>C: ok
    C->>M: commit (baseVersion, block list)
    M->>M: compare-and-set version, append change
    M-->>C: new version
    M->>N: change event for user
    N-->>D: notify
    D->>M: GET changes since cursor
    D->>B: download missing blocks

Deep dive

1. Block-level chunking, delta sync, and deduplication

Uploading a whole file after each edit wastes bandwidth. The client splits files into blocks (for example about 4 MB each), and each block is identified by a cryptographic hash of its content (SHA-256).

• Only blocks whose hash the server does not already hold are uploaded. Editing a small part of a large file changes one or a few blocks.
• Deduplication. Because the block key is its content hash, identical blocks across files, versions, and even users are stored once. ref_count tracks how many file versions reference a block, and a block is garbage-collected when the count reaches zero.
• With fixed-size blocks, inserting bytes at the start shifts every later boundary and changes all block hashes. Content-defined chunking places boundaries where a rolling hash matches a pattern, so an insertion changes only nearby blocks. It costs more CPU on the client.
• Cross-user deduplication saves space but lets an attacker test whether a given file exists in the system by observing whether the upload is skipped. Some services limit dedup to within an account for this reason.
• Compress blocks before upload, and encrypt them at rest.

Numbers: with 4 MB blocks, a 1 GB file is 250 blocks and a small edit uploads 4 MB, not 1 GB. With 1 MB blocks the same edit uploads 1 MB but the file has 1,000 block entries per version, four times the metadata. Content-defined chunking typically targets an average of a few MB with a minimum and maximum (for example 1 and 8 MB) so a pathological input cannot produce tiny or huge blocks. Upload blocks in parallel (4 to 8 connections) and make each block upload idempotent by its hash, so a retry after a timeout is harmless. Choose the chunk size by the file mix: 4 MB suits documents and photos; very large files favor bigger blocks.

Alternative to blocks, the rsync-style delta (send a diff against the server's copy), needs the server to keep and read the old file and does not dedup across files, which is why block hashes are preferred for a store that keeps versions.

Trade-off: small blocks give better dedup and smaller deltas but more metadata and requests. Large blocks reduce overhead but transfer more on edits.

2. Sync protocol and change detection

Each user has a monotonically increasing change sequence. Every commit appends an entry to it. A device stores the last sequence number it processed (its cursor).

1. A device receives a hint that changes exist, by long poll or a push channel (see WebSockets, SSE, and long polling).
2. It calls GET /changes?cursor=C and receives all entries after C.
3. For each entry it compares the version with its local state and downloads the missing blocks.
4. It stores the new cursor.

The commit step checks that every block in the submitted block list exists in the block index, and increments each block's ref_count in the same operation that creates the version row. A commit that lists a missing block is rejected.

Choices: long polling (hold the request for about 30 to 60 seconds, return early on a change) works through proxies and firewalls and needs no persistent socket protocol; WebSockets lower per-message overhead but cost more to operate at millions of connections. At 10 million concurrent devices with roughly 10 to 50 KB of memory per connection, the notification tier needs 100 to 500 GB of RAM, so tens of connection servers. The change log is per user (sequence numbers only need to be ordered within one user's shard, avoiding a global counter). Consistency: commits are linearizable on the owner's shard through the compare-and-set; other devices see the change after the notification and the /changes read, typically within seconds; cache metadata with invalidation on commit and a TTL of tens of seconds as a backstop. For shared folders, keep the change log in the folder owner's shard and fan out references to members, since a per-user log alone cannot order edits by two users.

Notifications carry no data, only a signal, so a lost notification is corrected by the next poll, and clients also poll at a low frequency as a fallback. Because the cursor makes the protocol idempotent, re-applying a change list is safe.

3. Conflict resolution

Two devices may edit the same file starting from the same version. The first commit wins and advances the version. The second commit carries an out-of-date baseVersion and receives a conflict response. The client then:

• keeps both versions: the server version stays as the file, and the local edit is saved as a separate "conflicted copy" file, or
• for known mergeable formats, attempts a merge.

Automatic merging of arbitrary binary files is not possible, so preserving both copies avoids data loss. Real-time editors use operational transformation or CRDTs instead (see CRDTs).

Offline edits create longer conflict windows. The client queues changes locally and reconciles when reconnected, using the same compare-and-set commit.

4. Versioning, deletion, and storage cost

Each commit creates a new file_versions row that references blocks, so unchanged blocks are shared between versions at no extra cost. Retention policy: keep the last N versions or versions within a time limit, and expire older ones by decrementing block reference counts. Deletes are soft (is_deleted flag) for a recovery period, then physical. Block deletion runs as an asynchronous garbage collection job that removes unreferenced blocks after a grace period, protecting against races with concurrent uploads that are about to reference the block.

Cold data (old versions, files not accessed for a long time) moves to cheaper storage tiers and uses erasure coding instead of full replication to cut storage overhead.

5. Sharing and permissions

A share adds a permission entry. Access checks read the permission on the file and its ancestors, so folder sharing grants access to descendants. Permission checks are cached, and revocation must invalidate cached entries. Downloads of shared content can use short-lived signed URLs to the block store.

Bottlenecks and scaling

• Metadata DB. Shard by user, cache hot reads, and use read replicas for listing operations (see database replication). Large shared folders are hot spots and may need separate handling.
• Block store. Partition by block hash, which distributes evenly. Object storage scales horizontally.
• Notification fan-out. Millions of open connections require a dedicated tier of connection servers. A shared folder change fans out to all members' devices.
• Large files. Parallel block upload and resumable sessions handle interruptions.
• Client load. Initial sync of a large library is throttled and prioritized so it does not saturate the device or network.
• Durability. Blocks are replicated across zones or erasure coded, and periodic integrity scans compare stored blocks with their hashes.

Failure modes

The critical safety rule is that a commit must never reference a block that could be collected, so the ref-count increment and the version insert share a transaction and GC only removes blocks with zero references past the grace period. At 10x (500 million users, 2.5 EB): erasure coding and cold tiering stop being optional, the block index (60 TB) and metadata (50 TB) need hundreds of shards with automatic rebalancing, popular shared folders need dedicated shard handling, and notification connection servers number in the hundreds; a regional deployment per user home region limits cross-region traffic.

Trade-offs and follow-up questions

• Fixed-size versus content-defined chunking: simplicity versus delta efficiency.
• Cross-user dedup: storage savings versus privacy leakage.
• Strong consistency for metadata reduces availability in a partition. Blocks are immutable and content-addressed, so they have no consistency problem.
• How to support end-to-end encryption? Blocks are encrypted with client-held keys, which disables server-side dedup across users and server-side content features.
• How to limit sync storms after a shared folder with many files changes? Batch changes and coalesce notifications.
• What if two devices edit the same file offline? Both commit with the same baseVersion; the first wins and the second becomes a conflicted copy. Nothing is overwritten.
• What if a user renames a folder with 100,000 files? Store parent pointers (parent_id), so rename is one row update, not 100,000; change events reference the folder and clients apply the move locally.
• What if a 50 GB file upload fails at 99 percent? Blocks already uploaded stay; the client resumes with init and uploads only the missing blocks.
• What if we must support a shared folder with 100,000 members? Fan out change notifications through a queue in batches, rate limit sync, and store the permission once on the folder, with per-user references cached lazily.
• What if the same file is uploaded by a million users (a viral PDF)? Cross-user dedup stores it once, at the cost of the existence-leak risk noted above; a hot block is served from CDN or cache.
• How to detect and repair corrupted blocks? Verify hash on read, scrub in the background, and repair from replicas.