Design WhatsApp

A messaging service delivers messages between users in near real time, stores them until the recipient is online, and reports delivery status. Compared to the general chat system design, this article focuses on the scale characteristics of a WhatsApp-style product: a very large number of persistent connections, transient server-side storage, receipts, groups, and end-to-end encryption.

Requirements

Functional

• One-to-one messaging and group chats (bounded group size).
• Message status: sent, delivered, read.
• Delivery to offline users when they reconnect.
• Media messages (images, video, documents).
• Presence (online, last seen) and typing indicators.
• Multi-device support.
• End-to-end encryption.

Non-functional

• Low delivery latency (well under a second when both are online).
• No message loss; at-least-once delivery with deduplication gives effectively-once display.
• Per-conversation ordering.
• High availability and horizontal scalability.
• Server holds messages only until delivered (a design choice that reduces storage).

Estimation

Assumptions: 2 billion registered users, 500 million concurrent connections at peak, 100 billion messages per day.

• Message rate: 100 x 10^9 / 86,400 ≈ 1.16 million messages/s average; peak about 3 million/s.
• Bandwidth: text message payload about 200 bytes with envelope: 1.16 x 10^6 x 200 B ≈ 230 MB/s average.
• Connections: 500 million concurrent. At 100,000 connections per gateway server (possible with tuned event-driven servers, memory around tens of KB per connection), 5,000 gateway servers.
• Offline queue: assume 10% of users are offline at a time and receive 50 messages per day each of 200 B: 200 x 10^6 x 50 x 200 B = 2 TB. Small, because messages are deleted after delivery.
• Media: assume 10% of messages carry media averaging 200 KB: 10 x 10^9 x 200 KB = 2 PB/day. Media dominates storage and bandwidth and goes to object storage and a CDN, not through chat servers.

What each number implies:

• 500 million connections and 5,000 gateways: the session registry holds 500 million entries at about 100 bytes, roughly 50 GB, so it is sharded over 10 to 20 in-memory nodes. Refreshing a TTL on every heartbeat (say every 30 s) would be about 16 million registry writes per second, so do not. Heartbeats terminate at the gateway. Write the registry on connect and disconnect only (assume 5 reconnects per device per day: about 30,000 writes per second), and protect against a dead gateway with a per-gateway lease rather than per-device TTLs.
• 1.16 million messages/s (3 million peak): every message is persisted then deleted on ack, so the pending store sees about 3 million writes plus 3 million deletes per second at peak. At about 50,000 operations per second per node, that is roughly 60 nodes per replica, about 180 with 3 replicas.
• Each message generates about 3 frames (message, delivered, read): internal event rate is about 3.5 million per second, so acks should be batched per connection.
• 2 TB offline queue: small; retention is bounded by a TTL (for example 30 days) rather than capacity.

API design

Clients speak a persistent binary protocol over TCP with TLS (WhatsApp historically used a customized XMPP-derived protocol; WebSocket is the common choice elsewhere). Logical operations:

send_message   {client_msg_id, to, ciphertext, type}        -> {server_msg_id, timestamp}
message        {server_msg_id, from, ciphertext}            (server -> client)
ack            {server_msg_id, status: delivered | read}
presence       {state}
group_create   {name, members}
upload_media   (HTTPS, returns media_id + decryption key sent inside the message)

client_msg_id is generated by the sender and used for deduplication when a send is retried.

Data model

Because the server does not need long-term history (clients keep chat history locally, and backups are user-controlled), the pending-message store is a write-heavy queue with short retention. Key-value or wide-column stores partitioned by recipient fit this access pattern.

High-level design

flowchart LR
    A[Sender device] <-->|persistent connection| GA[Gateway A]
    B[Recipient device] <-->|persistent connection| GB[Gateway B]
    GA --> MS[Message service]
    MS --> SR[(Session registry)]
    MS --> PQ[(Pending message store)]
    MS -->|route| GB
    MS --> Push[Push notification service]
    Push --> APNs[APNs / FCM]
    A -->|HTTPS| Media[Media service]
    Media --> Obj[(Object storage)]
    MS --> GS[Group service]
    GS --> GDB[(Group store)]

Components

• Gateways hold long-lived connections, authenticate devices, and forward frames. They are the only components aware of sockets.
• Session registry maps each connected device to its gateway.
• Message service assigns server IDs and timestamps, persists messages, and routes them to the recipient's gateway.
• Pending message store holds messages for offline devices.
• Push service wakes offline mobile clients through platform push channels (APNs, FCM). The push carries no message content.
• Media service stores encrypted blobs; the message carries a reference and key.

Delivery with receipts:

sequenceDiagram
    participant A as Sender
    participant GA as Gateway A
    participant M as Message service
    participant GB as Gateway B
    participant B as Recipient
    A->>GA: send_message (client_msg_id)
    GA->>M: forward
    M->>M: dedupe, persist pending
    M-->>A: sent (single check)
    M->>GB: route via session registry
    GB->>B: message
    B-->>GB: ack delivered
    GB->>M: delivered
    M->>M: delete pending
    M-->>A: delivered (double check)
    B-->>M: read (when opened)
    M-->>A: read

Deep dive

Connection management and routing

Each client keeps one connection to a gateway. When it connects, the gateway writes device_id -> gateway_id to the session registry with a TTL refreshed by heartbeats (see WebSockets and heartbeats). To deliver, the message service looks up the recipient's gateway and forwards the frame to it over an internal channel. If no entry exists, the recipient is offline and the message stays in the pending store while a push notification is sent.

Gateways are stateless with respect to messages, so a crashed gateway only drops connections; clients reconnect to another gateway through the load balancer and drain pending messages. Connection load is balanced with least-connections; graceful deploys drain connections gradually to avoid reconnect storms (see timeouts, retries, backoff and jitter).

Reliability, ordering, and deduplication

Delivery is at-least-once: a message stays in the pending store until the recipient acks it. Retries of a send by the sender reuse client_msg_id, and the server deduplicates on (sender, client_msg_id) (delivery semantics and idempotency). The recipient client deduplicates by server_msg_id. Ordering is preserved per conversation by assigning a monotonically increasing sequence number per conversation (or per sender-recipient pair) at the message service (conversations are partitioned so that one node assigns the numbers for a given conversation), and the client displays messages by that order. Global ordering is not required.

Group messaging

Groups are handled by server-side fan-out: the sender submits the message and the server copies it to each member. With end-to-end encryption, the sender either encrypts a separate ciphertext for each recipient device and uploads all of them, or encrypts once with a key shared by the group members (a sender-key scheme) and uploads a single ciphertext. The group service returns the member list; the message service enqueues one delivery per member. Group size limits bound the fan-out per message. For very large groups the write amplification of per-member pending entries becomes the limit, which is why groups have caps.

Numbers and alternatives for groups: cap membership (for example 1,024). Store one ciphertext per group message and one small pending entry (a pointer of about 50 bytes) per member, so a 256-member group message costs about 13 KB of entries rather than 256 copies of the payload. This assumes a sender-key scheme; with per-device pairwise encryption, storage is one ciphertext per device. For very large one-to-many channels, fan-out on write is too costly, so store the message once and let clients pull it (fan-out on read).

Ordering choice: a per-conversation sequence number assigned by the node that owns the conversation (partition by conversation_id) gives ordering at the price of a single writer per conversation. The alternative, client timestamps, breaks under clock skew. If the owner fails, its partition is taken over by another node, and sequence numbers continue from the last durable value.

End-to-end encryption and multi-device

With end-to-end encryption, only sender and recipient devices hold decryption keys; servers relay ciphertext. WhatsApp uses the Signal protocol: each device publishes an identity key and prekeys to the server; a sender uses them to establish a session (X3DH key agreement) and then ratchets keys per message (Double Ratchet), giving forward secrecy. Consequences for the design: the server cannot search or transform message content, cannot generate previews, and cannot rebuild history for a new device. Multi-device support requires encrypting a message separately for each of the recipient's devices, so the sender queries the recipient's device list and fans out per device. See encryption and TLS for the primitives.

Presence

Online status is derived from the session registry. Broadcasting every change to all contacts is too costly, so presence is fetched on demand when a chat is opened, or pushed only to users currently viewing that contact. Last-seen is a timestamp updated on disconnect, written lazily. Typing indicators are ephemeral, sent directly through the gateway path and never persisted.

Failure modes

• Gateway crash. About 100,000 clients reconnect. Spread over 30 seconds with jitter, that is about 3,000 connections per second (TLS handshake and authentication) landing on the rest of the fleet; keep at least 20% spare gateway capacity.
• Region failure. Hundreds of millions of reconnects can overwhelm authentication and the registry. Apply admission control (reject with a retry-after hint), and increase backoff windows to minutes.
• Stale registry entry. The message service routes to a gateway that no longer holds the device; the send fails or is not acked, so the message remains in the pending store and a push is sent. The pending store, not the registry, is the source of truth.
• Message service or pending store unavailable. The sender receives no "sent" tick and retries with the same client_msg_id, so no message is lost or duplicated. Write pending entries with a quorum of 2 of 3 replicas before acking the sender.
• Push provider outage. Messages wait in the pending store and are delivered when the app next connects; only the wake-up is delayed.
• At 10x message volume: the pending store grows to about 600 nodes per replica set; the registry needs more shards; acks must be batched; group fan-out becomes the dominant write load, so tighten group caps or move large groups to fan-out on read.

Bottlenecks and scaling

• Connection count. Scale gateways horizontally; tune the OS for many sockets and use event-driven I/O.
• Session registry load. Shard by user ID; entries are small and expire by TTL.
• Pending store writes. Partition by recipient; short retention keeps size bounded.
• Media. Upload directly to object storage, deduplicate identical encrypted blobs by hash only where encryption permits (per-message keys usually prevent this), and expire blobs after delivery.
• Reconnect storms. After a gateway or region failure, clients back off with jitter.
• Hot groups. Cap group sizes and process fan-out asynchronously.

Trade-offs and follow-up questions

• Server-side history vs transient relay. Transient storage lowers cost and privacy exposure but makes new-device history and backups a client responsibility.
• Push vs pull for offline users. Push is needed to wake the app, but delivery through platform push is best-effort, so the pending store remains authoritative.
• Follow-ups: How would you implement message search under E2EE (client-side index)? How do you handle a user changing phones (key change notification)? How would you add message edits or deletes for everyone? How would you scale to multi-region while keeping latency low (users pinned to a home region, cross-region routing for messages)?
• What if a device is offline for 40 days? Pending entries expire at the retention TTL; on reconnect the client detects a gap through sequence numbers and the sender's app may need to resend. Say this explicitly as a product decision.
• What if the recipient acks but the ack is lost? The message stays pending and is redelivered; the client deduplicates by server_msg_id and re-acks.
• When is a message "delivered" with several devices? Choose "delivered to at least one device". Each device has its own pending entry, deleted independently on its own ack.
• How do you send a "read" receipt in a large group? Aggregate per message and report counts, or disable receipts above a size threshold, because each receipt is itself a fan-out.
• Why not one TCP connection per chat? One connection per device multiplexes all conversations, keeping 500 million sockets, not billions.