Lesson 126 of 129
System Design Numbers and Cheat Sheet
This page collects reference values and comparison tables used in system design discussions. All numbers are orders of magnitude and vary with hardware, configuration, and workload. Use them for estimation and sanity checks, not as guarantees.
flowchart LR
C["Client"] --> CDN["CDN"]
CDN --> LB["Load balancer"]
LB --> APP["Stateless services"]
APP --> CA[("Cache")]
APP --> DB[("Database")]
APP --> Q["Queue or log"]
Q --> W["Workers"]
W --> DB
APP --> OS[("Object store")]Each component in this generic request path has a quick-facts section below: protocols (client to service), load balancing, cache, database, queue, and resilience patterns between services.
Latency numbers
| Operation | Approximate time |
|---|---|
| L1 cache reference | 1 ns |
| Main memory reference | 100 ns |
| SSD random read | 100 microseconds |
| Read 1 MB sequentially from memory | tens to a few hundred microseconds |
| Read 1 MB sequentially from SSD | about 1 ms |
| Round trip within a data center | 0.5 ms |
| HDD seek | about 10 ms |
| Round trip between continents | 100 to 150 ms |
Ratios: memory is about 1,000 times faster than SSD for random reads. A cross-region round trip is hundreds of times slower than one within a data center.
Data size reference
| Item | Approximate size |
|---|---|
| ASCII character | 1 byte |
| 64-bit integer | 8 bytes |
| UUID (binary) | 16 bytes |
| Short text record (tweet, chat message, URL mapping) | 100 bytes to 1 KB |
| Compressed photo | a few hundred KB to a few MB |
Unit and time conversions
| Item | Value |
|---|---|
| Seconds per day | 86,400 (round to 10^5) |
| Seconds per month | about 2.5 x 10^6 |
| Seconds per year | about 3 x 10^7 |
| 2^10 / 2^20 / 2^30 / 2^40 / 2^50 | KB / MB / GB / TB / PB (about 10^3 / 10^6 / 10^9 / 10^12 / 10^15 bytes) |
| 1 million requests per day | about 12 per second |
| 100 million requests per day | about 1,160 per second |
| 1 billion requests per day | about 11,600 per second |
Availability nines
| Availability | Downtime per year | Downtime per month |
|---|---|---|
| 99 percent | 3.65 days | about 7.2 hours |
| 99.9 percent | 8.8 hours | about 43 minutes |
| 99.99 percent | 52.6 minutes | about 4.3 minutes |
| 99.999 percent | 5.3 minutes | about 26 seconds |
Series availability multiplies. Parallel independent replicas: 1 - (1 - a)^n. See scalability, availability, and reliability.
Capacity rules of thumb
These are commonly used planning values. Measure before relying on them.
- A single relational database node serves on the order of thousands to low tens of thousands of simple queries per second.
- A single in-memory cache node (Redis, Memcached) serves on the order of 100,000 operations per second for small values.
- A single application server handles from hundreds to several thousand requests per second, depending on the work per request.
- A 1 Gbit per second network link carries about 125 MB per second.
- Peak traffic is commonly assumed to be 2 to 10 times the daily average.
- Storage is multiplied by the replication factor (commonly 3).
- Caching the hot 20 percent of data often captures most reads for skewed workloads.
Formulas
| Quantity | Formula |
|---|---|
| Average QPS | requests per day / 86,400 |
| Peak QPS | average QPS x peak factor |
| Storage | records per day x bytes per record x retention days x replicas |
| Bandwidth | QPS x bytes per request |
| Servers | peak QPS / QPS per server, plus headroom |
| Base-62 code space | 62^n (62^7 is about 3.5 x 10^12) |
| Bloom filter false-positive rate | about (1 - e^(-kn/m))^k for n items, m bits, k hash functions |
Details are in back-of-envelope estimation.
Protocol quick facts
| Protocol | Key properties | Use |
|---|---|---|
| TCP | Reliable, ordered, connection-oriented, congestion control. | Most application traffic. |
| UDP | Unreliable, no ordering, low overhead. | DNS queries, real-time media, QUIC underneath. |
| HTTP/1.1 | Text, keep-alive, one request at a time per connection. | Broad compatibility. |
| HTTP/2 | Binary, multiplexing over one TCP connection, header compression. | gRPC transport, web. |
| HTTP/3 | HTTP over QUIC (UDP), avoids TCP head-of-line blocking. | Lossy or mobile networks. |
| WebSocket | Full-duplex over one long-lived connection. | Chat, live updates. |
| SSE | Server-to-client stream over HTTP. | Notifications, feeds. |
| gRPC | HTTP/2 with Protobuf, streaming. | Internal service calls. |
See TCP and UDP, HTTP/1, HTTP/2, HTTP/3, and gRPC and Protobuf.
Database quick facts
| Type | Model | Strengths | Limits |
|---|---|---|---|
| Relational (PostgreSQL, MySQL) | Tables, SQL, joins, ACID. | Transactions, flexible queries. | Write scaling needs partitioning. |
| Key-value (DynamoDB, Redis) | Key to value. | Low-latency lookup by key, horizontal scaling. | Limited query patterns. |
| Document (MongoDB) | JSON-like documents. | Flexible schema, nested data. | Cross-document transactions and joins are limited. |
| Wide-column (Cassandra, Bigtable) | Partition key plus clustering columns. | High write throughput, scale-out. | Query-driven modeling. |
| Graph (Neo4j) | Nodes and edges. | Relationship traversal. | Harder to partition. |
| Time-series | Timestamped points. | Compression, retention, downsampling. | Specialized queries. |
| Search (Elasticsearch) | Inverted index. | Full-text search, ranking. | Not the primary source of truth. |
| Object store (S3) | Objects by key. | Cheap, durable large files. | No in-place update or low-latency small writes. |
| Structure | Read | Write | Notes |
|---|---|---|---|
| B-tree index | O(log n) |
O(log n) with in-place page updates |
Good for reads and range scans. |
| LSM tree | Checks memtable and multiple SSTables | Sequential appends, fast | Compaction cost, write amplification. |
| Hash index | O(1) lookup |
O(1) |
No range queries. |
See SQL vs NoSQL, database indexing, and storage engines: B-tree vs LSM tree.
Replication and consistency
| Concept | Fact |
|---|---|
| Quorum rule | R + W > N gives overlap between read and write sets. |
| Leader-follower | One writer; asynchronous replicas can serve stale reads. |
| Multi-leader | Writes accepted in several places; conflict resolution required. |
| Leaderless | Quorum reads and writes, read repair, hinted handoff. |
| CAP | During a partition, choose consistency or availability. |
| Majority quorum | Raft and Paxos tolerate f failures with 2f + 1 nodes. |
See quorums and replication protocols, CAP theorem and PACELC, and consensus: Paxos and Raft.
Cache quick facts
| Strategy | Behavior | Trade-off |
|---|---|---|
| Cache-aside | Application reads cache, on miss reads DB and fills cache. | Stale data possible; simple. |
| Read-through | Cache loads from DB on miss. | Application code simpler. |
| Write-through | Write to cache and DB together. | Higher write latency, consistent. |
| Write-back | Write to cache, flush to DB later. | Fast writes, risk of loss. |
| Write-around | Write to DB only. | Cold reads after writes. |
Eviction: LRU (recency), LFU (frequency), FIFO, TTL. See caching fundamentals and strategies and cache eviction policies.
Message queue and stream quick facts
| System | Model | Ordering | Retention | Typical use |
|---|---|---|---|---|
| Kafka | Partitioned distributed log; consumers track offsets. | Per partition. | Time or size based, replayable. | Event streams, high throughput pipelines. |
| RabbitMQ | Broker with exchanges and queues. | Per queue, with caveats under redelivery. | Classic queues delete a message after acknowledgment. | Task queues, routing. |
| SQS standard | Managed queue. | Best effort. | Bounded retention period. | Decoupled work queues. |
| SQS FIFO | Managed queue with ordering by group. | Per message group. | Bounded retention period. | Ordered processing with dedup. |
Delivery semantics: at-most-once (may lose), at-least-once (may duplicate, needs idempotent consumers), exactly-once (achieved through idempotency or transactions in a bounded scope). See Kafka internals and delivery semantics and idempotency.
Load balancing and scaling quick facts
- L4 balances by connection (IP and port); L7 balances by request content (path, header).
- Algorithms: round robin, weighted, least connections, IP hash, consistent hashing.
- Stateless services scale horizontally behind a load balancer; stateful stores scale by replication and partitioning.
- Consistent hashing moves about 1/N of keys when a node is added or removed from N nodes.
See load balancing and consistent hashing.
Resilience patterns
| Pattern | Purpose |
|---|---|
| Timeout | Bound the wait for a dependency. |
| Retry with exponential backoff and jitter | Recover from transient errors without synchronized retries. |
| Circuit breaker | Stop calling a failing dependency. |
| Bulkhead | Isolate resources per dependency. |
| Load shedding | Reject excess work to protect capacity. |
| Idempotency key | Make retries safe. |
Rate limiting algorithms
| Algorithm | Behavior |
|---|---|
| Token bucket | Allows bursts up to bucket size; refill at fixed rate. |
| Leaky bucket | Smooths output at a constant rate. |
| Fixed window | Counter per window; boundary bursts possible. |
| Sliding window log | Exact; stores timestamps. |
| Sliding window counter | Approximate; weights previous window. |
Interview questions
- Which numbers are most useful to memorize? Seconds per day (about 10^5), the memory versus disk versus network latency ordering, the nines table, and the power-of-two data sizes. These allow most estimates to be computed quickly.
- Why should the capacity rules of thumb be stated as assumptions? Real throughput depends on hardware, query cost, payload size, and configuration. Stating the assumption allows the interviewer to correct it and shows the estimate is provisional.
- How is the read/write ratio used? It splits total QPS into read and write load, which drives choices such as caching and read replicas for read-heavy systems and partitioning or log-structured storage for write-heavy systems.