Caching Fundamentals and Strategies

A cache is a small, fast storage layer that holds copies of data whose authoritative version lives in a slower store. Reads that find the data in the cache (a hit) skip the slower store. Reads that do not (a miss) fall through to it. Caching reduces latency, reduces load on the backing store, and increases the throughput a system can sustain.

Caching works because access patterns are skewed. A small fraction of keys receives most of the reads (temporal and popularity locality), so a cache holding a small fraction of the data can serve a large fraction of requests.

Cache layers

A request can pass through several caches. Each layer trades capacity and freshness control for proximity to the client.

Layer	Location	Typical contents	Invalidation control
Browser cache	client	static assets, API responses with cache headers	limited (TTL, versioned URLs)
CDN	edge servers	static files, cacheable HTML/API responses	purge APIs, TTL
Reverse proxy / gateway cache	in front of services	full HTTP responses	TTL, purge
Application in-process cache	inside the service process	parsed config, small hot lookups	full, but per instance
Distributed cache	Redis, Memcached cluster	query results, sessions, computed objects	full
Database buffer cache	database memory	table and index pages	managed by the database

graph LR
    C["Client + browser cache"] --> CDN["CDN"]
    CDN --> GW["Gateway / reverse proxy"]
    GW --> APP["Service with in-process cache"]
    APP --> DC["Distributed cache"]
    DC --> DB["Database + buffer cache"]

An in-process cache has the lowest latency (a memory read, no network hop) but each instance holds its own copy, so copies diverge and total memory is multiplied by the instance count. A distributed cache adds a network round trip (roughly sub-millisecond within a data center) but is shared by all instances.

Hit ratio and effective latency

The hit ratio is hits / (hits + misses). Average read latency is:

Code

T_avg = h * T_cache + (1 - h) * (T_cache + T_store)

With a cache read of 1 ms, a database read of 10 ms, and h = 0.9: T_avg = 1 + 0.1 * 10 = 2 ms. At h = 0.5 it is 1 + 5 = 6 ms. The miss path dominates as the hit ratio falls, and the load reaching the database is (1 - h) of the read traffic. Raising the hit ratio from 90% to 99% cuts database reads by 10 times.

Factors that determine the hit ratio:

Cache size relative to the working set (the set of keys actively requested).
Access skew. Uniformly random access gives a hit ratio close to cache size / data size.
Eviction policy. See cache eviction policies.
TTL and invalidation. Aggressive invalidation lowers the hit ratio.
Key design. Keys that include unnecessary variation (timestamps, session ids) prevent sharing.

Read strategies

Cache-aside (lazy loading)

The application manages the cache explicitly.

Read the key from the cache.
On a hit, return the value.
On a miss, read from the database, write the value into the cache with a TTL, and return it.

sequenceDiagram
    participant A as Application
    participant C as Cache
    participant D as Database
    A->>C: GET user:42
    C-->>A: miss
    A->>D: SELECT user 42
    D-->>A: row
    A->>C: SET user:42 (TTL 300 s)
    A-->>A: return row

Properties: only requested data is cached; a cache outage degrades to database reads (if the database can bear the load); the data model in the cache can differ from the database's. The first read of each key is always a miss. Between a database update and the cache refresh, readers can see stale data.

Read-through

The cache itself loads missing data. The application calls only the cache; on a miss, the cache (or its client library) fetches from the database, stores the result, and returns it. The logic is the same as cache-aside, but it lives in the cache layer, which keeps application code simple and centralizes loading. It requires a cache that supports loader plug-ins.

Cache-aside and read-through differ only in who performs the load. In both, a miss costs one cache lookup plus one database read, and concurrent misses on the same key each query the database (see cache invalidation, stampede, and hot keys).

Write strategies

Strategy	Write path	Read after write	Durability	Write latency
Write-through	write cache and database synchronously	always fresh	database has it before acknowledgement	higher (two writes)
Write-back (write-behind)	write cache, acknowledge, flush to database later	fresh from cache	data loss if cache fails before flush	lowest
Write-around	write database only, bypass cache	miss until next read loads it	database	database latency
Cache-aside with invalidation	write database, delete cache key	miss then reload	database	database latency

Write-through

Every write goes to the cache and the database as part of the same operation, and the write is acknowledged after both succeed. The cache always matches the database, so reads after writes hit. Cost: write latency includes both stores, and data that is written but never read occupies cache space.

Write-back

The write is acknowledged after updating the cache. The cache flushes dirty entries to the database asynchronously, often batching or coalescing multiple writes to the same key. This absorbs write bursts and reduces database writes. Risk: if the cache node fails before the flush, acknowledged writes are lost, unless the cache persists a log. Ordering and failure handling of the asynchronous flush must be designed explicitly.

Write-around

Writes go directly to the database. The cache is populated only when a read misses. This avoids filling the cache with data that is written once and rarely read (logs, bulk imports). The cost is a miss on the first read after each write.

Update vs invalidate on write

With cache-aside, a write can either update the cache entry or delete it. Deleting is the usual choice. Two concurrent writers can update the database in one order and the cache in the opposite order, leaving a permanently stale value. A delete cannot leave a wrong value, only an absent one, and the next read repopulates it. A race still exists: a reader that missed before the write can load the old row and store it after the delete. The interleaving is: reader misses and reads the old row from the database, writer updates the database and deletes the key, reader then writes the old row into the cache. The cache now holds the old value until its TTL expires or the next invalidation. TTLs bound how long such an entry survives. See cache invalidation, stampede, and hot keys.

TTL

A time to live (TTL) is an expiry time attached to a cache entry. After it elapses the entry is treated as absent. TTL is the last-resort consistency mechanism: even if an invalidation is lost, staleness is bounded by the TTL. Choose it from the tolerated staleness of the data, not from cache size. Add a random offset to TTLs of entries created together, so they do not all expire at once.

Choosing a strategy

Read-heavy data with tolerable staleness: cache-aside with TTL.
Data that must be read immediately after being written, with modest write rate: write-through.
Very high write rates on data that can tolerate loss of the last few updates (counters, view counts): write-back.
Large volumes of write-once data: write-around.
Do not cache data that is rarely re-read, changes on nearly every read, or must always be strongly consistent.

Interview questions

Why is cache-aside more common than read-through? It needs no special cache features, works with any store such as Redis or Memcached, and the application controls what is cached and how it is keyed.
What is the risk of write-back? Acknowledged writes exist only in the cache until flushed, so a cache failure loses them. It is acceptable only for data where that loss is tolerable, or when the cache has a durable log.
Why delete rather than update the cache entry on a write? Concurrent writers can apply cache updates in a different order than database updates, leaving a stale value indefinitely. A deleted key is repopulated from the database on the next read.
How does hit ratio affect database load? Database read load is proportional to 1 - h. Going from a 90% to a 99% hit ratio reduces database reads tenfold.