Technology Choice Guide

This guide maps requirements to categories of technology and gives representative products. Each choice is driven by an access pattern, a consistency need, or a scale characteristic. Products are examples, not the only options. In interviews, name the category and its properties first and a product second.

flowchart TD
    A["Requirement"] --> B{"Need transactions and joins?"}
    B -->|"yes"| C["Relational database"]
    B -->|"no"| D{"Access pattern"}
    D -->|"lookup by key"| E["Key-value store"]
    D -->|"nested documents"| F["Document store"]
    D -->|"very high write volume, known queries"| G["Wide-column store"]
    D -->|"relationship traversal"| H["Graph database"]
    D -->|"text relevance"| I["Search index"]
    D -->|"large files"| J["Object storage"]

Databases and stores

Requirement	Suitable choice	Why	Caveat
Multi-row transactions, joins, constraints	Relational (PostgreSQL, MySQL)	ACID transactions, SQL, secondary indexes.	Scaling writes past one primary needs sharding.
Lookup by primary key at very high volume, predictable latency	Key-value (DynamoDB, Redis)	Data partitioned by key; access is a single-partition operation.	Queries other than by key need extra indexes or design.
Flexible, nested records read as a unit	Document store (MongoDB)	Embedding related data avoids joins.	Cross-document relationships are harder to keep consistent.
Very high write throughput, multi-node, known query patterns	Wide-column (Cassandra, Bigtable)	LSM storage, partition-per-key design, scale-out.	Queries must be modeled up front; limited joins.
Traversing relationships (friends of friends, recommendations)	Graph database (Neo4j)	Traversal is a first-class operation.	Partitioning graphs is difficult.
Metrics and sensor data with retention and downsampling	Time-series database	Time-ordered compression, retention policies.	Poor for general-purpose queries.
Analytical queries over large history	Columnar warehouse (BigQuery, Redshift, ClickHouse)	Column storage and scans over few columns.	Not for low-latency row updates.
Large binary files (images, video, backups)	Object storage (S3-style)	Cheap, durable, scalable, addressed by key.	No partial in-place update; higher latency than a database.
Full-text search and relevance ranking	Search engine (Elasticsearch, OpenSearch, Solr)	Inverted index with scoring.	Typically a secondary index fed from the source of truth.

See SQL vs NoSQL, key-value and document stores, wide-column, graph, and time-series stores, object and blob storage, and inverted index and full-text search.

Caching

Requirement	Suitable choice	Why
Reduce database read load and latency	In-memory cache (Redis, Memcached) with cache-aside	Memory access is orders of magnitude faster than disk.
Rich structures (sorted sets, counters, lists)	Redis	Server-side data structures.
Simple, multi-threaded string caching	Memcached	Minimal feature set, simple scaling by client-side hashing.
Static and cacheable content near users	CDN	Serves from edge locations, reduces origin load and latency.
Per-process hot data with no network hop	In-process cache	Fastest, but per-instance and hard to invalidate.
Sessions, counters, and rate-limit state with expiry	Redis with TTL	Atomic increments and per-key expiry in a shared store.

See caching fundamentals and strategies, Redis and Memcached, and content delivery networks.

Messaging and streaming

Requirement	Suitable choice	Why
Decouple producer and consumer, absorb bursts, work queue	Message queue (RabbitMQ, SQS)	Buffering, acknowledgments, competing consumers.
High-throughput event log, multiple independent consumers, replay	Kafka	Partitioned log with retention and offsets.
Fan-out of events to many subscribers	Pub/sub (SNS, Kafka topics, Google Pub/Sub)	Each subscriber receives a copy.
Stateful stream computation with windows	Flink, Kafka Streams	Windowing, watermarks, state.
Large scheduled batch computation	Spark, MapReduce-style jobs	Throughput over latency.
Reliable publish of a DB change to a broker	Transactional outbox or change data capture	Avoids dual writes.

See message queues vs pub/sub, Kafka internals, RabbitMQ, SQS, and broker comparison, and stream vs batch processing.

Networking and traffic

Requirement	Suitable choice	Why
Spread traffic over instances	Load balancer (L4 or L7)	Distribution and health checks.
Auth, routing, rate limiting at the edge	API gateway or reverse proxy	Centralized cross-cutting concerns.
Server push to browsers	WebSocket or SSE	Persistent connection; SSE is one-directional.
Efficient internal service calls, streaming	gRPC	Binary Protobuf over HTTP/2.
Public, cacheable resource API	REST over HTTP	Uniform interface, HTTP caching.
Clients need flexible field selection	GraphQL	Client specifies the shape of the response.

See load balancing, reverse proxy and API gateway, WebSockets, SSE, and long polling, gRPC and Protobuf, and GraphQL.

Coordination and consensus

Requirement	Suitable choice	Why
Leader election, distributed locks with leases, configuration	etcd, ZooKeeper, Consul	Consensus-backed linearizable store.
Membership and failure detection at scale	Gossip protocol	Decentralized dissemination.
Unique identifiers without a central database	Snowflake-style IDs, UUIDs	Local generation, roughly time-ordered for Snowflake-style.
Distributed transaction across services	Saga	Local transactions with compensations, no global lock.

See leader election, gossip protocol and failure detection, and saga pattern.

Probabilistic and specialized structures

Requirement	Structure	Trade-off
Set membership at low memory, false positives acceptable	Bloom filter	No false negatives; cannot delete without variants.
Approximate distinct count	HyperLogLog	Small fixed memory, small relative error.
Approximate frequency counts	Count-Min sketch	Overestimates only.
Proximity search by location	Geohash, quadtree, S2, H3	Cell-based indexing.
Prefix search	Trie	Memory-heavy, fast prefix lookup.

See probabilistic data structures and geospatial indexing.

Worked example: product catalog with search

Requirements: browse products by ID (millions of reads per day), search by text with filters, record orders with payment, serve product images.

Data or operation	Choice	Reason
Orders and payments	Relational database	Multi-row transactions and constraints.
Product detail by ID	Relational or document store, fronted by a cache	Read by key, mostly unchanging, cache-aside reduces database load.
Text search with filters	Search engine fed from the product table	Inverted index provides relevance ranking. Index updates are asynchronous through change data capture, so search results can lag the database by seconds.
Product images	Object storage behind a CDN	Large immutable files, cacheable at the edge.
Order events to shipping and email	Message queue or Kafka topic, published through an outbox	Decouples consumers and avoids losing events between the database commit and the publish.

Each row states the access pattern that decides the choice. A different pattern, such as write volume of hundreds of thousands of events per second, changes the row.

Choosing by consistency and availability needs

Need	Direction
Strong consistency, correctness over availability	Single leader or consensus-based store (relational primary, etcd, Spanner-style).
High availability and write throughput, tolerance of stale reads	Leaderless or multi-leader replication, eventual consistency (Cassandra, DynamoDB default reads).
Global low-latency reads	Replicas or caches near users, CDN.
Financial ledger	Relational database with transactions, idempotency keys, audit trail.

Interview questions

How should a database be chosen in an interview? Start from the access patterns and consistency needs, then estimate volume. Use relational storage unless the estimates or access pattern require another model, and state the reason.
When is a search engine used with a primary database? When queries need text relevance, fuzzy matching, or faceting. The primary database remains the source of truth, and the search index is updated from it, typically asynchronously.
Kafka or a traditional queue? Kafka when multiple consumers need the same events, replay is required, or throughput is very high. A traditional queue when the need is per-task work distribution with per-message acknowledgment and routing.
Why use a cache and a CDN together? A CDN serves static or public content at the edge. A cache in front of the database serves dynamic data that changes and cannot be served from the edge.