Lesson 71 of 129
Event Sourcing and CQRS
Most systems store the current state of an entity and overwrite it on each change. Event sourcing stores instead the full sequence of events, immutable facts such as ItemAdded or OrderShipped, and derives current state from them. CQRS (Command Query Responsibility Segregation) separates the model that handles writes (commands) from the model that serves reads (queries). They are independent patterns that combine well.
Event sourcing
Event store
The event store is an append-only log of events, grouped into streams, usually one per aggregate instance. An aggregate is a cluster of objects treated as one consistency unit whose invariants are enforced together (for example, one order with its line items). Each event has a stream ID, a sequence number within the stream, a type, a payload, and metadata (timestamp, causation and correlation IDs).
Current state is computed by folding events in order:
state = fold(apply, initial_state, events)Example stream for order 42, with the state after each event:
| Version | Event | State after applying |
|---|---|---|
| 1 | OrderCreated(customer 7) |
status OPEN, items none, total 0 |
| 2 | ItemAdded(sku A, qty 2, price 10.00) |
items A x2, total 20.00 |
| 3 | ItemAdded(sku B, qty 1, price 5.00) |
items A x2, B x1, total 25.00 |
| 4 | OrderPlaced |
status PLACED, total 25.00 |
A command is validated against the state, not against the events directly: AddItem on a PLACED order is rejected because apply has set the status to PLACED. Applying an event never fails and never re-validates, since the event is a fact that already happened.
Handling a command:
- Load the aggregate's events and rebuild its state.
- Validate the command against the state and produce new events.
- Append the new events with an expected version. If another writer appended since the load, the append is rejected (optimistic concurrency), and the command is retried.
sequenceDiagram
participant C as Client
participant A as Command handler
participant E as Event store
C->>A: AddItem(order 42)
A->>E: load events of order 42
E-->>A: events v1..v3
A->>A: rebuild state, validate, create ItemAdded
A->>E: append ItemAdded, expected version 3
E-->>A: ok (v4)
A-->>C: acceptedProperties
- Complete audit trail: every change and its cause are recorded.
- History can be queried and replayed to debug or to answer new questions retroactively.
- New read models can be built from the full history.
- Events are natural integration messages; publishing them reliably uses the outbox or the event store's subscription mechanism.
Snapshots
Replaying a long stream is slow. A snapshot stores the aggregate state at a given version. Loading then reads the latest snapshot and only the events after it. For example, with a snapshot at version 1000 and a stream at version 1003, loading applies 3 events instead of 1003. A snapshot is typically written after a fixed number of events or by a background process. Snapshots are an optimization and can be discarded and regenerated. A snapshot's state format must be versioned as well, so a code change that alters the state shape invalidates old snapshots and requires a replay.
Schema evolution
Events are stored permanently, so old event formats must remain readable. Approaches:
- Additive changes with defaults for new fields (see API versioning and contract testing).
- Upcasting: converting old event versions to the current shape when reading.
- New event types for semantic changes, instead of changing the meaning of existing ones.
- Copy-and-transform migration of a stream into a new store, as a last resort.
Events must not be edited. Errors are corrected by appending compensating events.
CQRS
In CQRS, the command side validates and applies state changes using a model optimized for enforcing invariants. The query side uses one or more read models (projections) optimized for queries: denormalized tables, search indexes, caches.
flowchart LR
CL["Client"] -->|"commands"| CM["Command model"]
CM --> ES[("Event store / write DB")]
ES -->|"events"| PR["Projector"]
PR --> RM[("Read model: denormalized view")]
CL -->|"queries"| RMA projector subscribes to events and updates the read model. Multiple read models can serve different queries from the same events (for example, order history in a relational table, product search in Elasticsearch).
Benefits:
- Reads and writes scale independently.
- Read models are shaped for queries, avoiding joins (see data modeling and denormalization).
- Queries spanning several services can be served from a view built from their events (see eventual consistency and cross-service queries).
Eventual consistency
The read model is updated after the write. A client that writes and immediately reads may not see its change. Handling:
- Return the new version or the resulting data in the command response, and let the UI show it.
- Have the query accept a minimum version and wait or retry until the projection catches up.
- Route a user's reads to the write side briefly after a write.
Projectors must be idempotent, since events can be redelivered (see idempotent consumers and deduplication), and track the last processed position. Storing that position in the same transaction as the read-model update makes the update atomic with the position. A read model can be rebuilt by replaying all events into a fresh store, which is also the way to deploy a changed projection: build the new view in parallel, switch queries to it, and drop the old one.
CQRS without event sourcing
CQRS can use an ordinary database for the write side, with change events published (for example, through an outbox) to update read models. Event sourcing without CQRS is possible but querying by anything other than stream ID is impractical, so projections are almost always needed.
Costs and when not to use them
- Higher complexity: projections, eventual consistency, event versioning, and replay tooling.
- Learning curve, and the need for careful event design (past-tense facts describing business meaning, not table changes).
- Querying current state directly from the event store is not efficient.
- Deleting data is hard: privacy laws that require erasure conflict with immutable events. Techniques include storing personal data outside the events and deleting it there, or encrypting personal data per subject and discarding the key (crypto-shredding).
- Replay time grows with history, so snapshots and rebuild strategies are required.
Avoid these patterns for simple CRUD domains without audit or temporal requirements, for small systems where one relational model serves reads and writes, and where the team cannot operate the extra infrastructure. Use event sourcing where the history is itself valuable (finance, ledgers, audit-heavy domains) and CQRS where read and write loads or shapes differ substantially.
Interview questions
- How is current state derived in event sourcing? By applying the stream's events in order to an initial state, starting from the latest snapshot if one exists.
- How are concurrent writes to one aggregate handled? Appends carry the expected stream version. A mismatch means another write intervened, so the command is reloaded and retried.
- Why is the read side eventually consistent? Projections update asynchronously from the event stream, so a query can run before the projector has processed the latest event.
- How do you fix a wrong event? Events are immutable, so append a compensating or correcting event. Reads reflect the corrected state.
