Design a Digital Wallet

A digital wallet stores a monetary balance for each user and lets users move money between wallets (Venmo, PayPal balance, and similar). Unlike the payment system, which coordinates with an external PSP, the core of a wallet is an internal balance store. The central problem is a transfer between two accounts that may live on different database shards: it must be atomic (both balances change or neither), must never overdraw, and must be safe under retries.

Requirements

Functional

• Balance query for a wallet.
• Transfer from one wallet to another within the platform.
• Transaction history.
• Top-up from and withdrawal to external funding sources (delegated to the payment system).

Non-functional

• Strong consistency for balances: no negative balance, no lost or duplicated money.
• Exactly-once effect for each transfer, even with client retries and service crashes.
• Durability and auditability: every balance change is traceable to a transaction.
• Scale: the estimated peak is a few hundred transfers per second; the design targets a growth path to about 10,000 per second by adding shards.
• High availability, with correctness taking priority in a partition.

Estimation

Assumptions: 10 million active wallets, 5 million transfers per day, peak factor 10.

• Average: 5,000,000 / 86,400 is about 58 transfers per second; peak about 580 per second.
• A modern relational database node handles on the order of a thousand simple transactions per second, but a transfer touches two accounts and writes ledger rows, so assume about 500 to 1,000 transfers per second per node. The peak of 580 per second fits on one node with margin; a growth target of 10,000 per second needs about 10 to 20 shards.
• Storage: each transfer writes 2 ledger entries and 1 transfer record, about 500 bytes total. 5,000,000 x 500 B = 2.5 GB per day, about 900 GB per year.
• Balance rows: 10,000,000 x 100 B = 1 GB. This fits in memory of a single node.
• Bandwidth: 580 requests per second x about 1 KB request plus response is under 1 MB/s. Network is irrelevant; the constraint is transaction latency and lock contention, not throughput.
• Read traffic: assume 20 balance or history reads per transfer, so about 12,000 reads per second at peak. Reads dominate by 20x, which justifies replicas and a read model; it does not justify caching balances used for debit decisions.

What the numbers imply: a single primary is enough at launch, so start with a single-shard ACID transaction and avoid distributed transactions until growth forces sharding. Storage of about 900 GB per year means old ledger partitions (older than 90 days) move to cold storage while the balance table stays small. The design must keep the option of sharding from day one (wallet id in every key, no cross-wallet joins), not build it.

API design

POST /v1/transfers  { from_wallet, to_wallet, amount, currency, idempotency_key } -> { transfer_id, status }
GET  /v1/wallets/{id}/balance          -> { available, pending, currency }
GET  /v1/wallets/{id}/transactions?cursor=   -> paginated history
POST /v1/wallets/{id}/topups | /withdrawals  { amount, source, idempotency_key }

Amounts are integers in minor units. pending is the sum of amounts in in-flight transfers or withdrawals that are not yet completed. The idempotency key is required and stored with a unique constraint.

Data model

• wallet (wallet_id, user_id, currency, balance, version, updated_at): current balance, a cached aggregate.
• transfer (transfer_id, from_wallet, to_wallet, amount, status, idempotency_key unique, created_at).
• ledger_entry (entry_id, transfer_id, wallet_id, delta, balance_after, created_at): append-only. Each transfer produces one debit and one credit entry whose deltas sum to zero.

Two designs for the balance:

Common practice combines both: an immutable ledger is the source of truth and a balance column serves as a materialized projection updated in the same transaction. Storage is relational for ACID guarantees on the single-node path (see ACID and isolation).

High-level design

flowchart LR
    C["Client apps"] --> GW["API gateway (auth, rate limit)"]
    GW --> WS["Wallet service"]
    WS --> ID[("Idempotency / transfer table")]
    WS --> COORD["Transfer coordinator"]
    COORD --> S1[("Wallet shard 1")]
    COORD --> S2[("Wallet shard 2")]
    COORD --> SN[("Wallet shard N")]
    S1 --> LOG[["Ledger event stream"]]
    S2 --> LOG
    SN --> LOG
    LOG --> HIST["History / read model"]
    LOG --> REC["Reconciliation and audit"]
    LOG --> NT["Notifications"]
    WS --> PAY["Payment service (top-ups and withdrawals)"]

• Wallet service. Validates the request, checks the idempotency key, and starts the transfer.
• Transfer coordinator. Executes the debit and credit, possibly across shards.
• Wallet shards. Each holds a subset of wallets, partitioned by wallet_id (see sharding).
• Ledger event stream. Every committed change is published (through the outbox) to feed history views, notifications, and audit.

Deep dive

1. Atomic transfer when both wallets are on one shard

When both wallets live in the same database, a single ACID transaction handles the transfer:

BEGIN;
  UPDATE wallet SET balance = balance - :amt, version = version + 1
   WHERE wallet_id = :from AND balance >= :amt;      -- rows affected 0: insufficient funds, ROLLBACK
  UPDATE wallet SET balance = balance + :amt, version = version + 1
   WHERE wallet_id = :to;                             -- rows affected must be 1, else ROLLBACK
  INSERT INTO ledger_entry (...debit...), (...credit...);
  INSERT INTO transfer (..., status = 'COMPLETED', idempotency_key);  -- unique
COMMIT;

The balance >= :amt predicate is part of the same statement as the update, so two concurrent debits cannot both pass a stale check. The two updates are shown in debit-then-credit order for clarity; to avoid deadlocks between opposite transfers (A to B and B to A at the same time), an implementation issues them, or a preceding SELECT ... FOR UPDATE, in ascending wallet id order so that all transactions lock rows in the same order. A duplicate idempotency_key violates the unique constraint, the transaction aborts, and the service returns the original result.

Alternatives to the conditional update are optimistic locking (WHERE version = :v, retry on conflict) and pessimistic SELECT ... FOR UPDATE. Chosen: the conditional update, because it needs no read round trip and has one statement of lock time. Optimistic retries collapse under contention on a hot row (retry storms); pessimistic locking is acceptable but holds locks across round trips. Use READ COMMITTED with the conditional update, or SERIALIZABLE only when the transaction contains read-then-write logic such as limit checks; expect serialization failures that the service retries 2 to 3 times with jitter. Keep the transaction under about 10 ms so lock queues stay short; a row that sustains more than about 500 updates per second on one primary is a hot row (see section 4).

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class Main {
    sealed interface TransferResult {
        record Completed() implements TransferResult {}
        record Insufficient() implements TransferResult {}
        record NoWallet(String id) implements TransferResult {}
    }

    record Entry(String wallet, long delta) {}

    static final class Ledger {
        private final Map<String, Long> balances = new HashMap<>();
        private final List<Entry> entries = new ArrayList<>();
        private final Map<String, TransferResult> byKey = new HashMap<>();

        Ledger(Map<String, Long> opening) {
            balances.putAll(opening);
        }

        /** One transfer is one atomic unit: validate first, then change both balances and write both entries. */
        synchronized TransferResult transfer(String key, String from, String to, long amount) {
            if (amount <= 0) {
                throw new IllegalArgumentException("amount must be positive");
            }
            var previous = byKey.get(key);
            if (previous != null) {
                return previous; // retry with the same key: return the original result
            }
            TransferResult result;
            if (!balances.containsKey(from)) {
                result = new TransferResult.NoWallet(from);
            } else if (!balances.containsKey(to)) {
                result = new TransferResult.NoWallet(to);
            } else if (balances.get(from) < amount) {
                result = new TransferResult.Insufficient();
            } else {
                balances.merge(from, -amount, Long::sum);
                balances.merge(to, amount, Long::sum);
                entries.add(new Entry(from, -amount));
                entries.add(new Entry(to, amount));
                result = new TransferResult.Completed();
            }
            byKey.put(key, result);
            return result;
        }

        synchronized long balance(String wallet) {
            return balances.get(wallet);
        }

        synchronized long entriesSum() {
            return entries.stream().mapToLong(Entry::delta).sum();
        }

        synchronized long entriesSum(String wallet) {
            return entries.stream().filter(e -> e.wallet().equals(wallet)).mapToLong(Entry::delta).sum();
        }
    }

    public static void main(String[] args) {
        var ledger = new Ledger(Map.of("alice", 100L, "bob", 20L));
        System.out.println(ledger.transfer("k1", "alice", "bob", 70));
        System.out.println(ledger.transfer("k2", "alice", "bob", 70));
        System.out.println(ledger.transfer("k3", "alice", "carol", 5));
        System.out.println("retry k1: " + ledger.transfer("k1", "alice", "bob", 70));
        System.out.println("alice=" + ledger.balance("alice") + " bob=" + ledger.balance("bob"));
        System.out.println("entries sum to " + ledger.entriesSum());
    }
}

Completed[]
Insufficient[]
NoWallet[id=carol]
retry k1: Completed[]
alice=30 bob=90
entries sum to 0

The transfer method is synchronized, so the check and both balance changes happen as one unit, the in-memory form of the single-shard transaction. The result type is a sealed interface, so a caller handles Completed, Insufficient, and NoWallet explicitly. The idempotency key map returns the stored result for a retry, so the retry of k1 moves no money. A rejected transfer with a new key is also stored: a retry of k2 returns Insufficient even if the balance has changed since, which is the same rule as the unique key in the SQL version.

2. Transfers across shards

At scale the two wallets live on different shards, so no single local transaction covers both. Options:

A saga-style flow with an in-flight state:

sequenceDiagram
    participant C as Client
    participant W as Wallet service
    participant A as Shard A (payer)
    participant B as Shard B (payee)
    C->>W: transfer(A to B, 50, key K)
    W->>W: record transfer PENDING (unique K)
    W->>A: debit 50 (transfer id T)
    A-->>W: ok (ledger entry T)
    W->>B: credit 50 (transfer id T)
    B-->>W: ok
    W->>W: transfer COMPLETED
    W-->>C: success
    Note over W,B: If credit fails permanently: compensate by crediting A, mark REVERSED

Rules that make the saga safe:

• The transfer record with PENDING state is persisted before the first debit, so a crashed coordinator can resume from the stored state.
• Each shard operation is keyed by transfer id and is idempotent: applying debit T twice changes the balance once (a unique constraint on (transfer_id, wallet_id) in the ledger).
• Debit goes first, so the system never creates money; a failure leaves funds debited but in flight, which is recovered by retry or compensation. Crediting first would allow money to be spent that was never debited.
• A recovery process scans stale PENDING transfers and drives them to a terminal state.

Concrete parameters: retry the credit with exponential backoff for a bounded window (for example 5 minutes, since a shard failover typically takes seconds to tens of seconds); after that, compensate. Alert when the count of PENDING transfers older than 1 minute is non-zero. Shard key is wallet_id (hash), chosen over user_id ranges because it spreads load evenly; the cost is that most transfers are cross-shard (with N shards, roughly (N-1)/N of random pairs). Colocation by cohort raises the single-shard fraction only if transfer graphs are clustered, which is workload-dependent. A caveat of the saga: a compensating credit can itself fail if the source wallet was closed or frozen, so compensations go to a suspense account and are resolved manually.

3. Exactly-once effect and consistency checks

Client retries are handled by the idempotency key. Internal retries between services are handled by idempotent shard operations keyed by transfer id. Together these give at-least-once delivery with an exactly-once effect (see delivery semantics and idempotency).

Verification layers:

• Invariant per transaction: the sum of deltas of a transfer is zero.
• Invariant per wallet: balance equals the sum of its ledger entries. A periodic job recomputes and compares.
• Global invariant: the total of all wallets plus in-flight transfers plus external clearing accounts equals the total funded.

4. Event sourcing variant and hot wallets

With an event-sourced design, each wallet is an aggregate with an ordered stream of events (Debited, Credited). The current balance is a snapshot plus later events. Commands for one wallet are serialized on its partition (for example a log partition keyed by wallet id), which removes lock contention: a single writer per wallet processes commands in order. This also allows very high throughput per partition because commands are appended to a log and processed sequentially in memory. State can be rebuilt by replay, which also serves audit needs.

Hot wallets (a merchant receiving thousands of credits per second) become a bottleneck for row locks or single-partition ordering. Mitigations: split a hot wallet's balance into several sub-balances that are summed on read; batch credits into aggregated ledger writes; or model incoming payments as a ledger of pending credits settled periodically into the balance.

Failure modes

At 10x scale (about 6,000 transfers per second at peak): a single primary no longer fits, so shard into 10 to 20 nodes and most transfers become cross-shard sagas. The recovery scan must use an index on (status, updated_at) rather than a table scan. Hot merchant wallets appear first as lock queues, not as node CPU. Reconciliation moves from a nightly full recompute to incremental per-shard checks against the event stream.

Bottlenecks and scaling

• Shard by wallet id with a directory or hash mapping (see consistent hashing); resharding moves wallets by copying ledger and balance under a brief write pause per wallet.
• Single-shard transfers dominate if wallets are colocated by user cohort or region; cross-shard transfers pay the saga cost.
• Read scaling: history and balance reads go to replicas or read models; the authoritative balance for a debit decision is always read on the primary.
• Replication: synchronous replication for the ledger to avoid losing committed transfers on failover (see database replication).
• Fraud and limits: per-user limits and velocity checks are applied before the debit, using a rule service that must not block the transfer path for long.
• Backpressure: the coordinator caps in-flight transfers per shard and rejects with a retry hint above the cap.

Trade-offs and follow-up questions

• 2PC versus saga: atomic visibility against availability and throughput. The saga exposes an in-flight state that the user interface must represent as pending.
• Balance column versus derived balance: read speed against a single source of truth; a projection needs verification.
• Consistency of history reads: read models lag; a user may not see the newest transfer for a short time, so the transfer response should include the result.
• How to support multiple currencies? One wallet per currency; conversion is an explicit transfer through an exchange account at a recorded rate.
• How to reverse an erroneous transfer? A new compensating transfer; the ledger is never edited.
• What if the coordinator crashes after the debit but before the credit? The PENDING record and the idempotent shard operations let the recovery process finish the credit; money is never lost, only delayed.
• What if two clients send the same transfer with different idempotency keys? They are two transfers; the key defines identity. Duplicate-suspicion heuristics (same amount and payee within seconds) belong in a confirmation step in the client, not in the ledger.
• What if one wallet receives 5,000 credits per second? Sub-balances (for example 16 rows, credit to hash(transfer_id) mod 16), summed on read; debits from that wallet lock one sub-balance and may need to draw from several, which is why hot wallets are usually receive-heavy merchant accounts.
• Can balances be cached? Not for the debit decision; for display, a cache with a TTL of a few seconds is acceptable if invalidated on write.
• How do you prove no money was created? The global invariant job: sum of wallet balances plus in-flight plus clearing accounts equals total funded, alerting on any non-zero drift.
• How to handle withdrawals to a bank with unknown outcomes? Move funds to a pending-withdrawal account first, and settle it when the bank confirms (see payment system).