Service Discovery

Microservice instances start, stop, scale, and move between hosts, so their network addresses change. Service discovery is the mechanism by which a client obtains the current address of an instance of a service, identified by a stable logical name, instead of a hard-coded host and port.

Components

• Service registry. A database of service names mapped to the network locations of healthy instances. It must be highly available and replicated, since it is on the path of every new connection.
• Registration. How instances get into the registry.
• Lookup. How clients get addresses out of it.
• Health checking. How dead instances are removed.

Registration models

A registered entry typically contains the service name, the instance ID, the address and port, metadata (version, zone, tags), and the health state.

Self-registration puts registry logic in every service and language. Third-party registration keeps services unaware but requires the registrar to be reliable.

If an instance crashes without deregistering, the registry removes it after heartbeats stop for a timeout (a lease or TTL) or after failed health checks. Until then, clients may receive a dead address, so clients still need timeouts and retries.

Worked example: an instance renews its lease every 30 seconds, and the registry evicts an instance after 90 seconds without a renewal. If the instance crashes right after a renewal, the registry can return its address for up to 90 seconds. Shorter intervals shrink this window but increase registry load (N instances renewing every T seconds produce N/T renewals per second, so 3,000 instances at 30 seconds produce 100 requests per second) and increase the chance of evicting healthy instances during a brief network problem. The actual intervals are configurable and differ between registries.

Lookup models

Client-side discovery

The client queries the registry, receives the list of instances, and chooses one using a load-balancing algorithm (see load balancing).

flowchart LR
    I["Instances"] -->|"register + heartbeat"| R["Registry"]
    C["Client with LB logic"] -->|"1. query"| R
    C -->|"2. call chosen instance"| I

• Fewer network hops: the client calls the instance directly.
• Load-balancing policy can be application-specific.
• Discovery logic is needed in every client language, typically via a library.

Server-side discovery

The client calls a stable address, a load balancer or router. The router queries the registry and forwards the request.

flowchart LR
    I["Instances"] -->|"register + heartbeat"| R["Registry"]
    C["Client"] -->|"1. call service name"| L["Load balancer / router"]
    L -->|"2. query"| R
    L -->|"3. forward"| I

• Clients are simple.
• An extra hop and one more component to make highly available.
• Used by cloud load balancers and Kubernetes Services.

A service mesh moves the client-side logic into a sidecar proxy, giving client-side behavior without library code in the application.

Common implementations

• DNS. The service name resolves to one or more IP addresses (A/SRV records). Simple and universal, but DNS caching and TTLs delay updates, and some clients cache indefinitely or use only the first address. See DNS resolution.
• Consul. A registry with a key-value store, health checks, and a DNS and HTTP interface. Agents run on each node; servers use Raft for consensus (see consensus).
• Eureka. Netflix's registry using self-registration and heartbeats. It favors availability over consistency: registry nodes replicate to each other and can serve stale data during partitions.
• etcd and ZooKeeper. Consistent stores that can hold registry data using ephemeral keys or leases tied to client sessions.
• Kubernetes. A Service object gives a set of pods (selected by labels) a stable virtual IP and a DNS name such as orders.shop.svc.cluster.local, resolved by the cluster DNS. The control plane maintains the list of ready pod addresses (EndpointSlices), and kube-proxy on each node routes traffic sent to the virtual IP to one of them. A headless Service (no virtual IP) makes DNS return the pod addresses directly, which supports client-side discovery.

Registry consistency and staleness

Registry data changes often and is read far more often than it is written. Two designs exist:

• Consistent registry (Consul servers, etcd, ZooKeeper). Writes go through a consensus protocol, and reads reflect committed state. During a partition, the minority side cannot accept writes and may not serve reads.
• Available registry (Eureka). Nodes accept registrations independently and replicate asynchronously. Lookups always succeed, but they can return instances that have already died.

Stale results are acceptable for discovery because clients must handle unreachable instances in any case. Returning no address during a partition is generally worse than returning a possibly stale one.

Health integration

Only instances that can serve traffic should be returned. Registries and orchestrators use:

• Heartbeats or leases that expire.
• Active checks, where the registry or agent probes an HTTP or TCP endpoint.
• Readiness signals. In Kubernetes, a pod failing its readiness probe is removed from Service endpoints but is not restarted, unlike a failing liveness probe. See health checks and graceful degradation.

Client behavior when discovery data is stale

• Set connect timeouts that are short compared with request timeouts.
• On a connection failure, retry against a different instance from the list rather than the same one, and temporarily stop using the failed instance (passive health checking or outlier detection).
• Re-resolve names periodically. Some runtimes cache DNS results for the lifetime of the process unless configured otherwise, which prevents the client from noticing new instances.

Failure considerations

• The registry is critical infrastructure. Clients should cache the last known list so they continue working during a registry outage.
• Registries that prefer availability may return stale entries. Registries that prefer consistency may refuse lookups during partitions. See CAP theorem.
• Mass deregistration on a false failure signal (for example, network partition between instances and registry) can remove healthy instances. Some registries enter a self-preservation mode that stops eviction when too many leases expire at once.

Interview questions

• Client-side vs server-side discovery? In client-side, the client gets the instance list and picks one. In server-side, a router does both. Client-side saves a hop but needs library code per language; server-side keeps clients simple.
• Why is DNS alone often insufficient? Caching by resolvers and clients delays propagation of changes, records carry no health-aware load balancing by default, and clients may not re-resolve.
• What happens when an instance dies without deregistering? Its lease or heartbeat expires or health checks fail, then the registry drops it. Until then clients can be routed to it, so retries and timeouts are required.
• How does Kubernetes implement discovery? Services provide a stable virtual IP and DNS name, and the control plane tracks ready pod addresses that node-level proxies route to.