Service Mesh and Sidecars

Every service that calls another needs the same networking features: service discovery, load balancing, timeouts, retries, encryption, authentication, and telemetry. Implementing them in a library per language duplicates effort and makes upgrades depend on redeploying every service. A service mesh is an infrastructure layer that provides these features uniformly by running a proxy next to each service instance and managing all the proxies from a central component.

Architecture

A mesh has two parts:

• Data plane. A network proxy deployed with every service instance, called a sidecar when it runs as a separate process or container in the same pod or host. All inbound and outbound traffic of the service passes through it. Envoy is the most common proxy. Linkerd uses its own Rust-based proxy.
• Control plane. A set of components that holds configuration, computes routing and policy, issues certificates, and pushes configuration to the sidecars. In Istio this is istiod. In Linkerd it is a set of control-plane services.

flowchart TB
    CP["Control plane: config, policy, certificate authority"]
    subgraph PodA["Pod A"]
        A["Service A"] <--> PA["Sidecar proxy"]
    end
    subgraph PodB["Pod B"]
        B["Service B"] <--> PB["Sidecar proxy"]
    end
    PA <-->|"mTLS"| PB
    CP -.->|"config, certificates"| PA
    CP -.->|"config, certificates"| PB

Traffic redirection to the sidecar is typically set up at pod start with iptables rules (or an equivalent) so the application code needs no changes: it connects to service-b, and the sidecar intercepts and forwards the connection.

A request from Service A to Service B follows these steps:

1. A's application opens a connection to service-b. The redirect rules deliver it to A's sidecar.
2. A's sidecar selects a B endpoint using the endpoint list and routing rules received from the control plane, and applies the timeout and retry policy.
3. A's sidecar establishes an mTLS connection to B's sidecar. Both verify each other's certificates against the mesh's trust root.
4. B's sidecar checks the authorization policy against A's identity, decrypts the request, and forwards it to B's application over the local interface.
5. Both sidecars record metrics and trace spans for the hop.

The proxies receive configuration through a streaming API (in Istio and Envoy, the xDS APIs), so route and endpoint changes reach them without restarts.

Features

Mutual TLS

The control plane runs a certificate authority. Each workload receives a short-lived certificate encoding its identity, and sidecars perform mutual TLS (mTLS): both sides present certificates, so each verifies the peer's identity and the traffic is encrypted. Certificates rotate automatically. This gives service-to-service authentication without application code. See microservices security and zero trust.

Traffic management

• Load balancing across instances at the request level (L7), not only per connection.
• Weighted routing between versions, for canary releases and blue/green deployments (see CI/CD and deployment strategies).
• Routing by headers or paths, and traffic mirroring to test a new version with copies of real requests.
• Fault injection for testing (see chaos engineering).

An Istio canary rule sending 10% of requests for the reviews service to version v2 looks like the following. The v1 and v2 subsets are defined in a separate DestinationRule that maps them to pod labels.

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
    - route:
        - destination:
            host: reviews
            subset: v1
          weight: 90
        - destination:
            host: reviews
            subset: v2
          weight: 10

Resilience

Per-route configuration of:

• Timeouts and retries with limits on retry counts (see timeouts, retries, backoff and jitter).
• Outlier detection, which ejects instances returning repeated errors, and connection or request limits that act as circuit breakers (see circuit breaker).
• Rate limiting.

Retries in the mesh multiply with retries in application code. If the application makes up to 3 attempts and each attempt is sent by a sidecar that also makes up to 3 attempts, one logical call can produce 9 requests to the failing service, and the multiplication continues at each upstream hop. Configure retry policy in one layer and cap the fraction of requests that may be retries.

Observability

Sidecars emit uniform metrics per request (rate, errors, latency), access logs, and trace spans. Because every hop passes through a proxy, the mesh produces service-level metrics and dependency maps without instrumentation. Trace context must still be propagated by the application from inbound to outbound requests, since the proxy cannot associate them otherwise (see distributed tracing).

Authorization policy

Policies such as "service orders may call POST /payments on payments" are enforced at the sidecar using the mTLS identity.

Istio and Linkerd

Costs and drawbacks

• Latency. Each call traverses two extra proxies (caller side and callee side), adding a small per-hop latency.
• Resource use. A proxy per pod consumes CPU and memory across the whole fleet.
• Operational complexity. The mesh is another distributed system to upgrade, monitor, and debug. Misconfigured routing or policy can cause outages across all services.
• Debugging. Failures can originate in the proxy layer, which must be inspected separately from application logs.
• Scope. A mesh covers service-to-service traffic. Client-to-service traffic still enters through an API gateway or ingress.

When to use

A mesh is justified when many services need consistent mTLS, traffic policy, and telemetry, and platform teams can operate it. With a handful of services, libraries or platform-level features (for example, Kubernetes network policies and a load balancer) are less costly.

Interview questions

• What are the data plane and control plane? The data plane consists of the proxies that carry the traffic. The control plane configures them, distributes routing policy, and issues certificates.
• How does a mesh give mTLS without code changes? Sidecars intercept traffic, hold workload certificates issued by the mesh's certificate authority, and establish mutual TLS with peer sidecars.
• Why can mesh retries be dangerous? Retries combine with application-level retries and with retries in upstream services, so the load on a struggling service can multiply. Limits and a single owner of retry policy are needed.
• Can the mesh trace requests by itself? It generates spans for each hop, but the application must copy trace headers from incoming to outgoing requests to link them into one trace.