gRPC and Protocol Buffers

gRPC is a remote procedure call (RPC) framework. A client calls a method on a remote service as if it were a local function, with generated client and server code. gRPC transports calls over HTTP/2 and, by default, serializes messages with Protocol Buffers (Protobuf), a binary format with a schema language. It is widely used for service-to-service communication inside a system.

Defining a service

Services and messages are declared in a .proto file, which is the contract shared by client and server:

syntax = "proto3";

message GetUserRequest {
  int64 id = 1;
}

message ListUsersRequest {
  int32 page_size = 1;
}

message User {
  int64 id = 1;
  string name = 2;
  string email = 3;
  repeated string roles = 4;
}

service UserService {
  rpc GetUser(GetUserRequest) returns (User);
  rpc ListUsers(ListUsersRequest) returns (stream User);
}

A compiler (protoc plus a language plugin) generates message types, a client stub, and a server interface in many languages. Client and server can be written in different languages and share only the .proto file.

Protobuf encoding

A serialized message is a sequence of fields. Each field is encoded as:

1. A tag, a varint holding (field_number << 3) | wire_type.
2. The value, encoded according to the wire type.

Wire types include varint (integers), length-delimited (strings, bytes, nested messages, packed repeated fields), and fixed 32-bit and 64-bit values.

Properties:

• Field names are not transmitted, only field numbers. This makes messages compact.
• A varint uses 1 to 10 bytes: seven data bits per byte, with the high bit marking that more bytes follow. Small numbers take one byte.
• Fields with default values (zero, empty string) are omitted in proto3 for scalar fields without explicit presence.
• Signed integers that are often negative should use sint32/sint64, which apply ZigZag encoding so small negative numbers stay small. A plain int32 negative value takes 10 bytes.
• Parsing is faster and messages are smaller than JSON, and the encoding is not human-readable.

Example: id = 150 in field 1 is encoded as the tag byte 0x08 (field 1, wire type 0) followed by the varint 0x96 0x01. The value 150 is 10010110 in binary. The low seven bits (0010110) are emitted first with the continuation bit set (0x96), then the remaining bits (0000001) as 0x01. A string name = "Ada" in field 2 is the tag 0x12 (field 2, wire type 2), the length 0x03, and the three UTF-8 bytes 0x41 0x64 0x61.

Together the two fields take 8 bytes: 08 96 01 12 03 41 64 61. The equivalent JSON, {"id":150,"name":"Ada"}, takes 23 bytes.

Transport on HTTP/2

Each RPC is one HTTP/2 stream.

• The request is a POST to a path of the form /package.Service/Method.
• Messages are sent as length-prefixed frames in the body (a 1-byte compression flag and a 4-byte length, followed by the Protobuf bytes).
• The status of the call is returned in HTTP/2 trailers (grpc-status, grpc-message), which arrive after the body.
• Many concurrent calls multiplex over one long-lived connection, avoiding a new connection per call (see HTTP/1.1, HTTP/2, and HTTP/3).
• Headers are compressed with HPACK, and per-stream flow control provides backpressure.

Because browsers do not expose the HTTP/2 framing and trailers needed, browser clients use gRPC-Web, which requires a translating proxy.

Four call types

flowchart LR
    subgraph U["Unary"]
        u1["1 request"] --> u2["1 response"]
    end
    subgraph SS["Server streaming"]
        s1["1 request"] --> s2["N responses"]
    end
    subgraph CS["Client streaming"]
        c1["N requests"] --> c2["1 response"]
    end
    subgraph BD["Bidirectional streaming"]
        b1["N requests"] <--> b2["N responses"]
    end

Messages within a stream are ordered.

Schema evolution

Protobuf is designed so that old and new versions of a schema can interoperate. Rules for compatible changes:

• Never change the number of an existing field and never reuse a number for a different meaning. The number is its identity on the wire.
• Adding a field with a new number is compatible. Old readers skip unknown fields (preserving them in current implementations), and new readers see the default value when the field is missing.
• Removing a field: mark its number and name reserved so no one reuses them.
• Renaming a field is wire-compatible, since names are not encoded (it can still break JSON mappings and source code).
• Changing a type is generally unsafe; only certain pairs are compatible (for example int32, int64, and bool share the varint wire type, but with truncation semantics).
• Use enums with a zero value as the default, and prefer wrapper or explicit-presence (optional) fields when "unset" must be distinguished from zero.

Backward compatibility (new readers, old data) and forward compatibility (old readers, new data) both hold if these rules are followed. Tooling such as buf can check for breaking changes automatically. See API versioning and contract testing.

Deadlines, errors, and metadata

• Deadlines: a client sets a deadline or timeout per call. It is propagated to the server, and it can be forwarded to downstream calls so that the whole call chain shares one time budget. When it expires, the call fails with DEADLINE_EXCEEDED and the server can stop work.
• Status codes: gRPC defines its own set (OK, INVALID_ARGUMENT, NOT_FOUND, ALREADY_EXISTS, PERMISSION_DENIED, UNAUTHENTICATED, RESOURCE_EXHAUSTED, UNAVAILABLE, DEADLINE_EXCEEDED, INTERNAL, and others). UNAVAILABLE signals a transient condition suitable for retry.
• Metadata: key-value headers carried with a call, used for authentication tokens and trace context.
• Interceptors: middleware on client and server for logging, authentication, retries, and metrics.

Load balancing considerations

Because gRPC clients hold a few long-lived HTTP/2 connections, a Layer 4 load balancer balances connections, not requests. One busy client can then send all its calls to a single backend. Solutions are a Layer 7 proxy that understands HTTP/2 and balances per request, or client-side load balancing where the client resolves all backend addresses (through DNS or service discovery) and picks one per call.

gRPC versus REST

Choose gRPC for internal traffic between services with strict contracts and high call volume, and for streaming. Choose REST for public or browser-facing APIs where tooling, caching, and ease of use for third parties matter. Many systems expose REST or GraphQL at the edge through an API gateway and use gRPC between services.

Interview questions

• Why is Protobuf smaller and faster than JSON? It sends numeric field tags instead of field names, uses varints and binary encodings for numbers, and needs no text parsing.
• How can you add a field to a message without breaking existing clients? Add it with a new, unused field number. Old clients ignore it, and new clients treat a missing value as the default.
• Why does gRPC need special load balancing? Long-lived multiplexed HTTP/2 connections mean a connection-level balancer pins all of a client's calls to one backend. Per-request balancing at Layer 7 or in the client is needed.
• How do deadlines help in a chain of services? The deadline propagates with each call, so downstream services can abandon work that can no longer complete in time.