Microservices Communication¶

Prerequisites: Messaging patterns, Sagas

← Sagas | Next: Circuit Breakers →

Why This Exists¶

Every in-process function call you split across a network boundary inherits three new failure modes it never had before: the call can be slow, it can fail, or it can succeed on the far end while the response is lost — meaning the caller can never fully distinguish "it didn't happen" from "it happened but I don't know." Every communication pattern on this page is a different answer to living with that ambiguity.

"Great system design isn't about memorizing eight patterns" — it's picking the cheapest one that matches the actual coupling requirement, and being able to say why the other seven were wrong for this call.

flowchart TB
    subgraph Sync["Synchronous"]
        direction LR
        S1["1 consumer:<br/>REST, gRPC"]
    end
    subgraph Async["Asynchronous"]
        direction LR
        A1["1 consumer:<br/>Message queue"]
        A2["N consumers:<br/>Pub/Sub"]
    end

Synchronous: Request/Response¶

REST over HTTP — caller sends a request, blocks, gets a response. The default for anything user-facing: CRUD APIs, mobile/web clients, anywhere an immediate answer is the point.

Cost: both services must be up and responsive at the same instant. If the callee is slow, the caller is slow — and without a timeout and circuit breaker (see Circuit Breakers), one slow dependency turns into a thread-pool exhaustion cascade across every caller.

gRPC is the same synchronous shape with a different wire format: Protocol Buffers over HTTP/2 instead of JSON over HTTP/1.1. Smaller payloads, lower latency, and a strongly-typed schema contract (.proto files) — the trade is weaker browser support and a debugging experience that needs grpcurl/reflection instead of curl and eyeballing JSON. The default choice for internal service-to-service calls at scale; REST remains the default for anything a browser or third party talks to directly.

sequenceDiagram
    participant Caller
    participant Callee
    Caller->>Callee: HTTP/gRPC request
    Note over Caller: blocked, waiting
    Callee-->>Caller: response (or timeout)

Asynchronous, One Consumer: Message Queue¶

Producer drops work onto a queue; a consumer picks it up whenever it's free. The caller does not wait for the work to finish — appropriate for anything without an immediate-response requirement: sending a confirmation email, processing an uploaded file, billing a background job.

What it buys: the producer keeps moving even if the consumer is temporarily down or backed up (the queue absorbs the burst — see Messaging Patterns for backpressure and consumer-group mechanics in depth). What it costs: ordering isn't free (most queues guarantee order only within a partition/shard, not globally), and "at least once" delivery means your consumer must be idempotent or you'll double-process on a redelivered message. A dead-letter queue is not optional — without one, a poison message that always fails retries forever and blocks everything behind it.

Asynchronous, Many Consumers: Pub/Sub¶

One event, many independent subscribers — a new-order event might trigger inventory reservation, an email, an analytics write, and a fraud check, none of which know about each other.

What it buys: the producer can stay ignorant of who's listening — adding a ninth subscriber requires zero changes to the publisher. This is the loosest coupling on this page. What it costs: that same ignorance makes the full business flow hard to trace — "what happens when an order is placed" is no longer readable in one file; it's scattered across N services' event handlers, each reacting independently, and distributed tracing becomes mandatory rather than a nice-to-have.

Orchestration vs. Choreography¶

These aren't separate transport mechanisms — they're two different ways to coordinate a multi-step business process built from the primitives above.

flowchart TB
    subgraph Choreo["Choreography — decentralized"]
        O1["Order svc publishes<br/>OrderCreated"] --> O2["Inventory svc reacts,<br/>publishes StockReserved"]
        O2 --> O3["Payment svc reacts,<br/>publishes PaymentCharged"]
        O3 --> O4["Shipping svc reacts"]
    end
    subgraph Orch["Orchestration — centralized"]
        C["Orchestrator"] --> C1["1. Reserve stock"]
        C --> C2["2. Charge card"]
        C --> C3["3. Ship"]
        C1 & C2 & C3 -.on failure, orchestrator runs compensations.-> C
    end

Choreography — each service reacts to events independently, no central controller. Maximizes service autonomy; costs visibility, since there's no single place that shows the whole flow, only N places that each show one reaction.

Orchestration (the Saga pattern) — a central coordinator drives each step and runs compensating actions on failure. This is the pattern covered in depth in Sagas: a distributed-transaction substitute where each step commits locally and undo is explicit, not automatic. Preferred for business-critical multi-step flows (checkout, booking) where "what state is this order actually in" needs one authoritative answer — at the cost of the orchestrator becoming a bottleneck or single point of coordination if poorly designed.

Event Sourcing + CQRS¶

Two patterns that usually show up together, addressing a different pressure than plain pub/sub:

• Event sourcing — persist every state change as an immutable event, not just the current row. The current state is a derived view (replay the events), not the source of truth.
• CQRS (Command Query Responsibility Segregation) — separate the write model (commands, validated against business rules) from the read model (denormalized, optimized for the queries the UI actually needs).

What it buys: a complete audit trail for free (you can answer "why is the balance what it is," not just "what is the balance"), replayable history to rebuild projections after a bug, and independently-scalable reads. What it costs: the read model is eventually consistent with the write model by some window you must be able to state; evolving an event's schema after the fact is genuinely hard (you can't just ALTER TABLE history); and it's a steep enough learning curve that adopting it for a CRUD app with no audit requirement is the textbook "pattern without a pressure" mistake (see Architecture Patterns).

Service Mesh¶

Not a communication pattern between two services — an infrastructure layer that intercepts the calls made by every pattern above. A sidecar proxy runs next to each service instance and handles mTLS, retries, timeouts, traffic splitting (useful for canary deployments), and distributed tracing, so individual services don't each reimplement retry logic and certificate rotation.

What it buys: cross-cutting concerns move out of application code into infrastructure, applied uniformly instead of "some services remembered to add retries and some didn't." What it costs: real operational overhead — another control plane to run, understand, and debug when a request behaves strangely and the mesh itself is a suspect. Standard on large Kubernetes platforms with dozens of services; overkill for five services that could just call each other directly with a shared HTTP client library.

Putting It Together¶

A realistic e-commerce checkout typically uses several of these at once, not one pattern chosen for the whole system:

The same checkout path, as one flow, annotated by which pattern each hop uses:

flowchart LR
    M["Mobile app"] -->|"REST"| GW["API Gateway"]
    GW -->|"gRPC"| ORD["Order service"]
    ORD -->|"queue: confirmation email"| MQ["Message Queue"]
    ORD -->|"pub/sub: OrderPlaced"| PS["Pub/Sub Topic"]
    PS --> INV["Inventory"]
    PS --> ANA["Analytics"]
    PS --> FRD["Fraud check"]
    ORD -->|"saga: orchestration"| SAGA["Checkout Saga"]
    SAGA --> RES["Reserve stock"]
    SAGA --> CHG["Charge card"]
    SAGA --> SHP["Schedule shipment"]
    CHG -->|"event sourcing"| LED[("Ledger<br/>event store")]
    ORD -.->|"mesh: mTLS + retries<br/>on every internal hop"| MESH["Service Mesh"]

    style GW fill:#1565c0,color:#fff
    style PS fill:#6a1b9a,color:#fff
    style SAGA fill:#e65100,color:#fff
    style LED fill:#1b5e20,color:#fff
    style MESH fill:#37474f,color:#fff

Trade-offs¶

Interview Questions¶

Key Takeaways¶

Previous: Sagas | Next: Circuit Breakers