---
# === IDENTITY ===
id: software/system-design/microservices-communication/2026
canonical_question: "What are the best microservices communication patterns?"
aliases:
  - "microservices communication patterns"
  - "inter-service communication microservices"
  - "REST vs gRPC microservices"
  - "async vs sync microservices"
  - "message queue vs direct call microservices"
  - "event-driven microservices architecture"
  - "microservice API communication"
  - "service-to-service communication patterns"
entity_type: software_reference
domain: software > system-design > microservices_communication
region: global
jurisdiction: global
temporal_scope: 2020-2026

# === VERIFICATION ===
last_verified: 2026-02-23
confidence: 0.93
version: 1.0
first_published: 2026-02-23

# === TEMPORAL VALIDITY ===
temporal_validity:
  status: stable
  last_breaking_change: "gRPC v1.60+ default xDS support; Kafka 3.7+ KRaft mode replaces ZooKeeper (2024)"
  next_review: 2026-08-22
  change_sensitivity: medium

# === CONSTRAINTS ===
constraints:
  - "Never use synchronous chains deeper than 2 hops -- cascading timeouts and failures create a distributed monolith"
  - "Always implement idempotency on message consumers -- at-least-once delivery means duplicates will arrive"
  - "gRPC requires HTTP/2 -- do not expose gRPC directly through HTTP/1.1-only proxies or load balancers without gRPC-Web or Envoy transcoding"
  - "Message ordering guarantees vary by broker -- Kafka guarantees order within a partition, RabbitMQ does not guarantee order across consumers"
  - "Circuit breakers are mandatory for all synchronous calls -- without them, a single slow service cascades failures across the entire mesh"

# === SKIP CONDITIONS ===
skip_this_unit_if:
  - condition: "Designing a monolith-to-microservices migration strategy (not communication patterns)"
    use_instead: "software/migrations/monolith-to-microservices/2026"
  - condition: "Designing a message queue / event-driven architecture from scratch"
    use_instead: "software/system-design/message-queue-event-driven/2026"
  - condition: "Implementing CQRS and Event Sourcing patterns"
    use_instead: "software/system-design/cqrs-event-sourcing/2026"

# === AGENT HINTS ===
inputs_needed:
  - key: latency_requirement
    question: "What is your latency requirement for inter-service calls?"
    type: choice
    options: ["<10ms (real-time)", "10-100ms (interactive)", "100ms-5s (near real-time)", ">5s (background/batch)"]
  - key: consistency_model
    question: "Do you need strong consistency (immediate read-after-write) or eventual consistency?"
    type: choice
    options: ["Strong consistency", "Eventual consistency", "Mixed (some operations need strong)"]
  - key: scale
    question: "What is your expected request volume?"
    type: choice
    options: ["<1K req/s", "1K-10K req/s", "10K-100K req/s", ">100K req/s"]

# === DISTRIBUTION ===
canonical_source: "https://knowledgelib.io/software/system-design/microservices-communication/2026"
suggested_citation: "Source: knowledgelib.io -- AI Knowledge Library (verified 2026-02-23)"

# === RELATED UNITS ===
related_kos:
  depends_on:
    - id: "software/system-design/api-gateway/2026"
      label: "API Gateway Design"
  related_to:
    - id: "software/system-design/message-queue-event-driven/2026"
      label: "Message Queue and Event-Driven Architecture"
    - id: "software/system-design/cqrs-event-sourcing/2026"
      label: "CQRS and Event Sourcing"
    - id: "software/system-design/rate-limiter/2026"
      label: "Distributed Rate Limiter Design"
  solves:
    - id: "software/system-design/notification-system/2026"
      label: "Scalable Notification System Design"
  alternative_to:
    - id: "software/system-design/graphql-api-architecture/2026"
      label: "GraphQL API Architecture at Scale"
  often_confused_with:
    - id: "software/migrations/monolith-to-microservices/2026"
      label: "Monolith to Microservices Migration"

# === SOURCES (7 authoritative sources) ===
sources:
  - id: src1
    title: "Communication in a microservice architecture"
    author: Microsoft
    url: https://learn.microsoft.com/en-us/dotnet/architecture/microservices/architect-microservice-container-applications/communication-in-microservice-architecture
    type: official_docs
    published: 2024-08-21
    reliability: high
  - id: src2
    title: "Interservice communication in microservices"
    author: Microsoft Azure Architecture Center
    url: https://learn.microsoft.com/en-us/azure/architecture/microservices/design/interservice-communication
    type: official_docs
    published: 2024-11-15
    reliability: high
  - id: src3
    title: "Building Microservices, 2nd Edition -- Chapter 4: Microservice Communication Styles"
    author: Sam Newman (O'Reilly)
    url: https://www.oreilly.com/library/view/building-microservices-2nd/9781492034018/ch04.html
    type: technical_blog
    published: 2021-08-05
    reliability: high
  - id: src4
    title: "gRPC Documentation -- Core Concepts"
    author: gRPC Authors (CNCF)
    url: https://grpc.io/docs/what-is-grpc/core-concepts/
    type: official_docs
    published: 2024-06-01
    reliability: high
  - id: src5
    title: "Event-Driven Architecture (EDA): A Complete Introduction"
    author: Confluent
    url: https://www.confluent.io/learn/event-driven-architecture/
    type: technical_blog
    published: 2024-09-10
    reliability: moderate_high
  - id: src6
    title: "Communication mechanisms -- Implementing Microservices on AWS"
    author: Amazon Web Services
    url: https://docs.aws.amazon.com/whitepapers/latest/microservices-on-aws/communication-mechanisms.html
    type: official_docs
    published: 2023-11-01
    reliability: high
  - id: src7
    title: "gRPC vs REST in 2025: Performance Benchmarks for Microservices"
    author: Markaicode
    url: https://markaicode.com/grpc-vs-rest-benchmarks-2025/
    type: technical_blog
    published: 2025-01-15
    reliability: moderate_high
---

# Microservices Communication Patterns

## TL;DR

- **Bottom line**: Choose synchronous (REST/gRPC) for request-response queries needing immediate answers; choose asynchronous (message queues/event bus) for fire-and-forget commands and event-driven workflows. Most production systems use both.
- **Key tool/command**: gRPC for internal service-to-service; Kafka/RabbitMQ for async event streams; REST for public-facing APIs.
- **Watch out for**: The distributed monolith anti-pattern -- synchronous call chains deeper than 2 hops create cascading failures that are worse than a monolith.
- **Works with**: Any language/platform -- REST (HTTP/1.1+), gRPC (HTTP/2), AMQP (RabbitMQ), Kafka protocol. Service meshes (Istio, Linkerd) handle cross-cutting concerns.

## Constraints
<!-- Agents: read this section before recommending any action from this unit.
     These are hard constraints that must not be violated. -->

- Never use synchronous chains deeper than 2 hops -- cascading timeouts and failures create a distributed monolith
- Always implement idempotency on message consumers -- at-least-once delivery means duplicates will arrive
- gRPC requires HTTP/2 -- do not expose gRPC directly through HTTP/1.1-only proxies or load balancers without gRPC-Web or Envoy transcoding
- Message ordering guarantees vary by broker -- Kafka guarantees order within a partition, RabbitMQ does not guarantee order across consumers
- Circuit breakers are mandatory for all synchronous calls -- without them, a single slow service cascades failures across the entire mesh

## Quick Reference

| Pattern | Type | Protocol | Latency | Throughput | Coupling | Best For | Technology Options |
|---|---|---|---|---|---|---|---|
| REST (JSON/HTTP) | Sync request-response | HTTP/1.1+ | 10-100ms | Moderate (~2-4K rps/instance) | Temporal + spatial | Public APIs, CRUD operations | Express, FastAPI, Spring Boot, Go net/http |
| gRPC (Protobuf) | Sync request-response | HTTP/2 | 1-20ms | High (~7-9K rps/instance) | Temporal + spatial | Internal service calls, polyglot | grpc-go, grpc-java, grpc-node, grpc-python |
| gRPC Streaming | Sync bidirectional | HTTP/2 | Sub-ms per msg | Very high | Temporal + spatial | Real-time data feeds, chat | Same gRPC libs + streaming APIs |
| GraphQL | Sync request-response | HTTP/1.1+ | 10-200ms | Moderate | Temporal + spatial | API aggregation, BFF pattern | Apollo Server, Hasura, graphql-go |
| Message Queue (P2P) | Async point-to-point | AMQP/STOMP | 5-50ms | High | Loose | Task distribution, work queues | RabbitMQ, Amazon SQS, Azure Service Bus |
| Pub/Sub Event Bus | Async publish-subscribe | Kafka/NATS | 2-20ms | Very high (1M+ msgs/s) | Very loose | Event-driven, domain events | Apache Kafka, NATS, Google Pub/Sub |
| Event Sourcing | Async event log | Kafka/EventStore | 10-100ms | High | Very loose | Audit trails, CQRS | EventStoreDB, Kafka + custom projections |
| Saga (Choreography) | Async distributed tx | Events via broker | 100ms-10s | Moderate | Loose | Multi-service transactions | Kafka, RabbitMQ + saga state tracking |
| Saga (Orchestration) | Mixed sync+async | HTTP/gRPC + broker | 50ms-5s | Moderate | Moderate | Complex workflows, compensations | Temporal, Camunda, Step Functions |
| Service Mesh Sidecar | Sync (transparent) | HTTP/2, mTLS | +1-3ms overhead | Proxied | Loose (infra) | mTLS, retries, observability | Istio, Linkerd, Consul Connect |
| Webhooks | Async callback | HTTP/1.1+ | 100ms-30s | Low-moderate | Loose | External integrations, notifications | Custom HTTP endpoints |
| Shared Database | Sync shared state | SQL/NoSQL | 1-10ms | High | Very tight | Legacy migration only (anti-pattern) | PostgreSQL, MongoDB (avoid in greenfield) |

## Decision Tree

> Full script: [decision-tree.txt](scripts/decision-tree.txt) (26 lines)

```
START
├── Need immediate response (request-response)?
│   ├── YES → Is this a public/external API?
│   │   ├── YES → REST (JSON over HTTP) -- universal client support
│   │   └── NO → Internal service-to-service?
# ... (see full script)
```

## Step-by-Step Guide

### 1. Define service boundaries and communication needs

Map each service interaction as synchronous (needs response) or asynchronous (fire-and-forget / eventual). Draw a service dependency graph. Any cycle indicates incorrect boundaries. [src3]

```
Service A --[sync query]--> Service B
Service A --[async event]--> Event Bus --[subscribe]--> Service C
Service A --[async event]--> Event Bus --[subscribe]--> Service D
```

**Verify**: No service should have more than 2 synchronous downstream dependencies. Count sync edges per node.

### 2. Set up synchronous communication (gRPC)

Define service contracts using Protocol Buffers. gRPC generates client/server stubs in all major languages from a single `.proto` file. [src4]

```protobuf
// order_service.proto
syntax = "proto3";
package order;

service OrderService {
  rpc GetOrder (GetOrderRequest) returns (OrderResponse);
  rpc CreateOrder (CreateOrderRequest) returns (OrderResponse);
  rpc StreamOrderUpdates (GetOrderRequest) returns (stream OrderEvent);
}

message GetOrderRequest {
  string order_id = 1;
}

message CreateOrderRequest {
  string customer_id = 1;
  repeated OrderItem items = 2;
}

message OrderItem {
  string product_id = 1;
  int32 quantity = 2;
  int64 price_cents = 3;
}

message OrderResponse {
  string order_id = 1;
  string status = 2;
  int64 total_cents = 3;
  string created_at = 4;
}

message OrderEvent {
  string order_id = 1;
  string event_type = 2;
  string timestamp = 3;
}
```

**Verify**: `protoc --lint_out=. order_service.proto` -- no warnings. Generate stubs: `protoc --go_out=. --go-grpc_out=. order_service.proto`

### 3. Set up asynchronous communication (Event Bus)

Choose a message broker based on your throughput and ordering needs. Kafka for high-throughput ordered event streams; RabbitMQ for flexible routing and work queues. [src5]

```yaml
# docker-compose.yml -- local development Kafka setup (KRaft mode, no ZooKeeper)
services:
  kafka:
    image: apache/kafka:3.7.0
    ports:
      - "9092:9092"
    environment:
      KAFKA_NODE_ID: 1
      KAFKA_PROCESS_ROLES: broker,controller
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@localhost:9093
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_LOG_DIRS: /tmp/kraft-logs
      CLUSTER_ID: MkU3OEVBNTcwNTJENDM2Qk
```

**Verify**: `docker exec kafka kafka-topics.sh --bootstrap-server localhost:9092 --list` returns empty list (broker is healthy).

### 4. Implement circuit breakers on synchronous paths

Wrap every synchronous outbound call with a circuit breaker. This prevents cascading failures when a downstream service is slow or unavailable. [src1]

```python
# Python with tenacity + custom circuit breaker
import httpx
from tenacity import retry, stop_after_attempt, wait_exponential, CircuitBreaker

breaker = CircuitBreaker(fail_max=5, reset_timeout=30)

@breaker
@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=0.5, max=5))
async def call_inventory_service(product_id: str) -> dict:
    async with httpx.AsyncClient(timeout=5.0) as client:
        resp = await client.get(f"http://inventory-service/api/v1/stock/{product_id}")
        resp.raise_for_status()
        return resp.json()
```

**Verify**: Kill inventory service, call 6 times -> circuit opens. Wait 30s -> circuit half-opens.

### 5. Implement idempotent consumers

Every async message consumer must handle duplicate deliveries gracefully. Use an idempotency key (event ID) stored in a deduplication table. [src6]

```python
# Idempotent Kafka consumer with deduplication
import json
from kafka import KafkaConsumer

consumer = KafkaConsumer(
    'order-events',
    bootstrap_servers='localhost:9092',
    group_id='payment-service',
    auto_offset_commit=False,
    value_deserializer=lambda m: json.loads(m.decode('utf-8'))
)

processed_ids = set()  # In production: use Redis or DB table

for message in consumer:
    event = message.value
    event_id = event['event_id']

    if event_id in processed_ids:
        consumer.commit()  # Skip duplicate, commit offset
        continue

    process_payment(event)
    processed_ids.add(event_id)
    consumer.commit()
```

**Verify**: Publish same event twice with identical `event_id` -> consumer processes it only once.

### 6. Add observability (distributed tracing)

Propagate trace context (W3C Trace Context or B3) across all communication boundaries -- sync and async. Without this, debugging cross-service issues is nearly impossible. [src2]

```python
# OpenTelemetry instrumentation for gRPC + Kafka
from opentelemetry import trace
from opentelemetry.instrumentation.grpc import GrpcInstrumentorClient
from opentelemetry.instrumentation.kafka import KafkaInstrumentor

# Auto-instrument gRPC client calls
GrpcInstrumentorClient().instrument()

# Auto-instrument Kafka producer/consumer
KafkaInstrumentor().instrument()

# Manual span for business logic
tracer = trace.get_tracer("order-service")
with tracer.start_as_current_span("process_order") as span:
    span.set_attribute("order.id", order_id)
    # gRPC call -- trace context propagated automatically
    inventory = inventory_stub.CheckStock(request)
    # Kafka publish -- trace context injected into headers
    producer.send('order-events', value=event)
```

**Verify**: `curl http://jaeger:16686/api/traces?service=order-service` -> traces span across services.

## Code Examples

### Python: Async Event Handler with Kafka

> Full script: [python-async-event-handler-with-kafka.py](scripts/python-async-event-handler-with-kafka.py) (30 lines)

```python
# Input:  Kafka messages on 'order-created' topic
# Output: Payment processing + 'payment-completed' event
import asyncio
import json
from aiokafka import AIOKafkaConsumer, AIOKafkaProducer
# ... (see full script)
```

### Go: gRPC Server with Interceptors

> Full script: [go-grpc-server-with-interceptors.go](scripts/go-grpc-server-with-interceptors.go) (28 lines)

```go
// Input:  gRPC requests to OrderService
// Output: Order responses with circuit breaker + logging
package main
import (
    "context"
# ... (see full script)
```

### TypeScript: REST with Circuit Breaker (Node.js)

```typescript
// Input:  HTTP requests to downstream services
// Output: Resilient responses with fallback on failure

import CircuitBreaker from 'opossum';

const breakerOptions = {
  timeout: 3000,       // 3s timeout per request
  errorThresholdPercentage: 50,
  resetTimeout: 30000  // 30s before half-open
};

async function fetchInventory(productId: string): Promise<InventoryResponse> {
  const res = await fetch(`http://inventory-svc/api/v1/stock/${productId}`);
  if (!res.ok) throw new Error(`Inventory service error: ${res.status}`);
  return res.json();
}

const breaker = new CircuitBreaker(fetchInventory, breakerOptions);
breaker.fallback((productId: string) => ({ productId, inStock: null, cached: true }));
breaker.on('open', () => console.warn('Circuit OPEN: inventory-svc'));

// Usage: const stock = await breaker.fire('product-123');
```

## Anti-Patterns

### Wrong: Distributed Monolith (synchronous call chains)

```
// BAD -- 5-hop synchronous chain: one slow service kills everything
// Order -> Inventory -> Pricing -> Tax -> Shipping -> Notification
// Total latency = sum of all latencies; one failure = total failure

POST /orders
  -> GET inventory-svc/stock/{id}          // 50ms
    -> GET pricing-svc/price/{id}          // 30ms
      -> GET tax-svc/calculate             // 40ms
        -> POST shipping-svc/estimate      // 100ms
          -> POST notification-svc/send    // 200ms
// Total: 420ms best case. Any timeout cascades upward.
```

### Correct: Hybrid sync + async with bounded sync depth

```
// GOOD -- Max 1 sync hop; rest is async via events
POST /orders
  -> GET inventory-svc/stock/{id}    // 1 sync hop (needs real-time answer)
  <- 201 Created (return to client)

  -> publish 'order-created' event   // Async from here
     -> payment-svc consumes         // Independent
     -> shipping-svc consumes        // Independent
     -> notification-svc consumes    // Independent
// Total sync latency: ~50ms. Async services process in parallel.
```

### Wrong: No idempotency on consumers

```python
# BAD -- processing duplicate messages charges customer twice
def handle_payment(event):
    charge_customer(event['customer_id'], event['amount'])  # No dedup!
    db.insert('payments', event)  # Duplicate row on retry
```

### Correct: Idempotent consumer with deduplication

```python
# GOOD -- idempotency key prevents double-processing
def handle_payment(event):
    if db.exists('processed_events', event['event_id']):
        return  # Already processed, skip
    charge_customer(event['customer_id'], event['amount'])
    db.insert('payments', {**event, 'processed_at': now()})
    db.insert('processed_events', {'event_id': event['event_id']})
    # Use a DB transaction to make both inserts atomic
```

### Wrong: Exposing gRPC directly to browsers

```
// BAD -- browsers don't support HTTP/2 trailers (gRPC requirement)
// Frontend JS cannot call gRPC endpoints directly
const client = new OrderServiceClient('https://api.example.com:50051');
// This fails: browsers use HTTP/1.1 or HTTP/2 without trailer support
```

### Correct: Use gRPC-Web or REST gateway for browser clients

```
// GOOD -- Envoy proxy transcodes gRPC to gRPC-Web for browsers
// envoy.yaml
listeners:
  - filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          http_filters:
            - name: envoy.filters.http.grpc_web    # Transcodes for browsers
            - name: envoy.filters.http.router

// OR: Use an API gateway that exposes REST -> gRPC internally
// Browser -> REST (API Gateway) -> gRPC (internal services)
```

### Wrong: Shared database between microservices

```
// BAD -- two services reading/writing the same 'orders' table
// Order Service and Shipping Service both do:
SELECT * FROM orders WHERE status = 'pending';
UPDATE orders SET status = 'shipped' WHERE id = ?;
// Schema changes in one service break the other. Tight coupling.
```

### Correct: Each service owns its data; communicate via events

```
// GOOD -- Order Service owns 'orders' table
// Shipping Service owns 'shipments' table
// Communication via events:

// Order Service publishes:
{ "event": "order_placed", "order_id": "123", "items": [...] }

// Shipping Service consumes event, writes to its own table:
INSERT INTO shipments (order_id, status) VALUES ('123', 'pending');
// No shared database. Schema changes are independent.
```

## Common Pitfalls

- **Chatty services (too many small calls)**: Making 10+ REST calls to assemble one response kills performance. Fix: Use an API Gateway or BFF pattern to batch and aggregate; or use gRPC streaming for bulk data transfer. [src2]
- **Missing timeouts on HTTP clients**: Default HTTP clients often have no timeout (or 60s+), causing thread/connection pool exhaustion. Fix: Set explicit `connect_timeout` (1-3s) and `read_timeout` (3-10s) on every HTTP client instance. [src1]
- **Ignoring backpressure in async consumers**: If consumers are slower than producers, queues grow unbounded, leading to OOM or disk exhaustion. Fix: Configure `max.poll.records` (Kafka) or `prefetch_count` (RabbitMQ) to limit in-flight messages. [src5]
- **No dead letter queue (DLQ)**: Poison messages (unparseable, schema mismatch) block the queue forever. Fix: Configure a DLQ after N retry attempts; monitor DLQ depth as an alert. [src6]
- **Sync calls inside event handlers**: Processing an event synchronously calls another service, reintroducing temporal coupling. Fix: Event handlers should only read local state or emit new events -- never make synchronous outbound calls. [src3]
- **Skipping schema evolution**: Changing a Protobuf message or Avro schema without backward compatibility breaks all consumers. Fix: Only add optional fields; never remove or renumber existing fields; use a schema registry. [src4]
- **No correlation ID propagation**: Without a shared trace/correlation ID, debugging a request across 5+ services requires log timestamp correlation (fragile and slow). Fix: Inject `X-Correlation-ID` or W3C `traceparent` header in every outbound call and message. [src2]
- **Choosing Kafka for simple work queues**: Kafka excels at ordered event streams but is overkill for task distribution where you just need competing consumers. Fix: Use RabbitMQ or SQS for work queues; reserve Kafka for event streaming and log aggregation. [src5]

## Diagnostic Commands

```bash
# Check if gRPC service is healthy
grpcurl -plaintext localhost:50051 grpc.health.v1.Health/Check

# List available gRPC services
grpcurl -plaintext localhost:50051 list

# Check Kafka topic lag (consumer behind producer)
kafka-consumer-groups.sh --bootstrap-server localhost:9092 \
  --describe --group payment-service

# Check RabbitMQ queue depth
rabbitmqctl list_queues name messages_ready messages_unacknowledged

# Test REST endpoint with timing
curl -w "\nDNS: %{time_namelookup}s\nConnect: %{time_connect}s\nTTFB: %{time_starttransfer}s\nTotal: %{time_total}s\n" \
  http://order-service:8080/api/v1/orders/123

# Check service mesh proxy status (Istio)
istioctl proxy-status

# Verify mTLS between services (Istio)
istioctl authn tls-check order-service.default.svc.cluster.local
```

## Version History & Compatibility

| Technology | Current Version | Key Change | Notes |
|---|---|---|---|
| gRPC | v1.62+ (2024) | xDS load balancing by default | Enable via `GRPC_XDS_BOOTSTRAP` env var |
| Apache Kafka | 3.7+ (2024) | KRaft mode GA (no ZooKeeper) | Migrate from ZooKeeper before Kafka 4.0 removes support |
| RabbitMQ | 3.13+ (2024) | Khepri metadata store (replaces Mnesia) | Optional; improves cluster stability |
| Istio | 1.21+ (2024) | Ambient mesh (sidecar-less option) | Reduces per-pod overhead by ~50% |
| gRPC-Web | 1.5+ (2023) | Stable for production | Use with Envoy or grpc-web npm package |
| NATS | 2.10+ (2024) | JetStream improvements | Competing alternative to Kafka for lighter workloads |

## When to Use / When Not to Use

| Use When | Don't Use When | Use Instead |
|---|---|---|
| Services need independent deployment and scaling | Team is <5 engineers or domain is not well understood | Modular monolith with clear module boundaries |
| Different services need different languages/frameworks | All services share the same database anyway | Monolith or modular monolith |
| You need fault isolation (one service failure != total failure) | Latency budget is <5ms for the full request path | In-process function calls (monolith) |
| Event-driven workflows with multiple independent consumers | You need strict ACID transactions across services | Shared database or distributed transaction coordinator |
| High-throughput async processing (>10K events/s) | Simple CRUD app with <1K users | REST monolith or serverless functions |

## Important Caveats

- gRPC benchmarks showing 2-7x over REST assume internal networks with low latency; over the public internet with TLS, the gap narrows significantly due to connection setup overhead [src7]
- Kafka's ordering guarantee is per-partition only -- if you need global ordering, use a single partition (sacrificing throughput) or implement sequence numbers at the application level [src5]
- Service meshes (Istio, Linkerd) add 1-3ms of latency per hop due to sidecar proxy; ambient mesh (Istio 1.21+) reduces this but is still maturing [src2]
- "Async everywhere" is not a silver bullet -- debugging event-driven systems is significantly harder than synchronous call chains; invest in distributed tracing and event catalog documentation before going fully async [src3]
- Message broker failures are correlated (all consumers affected simultaneously) -- design for broker unavailability with local retry buffers or dual-write patterns [src6]

## Related Units

- [API Gateway Design](/software/system-design/api-gateway/2026)
- [Message Queue and Event-Driven Architecture](/software/system-design/message-queue-event-driven/2026)
- [CQRS and Event Sourcing](/software/system-design/cqrs-event-sourcing/2026)
- [Distributed Rate Limiter Design](/software/system-design/rate-limiter/2026)
- [Monolith to Microservices Migration](/software/migrations/monolith-to-microservices/2026)
- [GraphQL API Architecture at Scale](/software/system-design/graphql-api-architecture/2026)