--- # === IDENTITY === id: software/system-design/chat-app-at-scale/2026 canonical_question: "How do I design a scalable chat application (WhatsApp/Slack clone)?" aliases: - "design scalable chat system" - "WhatsApp system design architecture" - "Slack clone system design" - "real-time messaging system design" - "chat application architecture at scale" - "design messaging platform like Discord" - "scalable instant messaging backend" - "chat system design interview" entity_type: software_reference domain: software > system-design > chat_app_at_scale region: global jurisdiction: global temporal_scope: 2020-2026 # === VERIFICATION === last_verified: 2026-02-23 confidence: 0.91 freshness: quarterly version: 1.0 first_published: 2026-02-23 # === TEMPORAL VALIDITY === temporal_validity: status: stable last_breaking_change: null next_review: 2026-08-22 change_sensitivity: medium # === CONSTRAINTS === constraints: - "WebSocket connections are stateful — never route the same user to different gateway servers mid-session without a reconnection protocol" - "Message ordering must be guaranteed per-conversation — use a monotonic sequence ID per chat, not wall-clock timestamps" - "End-to-end encryption (Signal Protocol) means the server cannot read message content — all search/indexing must happen client-side" - "Database choice is load-dependent: PostgreSQL for <10M messages/day, Cassandra/ScyllaDB for >100M messages/day" - "Never fan out writes to all group members synchronously — use async message queues (Kafka/NATS) to decouple send from deliver" # === SKIP CONDITIONS === skip_this_unit_if: - condition: "Building a simple chatbot or customer support widget with <100 concurrent users" use_instead: "Use a managed service like Intercom, Crisp, or a simple WebSocket server — no distributed architecture needed" - condition: "Need to design a social media feed (Twitter/Instagram) rather than bidirectional messaging" use_instead: "software/system-design/news-feed-system/2026" - condition: "Need video/voice calling architecture rather than text messaging" use_instead: "software/system-design/video-calling-system/2026" # === AGENT HINTS === inputs_needed: - key: scale question: "What is the expected number of concurrent users?" type: choice options: ["Small (<1K)", "Medium (1K-100K)", "Large (100K-10M)", "Massive (>10M)"] - key: chat_type question: "What types of conversations are needed?" type: choice options: ["1:1 only", "1:1 + small groups (<100)", "1:1 + groups + channels (unlimited)", "All including broadcast"] - key: encryption question: "Is end-to-end encryption required?" type: choice options: ["Yes (Signal Protocol)", "No (TLS in transit only)", "Optional per conversation"] # === DISTRIBUTION === canonical_source: "https://knowledgelib.io/software/system-design/chat-app-at-scale/2026" suggested_citation: "Source: knowledgelib.io — AI Knowledge Library (verified 2026-02-23)" # === RELATED UNITS === related_kos: related_to: - id: "software/system-design/rate-limiter/2026" label: "Rate Limiter Design" - id: "software/system-design/notification-system/2026" label: "Push Notification System Design" - id: "software/migrations/monolith-to-microservices/2026" label: "Monolith to Microservices Migration" depends_on: - id: "software/system-design/load-balancer/2026" label: "Load Balancer Design" - id: "software/system-design/message-queue/2026" label: "Message Queue Architecture" solves: - id: "software/system-design/real-time-presence/2026" label: "Real-Time Presence System" alternative_to: - id: "software/system-design/email-system/2026" label: "Email System Design" # === SOURCES (7 authoritative sources) === sources: - id: src1 title: "How Discord Stores Trillions of Messages" author: Discord Engineering url: https://discord.com/blog/how-discord-stores-trillions-of-messages type: technical_blog published: 2023-03-06 reliability: high - id: src2 title: "Real-time Messaging at Slack" author: Slack Engineering url: https://slack.engineering/real-time-messaging/ type: technical_blog published: 2023-03-15 reliability: high - id: src3 title: "How WhatsApp Handles 40 Billion Messages Per Day" author: ByteByteGo url: https://blog.bytebytego.com/p/how-whatsapp-handles-40-billion-messages type: technical_blog published: 2024-01-15 reliability: high - id: src4 title: "Design a Chat Application like WhatsApp" author: AlgoMaster url: https://blog.algomaster.io/p/design-a-chat-application-like-whatsapp type: community_resource published: 2024-06-10 reliability: moderate_high - id: src5 title: "Messenger End-to-End Encryption Overview" author: Meta Engineering url: https://engineering.fb.com/wp-content/uploads/2023/12/MessengerEnd-to-EndEncryptionOverview_12-6-2023.pdf type: official_docs published: 2023-12-06 reliability: high - id: src6 title: "Scaling Pub/Sub with WebSockets and Redis" author: Ably url: https://ably.com/blog/scaling-pub-sub-with-websockets-and-redis type: technical_blog published: 2024-02-20 reliability: moderate_high - id: src7 title: "WhatsApp Engineering — System Design Newsletter" author: Neo Kim url: https://newsletter.systemdesign.one/p/whatsapp-engineering type: technical_blog published: 2024-03-01 reliability: moderate_high --- # Designing a Scalable Chat Application (WhatsApp/Slack Clone) ## TL;DR - **Bottom line**: A scalable chat app requires WebSocket gateways for persistent connections, a message queue (Kafka/NATS) for async delivery, a write-optimized database (Cassandra/ScyllaDB) for message storage, and Redis for presence/pub-sub — separated into stateless services behind a load balancer. - **Key tool/command**: `WebSocket + Kafka + Cassandra + Redis Pub/Sub` - **Watch out for**: Storing messages in a relational database without partitioning — PostgreSQL works for <10M messages/day, but unsharded SQL becomes a bottleneck at scale. - **Works with**: Any language/framework. Production stacks: Erlang/Elixir (WhatsApp/Discord), Java (Slack), Go (large-scale custom), Node.js (startups/MVPs). ## Constraints - WebSocket connections are stateful — never route the same user to different gateway servers mid-session without a reconnection protocol - Message ordering must be guaranteed per-conversation — use a monotonic sequence ID per chat, not wall-clock timestamps - End-to-end encryption (Signal Protocol) means the server cannot read message content — all search/indexing must happen client-side - Database choice is load-dependent: PostgreSQL for <10M messages/day, Cassandra/ScyllaDB for >100M messages/day - Never fan out writes to all group members synchronously — use async message queues to decouple send from deliver - Group messages create O(N) fanout per message — cap group size or use tiered delivery (online members via WebSocket, offline via push queue) ## Quick Reference | Component | Role | Technology Options | Scaling Strategy | |---|---|---|---| | API Gateway / Load Balancer | Route client connections, TLS termination, rate limiting | Nginx, HAProxy, AWS ALB, Envoy | Horizontal — add more LB instances behind DNS round-robin | | WebSocket Gateway | Maintain persistent bidirectional connections with clients | Custom (Elixir/Erlang, Go, Node.js), Socket.IO | Horizontal — shard by user_id; each server handles 100K-1M connections | | Chat Service | Message validation, routing, conversation logic | Java, Go, Node.js, Elixir | Stateless — scale horizontally behind gateway | | Message Queue | Decouple send from deliver, buffer during spikes, guarantee delivery | Apache Kafka, NATS JetStream, RabbitMQ, Amazon SQS | Partition by conversation_id; add partitions for throughput | | Message Storage | Persist all messages durably | Cassandra, ScyllaDB, PostgreSQL (small scale), TiDB | Partition by (conversation_id, message_time); add nodes for capacity | | User/Group Metadata DB | User profiles, contacts, group membership, settings | PostgreSQL, MySQL, CockroachDB | Read replicas + connection pooling; shard at extreme scale | | Presence Service | Track online/offline/typing status per user | Redis (TTL keys + Pub/Sub), custom Erlang process | Redis Cluster with key-based sharding; TTL for auto-expiry | | Media Storage | Store images, videos, voice messages, file attachments | AWS S3, Google Cloud Storage, Cloudflare R2 | Object storage is inherently scalable; use CDN for delivery | | Push Notification Service | Deliver messages to offline/backgrounded users | FCM (Android), APNs (iOS), custom push gateway | Queue-based — consume from message queue for offline users | | Search / Indexing | Full-text search across message history | Elasticsearch, OpenSearch, Meilisearch | Shard by user_id or workspace_id; index asynchronously | | Cache Layer | Hot data: recent messages, session data, user profiles | Redis, Memcached | Redis Cluster; cache aside pattern with TTL | | CDN | Deliver static assets and media globally | Cloudflare, CloudFront, Fastly | Edge-based; auto-scales | | Encryption Service | Key exchange, E2E encryption (Signal Protocol) | libsignal, custom X3DH + Double Ratchet | Stateless — keys stored client-side; server only brokers key exchange | ## Decision Tree > Full script: [decision-tree.txt](scripts/decision-tree.txt) (37 lines) ``` START — How many concurrent users? ├── <1K concurrent users? │ ├── YES → SIMPLE STACK │ │ ├── Single Node.js/Go server with WebSocket (Socket.IO or ws) │ │ ├── PostgreSQL for messages + users # ... (see full script) ``` ## Step-by-Step Guide ### 1. Define message data model and API contract Design the core message schema before writing any code. Every message needs a globally unique ID, a conversation reference, sender, timestamp, and content. Use a snowflake-style ID or ULID for sortable unique IDs. [src4] ```sql -- Core message schema (PostgreSQL for small scale, adapt for Cassandra) CREATE TABLE messages ( message_id BIGINT PRIMARY KEY, -- Snowflake ID (sortable, unique) conversation_id BIGINT NOT NULL, sender_id BIGINT NOT NULL, content TEXT, -- encrypted blob if E2E content_type VARCHAR(20) DEFAULT 'text', -- text, image, video, file media_url TEXT, created_at TIMESTAMP NOT NULL DEFAULT NOW(), status VARCHAR(10) DEFAULT 'sent', -- sent, delivered, read sequence_num BIGINT NOT NULL -- monotonic per conversation ); -- Partition by conversation_id for locality CREATE INDEX idx_messages_conv_seq ON messages (conversation_id, sequence_num DESC); -- Conversation table CREATE TABLE conversations ( conversation_id BIGINT PRIMARY KEY, type VARCHAR(10) NOT NULL, -- 'direct' or 'group' created_at TIMESTAMP NOT NULL DEFAULT NOW(), last_message_at TIMESTAMP ); -- Conversation membership CREATE TABLE conversation_members ( conversation_id BIGINT NOT NULL, user_id BIGINT NOT NULL, role VARCHAR(10) DEFAULT 'member', -- member, admin, owner joined_at TIMESTAMP NOT NULL DEFAULT NOW(), muted_until TIMESTAMP, PRIMARY KEY (conversation_id, user_id) ); ``` **Verify**: Schema supports both 1:1 and group conversations with the same table structure. ### 2. Build the WebSocket gateway layer The gateway is the most connection-intensive component. Each server maintains persistent WebSocket connections with clients. Use consistent hashing on user_id to route reconnections to the same server when possible. [src2] ``` ┌─────────────────────────────────────────┐ │ Load Balancer (L4/L7) │ │ (sticky sessions by user_id hash) │ └──────────┬──────────┬──────────┬────────┘ │ │ │ ┌──────────▼┐ ┌──────▼──────┐ ┌▼─────────┐ │ Gateway-1 │ │ Gateway-2 │ │ Gateway-3 │ │ 500K conns │ │ 500K conns │ │ 500K conns│ └──────┬─────┘ └──────┬──────┘ └─────┬─────┘ │ │ │ ┌──────▼───────────────▼───────────────▼──────┐ │ Redis Pub/Sub Cluster │ │ (cross-server message routing) │ └──────────────────┬──────────────────────────┘ │ ┌──────────────────▼──────────────────────────┐ │ Kafka Cluster │ │ (durable message queue, partitioned by │ │ conversation_id) │ └─────────────────────────────────────────────┘ ``` **Verify**: Load test with `wscat` or `artillery` — each gateway should handle 100K+ concurrent WebSocket connections on a 16GB RAM instance. ### 3. Implement message routing and delivery When User A sends a message to User B, the flow is: Gateway receives -> Chat Service validates -> Kafka queue -> Consumer delivers to User B's gateway (or push notification if offline). [src3] ``` Message Flow (1:1): User A Gateway-1 Chat Service Kafka Gateway-2 User B │ │ │ │ │ │ │──── WS: send ───────>│ │ │ │ │ │ │──── validate ─────>│ │ │ │ │ │<─── ack (msg_id) ──│ │ │ │ │<── WS: ack ──────────│ │ │ │ │ │ │ │── produce ─────>│ │ │ │ │ │ │── consume ────────>│ │ │ │ │ │ │── WS: deliver ─>│ │ │ │ │ │<─ WS: delivered ─│ │<── WS: delivered ────│<──────────────────────────────────── Redis Pub/Sub ──────│ │ Message Flow (Group — fanout): User A ─> Gateway ─> Chat Service ─> Kafka[conv_id partition] │ ┌─────────────┼──────────────┐ │ │ │ Consumer-1 Consumer-2 Consumer-3 (online) (online) (offline) │ │ │ Gateway-X Gateway-Y Push Service │ │ │ User B User C APNs/FCM │ User D (phone) ``` **Verify**: Send a message between two users connected to different gateway servers — message should arrive in <500ms. ### 4. Set up message storage with write optimization For high-volume messaging (>100M messages/day), use Cassandra or ScyllaDB with a partition key of conversation_id and clustering key of message timestamp. This gives you O(1) writes and efficient range reads for conversation history. [src1] ```cql -- Cassandra/ScyllaDB schema for message storage CREATE KEYSPACE chat WITH replication = { 'class': 'NetworkTopologyStrategy', 'us-east-1': 3, 'eu-west-1': 3 }; CREATE TABLE chat.messages ( conversation_id bigint, bucket text, -- monthly bucket: '2026-02' message_id bigint, -- snowflake ID sender_id bigint, content blob, -- encrypted content content_type text, media_url text, created_at timestamp, PRIMARY KEY ((conversation_id, bucket), message_id) ) WITH CLUSTERING ORDER BY (message_id DESC) AND compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_unit': 'DAYS', 'compaction_window_size': 7}; ``` **Verify**: Write throughput test — Cassandra should handle >100K writes/sec per node with proper tuning. Read a conversation's recent 50 messages in <10ms. ### 5. Implement presence and typing indicators Use Redis with TTL keys for presence. Each connected user has a key that expires after 30 seconds. Clients send heartbeats every 15 seconds to refresh the TTL. Typing indicators are ephemeral events — never persist them. [src6] ``` Presence Architecture: Client heartbeat (every 15s) │ ▼ ┌─────────────────┐ ┌─────────────────┐ │ Gateway Server │────>│ Redis Cluster │ │ (receives WS │ │ │ │ heartbeat) │ │ presence:{uid} │ └─────────────────┘ │ TTL = 30s │ │ │ │ Channel: │ │ presence:notify │ └────────┬────────┘ │ Pub/Sub ┌─────────────┼──────────────┐ │ │ │ Gateway-1 Gateway-2 Gateway-3 (pushes to (pushes to (pushes to online online online contacts) contacts) contacts) ``` **Verify**: Set a presence key with 30s TTL, wait 31s, confirm key is gone. Verify typing indicator latency is <200ms end-to-end. ### 6. Add push notifications for offline delivery When the message consumer detects the recipient is offline (no presence key in Redis), route the message to a push notification service instead of a gateway. Use FCM for Android and APNs for iOS. Batch notifications to avoid spamming. [src7] ``` Offline Delivery Flow: Kafka Consumer │ ├── Check Redis: presence:{user_id} exists? │ ├── YES ──> Route to user's Gateway server via Redis Pub/Sub │ └── NO ───> Push Notification Service │ ├── Lookup device tokens from User DB ├── Format notification payload ├── Send via FCM / APNs └── Store message for later sync (user fetches on reconnect) ``` **Verify**: Disconnect a test user, send them a message, verify push notification arrives within 3 seconds. On reconnect, verify all missed messages sync correctly. ### 7. Design media handling pipeline Never send media through the WebSocket connection or message queue. Instead, use a pre-signed URL upload flow: client requests an upload URL, uploads directly to object storage, then sends a message with the media reference. [src3] ``` Media Upload Flow: Client API Server S3/R2 CDN │ │ │ │ │── POST /media/upload ────>│ │ │ │ (file metadata) │── generate ────────>│ │ │ │ presigned URL │ │ │<── { upload_url, key } ───│ │ │ │ │ │ │ │── PUT upload_url ─────────────────────────────>│ │ │ (binary file data) │ │ │ │<── 200 OK ────────────────────────────────────│ │ │ │ │ │ │── WS: send_message ──────>│ │ │ │ { media_key: "abc123" } │ │ │ │ │── (normal message │ │ │ │ routing flow) │ │ │ │ │ │ │ Recipient receives message with media_key │ │ │ Recipient fetches: GET cdn.example.com/abc123 ──────────────────>│ │<── image/video data ──────────────────────────────────────────────│ ``` **Verify**: Upload a 10MB image, confirm it is accessible via CDN URL within 2 seconds. Message containing media reference is <1KB. ## Code Examples ### Node.js: WebSocket Chat Gateway Server > Full script: [node-js-websocket-chat-gateway-server.js](scripts/node-js-websocket-chat-gateway-server.js) (82 lines) ```javascript // Input: WebSocket connections from authenticated clients // Output: Real-time message delivery between connected users // Deps: npm install ws ioredis uuid (ws@8.x, ioredis@5.x) const WebSocket = require('ws'); const Redis = require('ioredis'); # ... (see full script) ``` ### Python: Kafka Message Consumer with Delivery Logic > Full script: [python-kafka-message-consumer-with-delivery-logic.py](scripts/python-kafka-message-consumer-with-delivery-logic.py) (93 lines) ```python # Input: Kafka messages from chat service # Output: Routes messages to online users (via Redis) or push notifications (offline) # Deps: pip install confluent-kafka==2.3.0 redis==5.0.0 firebase-admin==6.4.0 import json import time # ... (see full script) ``` ## Anti-Patterns ### Wrong: Polling for new messages via HTTP ```javascript // BAD -- HTTP polling wastes bandwidth and adds latency. // At 100K users polling every 2s = 50K requests/sec for mostly empty responses. setInterval(async () => { const res = await fetch('/api/messages?since=' + lastTimestamp); const messages = await res.json(); messages.forEach(renderMessage); }, 2000); ``` ### Correct: Use persistent WebSocket connections ```javascript // GOOD -- WebSocket: server pushes instantly, zero wasted requests. // 100K users = 100K persistent connections, messages delivered in <100ms. const ws = new WebSocket('wss://chat.example.com?token=JWT_HERE'); ws.onmessage = (event) => { const message = JSON.parse(event.data); renderMessage(message); }; // Reconnect with exponential backoff on disconnect ws.onclose = () => setTimeout(connect, Math.min(1000 * 2 ** retries++, 30000)); ``` ### Wrong: Sending media files through the message pipeline ```python # BAD -- 10MB video through Kafka + WebSocket = memory pressure, # message queue backlog, and 30-second delivery delays. message = { "conversation_id": 123, "content": base64.b64encode(video_bytes).decode(), # 13MB base64! "content_type": "video" } kafka_producer.produce('chat.messages', json.dumps(message)) ``` ### Correct: Upload media to object storage, send reference only ```python # GOOD -- Upload to S3, send <1KB reference through message pipeline. # Media delivered via CDN, message pipeline stays fast. upload_url = s3.generate_presigned_url('put_object', Params={'Bucket': 'chat-media', 'Key': f'{uuid4()}.mp4'}, ExpiresIn=3600) # Client uploads directly to S3, then sends: message = { "conversation_id": 123, "media_key": "abc123.mp4", # just the reference "content_type": "video" } ``` ### Wrong: Using wall-clock timestamps for message ordering ```python # BAD -- Clock skew between servers causes out-of-order messages. # User sees "Thanks!" before "Happy birthday!" in the conversation. message = { "id": uuid4(), "created_at": datetime.utcnow(), # different servers = different clocks "content": "Hello" } ``` ### Correct: Use per-conversation monotonic sequence numbers ```python # GOOD -- Atomic increment per conversation guarantees order. # Even with clock skew, sequence_num is always correct. sequence_num = redis_client.incr(f"seq:{conversation_id}") message = { "id": generate_snowflake_id(), "sequence_num": sequence_num, # monotonic, gap-free per conversation "created_at": datetime.utcnow(), # informational only, not for ordering "content": "Hello" } ``` ### Wrong: Shared database for all microservices ```sql -- BAD -- Chat service, presence service, and notification service -- all reading/writing the same PostgreSQL instance. -- One slow query in notifications blocks message delivery. SELECT * FROM messages WHERE conversation_id = 123 ORDER BY created_at DESC; -- ^ This query runs on the same DB that presence is hammering with writes ``` ### Correct: Dedicated databases per service domain ```yaml # GOOD -- Each service owns its data store. Failures are isolated. # Chat: Cassandra (write-heavy, partitioned by conversation) # Users: PostgreSQL (relational, read-heavy with replicas) # Presence: Redis (ephemeral, TTL-based) # Search: Elasticsearch (async-indexed from Kafka) services: chat-service: database: cassandra-cluster user-service: database: postgresql-primary presence-service: database: redis-cluster search-service: database: elasticsearch-cluster ``` ## Common Pitfalls - **Thundering herd on reconnect**: When a gateway server restarts, all its clients (100K+) reconnect simultaneously to surviving servers, causing cascading failures. Fix: implement exponential backoff with jitter on client reconnect — `delay = min(base * 2^attempt + random(0, 1000), 30000)`. [src2] - **Unbounded group fanout**: A message to a 10K-member group triggers 10K write operations synchronously. Fix: cap group delivery at 256 via WebSocket, use push notification batching for the rest, and implement tiered delivery (active members first). [src3] - **Hot partition in Cassandra**: A viral group chat puts millions of messages into one partition. Fix: add a time-bucket to the partition key — `PRIMARY KEY ((conversation_id, bucket), message_id)` where bucket is `YYYY-MM`. [src1] - **Missing message on reconnect**: User goes offline for 2 hours, reconnects, but only gets messages since their last seen timestamp (clock skew loses messages). Fix: use the last received `sequence_num` per conversation to request sync — `GET /sync?conversation_id=X&after_seq=N`. [src4] - **Presence storm**: Broadcasting online/offline status to all contacts on every state change overwhelms the system. Fix: batch presence updates (publish every 5s, not on every heartbeat) and only send to contacts who have the app open. [src6] - **No backpressure on WebSocket sends**: Server pushes messages faster than client can process, filling OS send buffers and eventually crashing. Fix: monitor `ws.bufferedAmount`; if it exceeds threshold, queue messages server-side and deliver on drain. [src2] - **Single Redis instance for Pub/Sub**: All cross-server routing through one Redis instance creates a single point of failure and throughput ceiling (~200K msg/sec). Fix: use Redis Cluster with hash-tag routing — `deliver:{user_id}` ensures all operations for a user hit the same shard. [src6] - **Not separating read and write paths**: Read-heavy operations (fetching conversation history, search) compete with write-heavy operations (new messages) on the same database. Fix: CQRS — writes go to Cassandra, reads from a separate read-optimized store or cache layer. [src1] ## Version History & Compatibility | Pattern/Technology | Status | Notable Changes | Migration Notes | |---|---|---|---| | WebSocket (RFC 6455) | Standard since 2011 | HTTP/2 WebSocket (RFC 8441, 2018) | All modern browsers support; use `wss://` in production | | Cassandra 4.x → ScyllaDB | Industry shift 2022-2024 | Discord migrated 2022; 5x perf improvement | ScyllaDB is Cassandra-compatible; drop-in replacement with CQL | | MongoDB → Cassandra | Common migration path | Discord migrated 2017 (data growth) | Denormalize data model for Cassandra's partition-key access pattern | | Signal Protocol (libsignal) | Current standard for E2EE | PQXDH (2023): post-quantum key agreement | WhatsApp, Messenger, Google Messages all use Signal Protocol | | Kafka → NATS JetStream | Emerging alternative 2024+ | Lower latency, simpler operations | NATS better for <1M msg/sec; Kafka for higher throughput and ecosystem | | Socket.IO → native WebSocket | Simplification trend | Socket.IO adds overhead (fallback, rooms) | For >100K connections, use raw `ws` library for lower memory footprint | ## When to Use / When Not to Use | Use When | Don't Use When | Use Instead | |---|---|---| | Building a real-time 1:1 and group messaging product | Need async communication (email-like, hours between replies) | Email system design or task queue architecture | | Users expect <500ms message delivery latency | Message delivery can be delayed 5-30 seconds | HTTP-based notification system with polling | | Need to support 10K+ concurrent connected users | <100 concurrent users on a single server | Simple WebSocket server without distributed infrastructure | | Messages must be persisted and searchable | Ephemeral messages only (no persistence needed) | WebRTC data channels for P2P ephemeral messaging | | Multiple client platforms (web, iOS, Android, desktop) | Single-platform internal tool | Platform-specific SDK (e.g., Firebase Realtime Database) | | End-to-end encryption is a requirement | Server needs to read/process message content (moderation, NLP) | Standard TLS + server-side encryption at rest | ## Important Caveats - End-to-end encryption eliminates server-side search, content moderation, and spam filtering — these must be implemented client-side or on metadata only, which significantly limits anti-abuse capabilities - WebSocket connections consume server memory (2-10KB per connection) — at 1M connections, that is 2-10GB of RAM just for connection state, before any application logic - Cassandra/ScyllaDB require careful data modeling upfront — changing partition keys later means migrating the entire dataset, which at trillions of rows takes weeks - Group size limits have cascading effects: a 256-member group with 100 messages/day generates 25,600 fanout events/day; a 100K-member channel (Discord-style) requires a fundamentally different fanout architecture (lazy pull vs eager push) - Multi-region deployment introduces message ordering challenges — use a single Kafka cluster per conversation partition or implement vector clocks for conflict resolution - Mobile clients have battery and bandwidth constraints — implement message compression (protobuf instead of JSON), batch sync on reconnect, and respect OS background execution limits ## Related Units - [Monolith to Microservices Migration](/software/migrations/monolith-to-microservices/2026)