Deep dive · Core

Design a chat / messaging

A stateful-connection problem in a world of stateless servers. It tests whether you understand connection routing, message ordering, at-least-once delivery, and the offline case that candidates always forget.

What it trains: WebSockets, ordering, delivery guarantees, presence

1. Requirements

Functional

1:1 and group messaging with history
Delivery states: sent, delivered, read
Online/offline presence and typing indicators
Push notification when the recipient is offline

Non-functional

End-to-end message delivery under ~200ms for online users
Messages must never be lost, and must appear in a consistent order per conversation
Support tens of millions of concurrent long-lived connections
Availability over strict consistency — a message arriving twice beats one arriving never

2. Back-of-envelope numbers

50M daily users × 40 messages ≈ 2B messages/day ≈ 23K writes/sec, peaking ~3× that
10M concurrent WebSocket connections ÷ ~50K per server ≈ 200 chat servers
1 KB per message × 2B/day ≈ 2 TB/day of message history — cheap storage, expensive indexing
Group of 500 members means one write fans out to 500 deliveries: fan-out dominates, not ingest

3. Architecture

  ┌────────┐  WebSocket   ┌──────────────┐
  │ Client │◀────────────▶│  Chat server │──┐
  └────────┘              │   (node 7)   │  │
                          └──────┬───────┘  │
  ┌────────┐              ┌──────▼───────┐  │  lookup / publish
  │ Client │◀────────────▶│  Chat server │  │
  └────────┘              │   (node 12)  │  │
                          └──────┬───────┘  │
                                 │          ▼
                     ┌───────────▼──────────────────┐
                     │  Redis: connection registry  │
                     │  user → chat server node     │
                     │  + pub/sub routing channel   │
                     └───────────┬──────────────────┘
                                 │
      ┌──────────────┐   ┌───────▼────────┐   ┌───────────────┐
      │ Message store│◀──│  Message queue │──▶│ Push service  │
      │ (Cassandra)  │   │    (Kafka)     │   │ (APNs / FCM)  │
      └──────────────┘   └────────────────┘   └───────────────┘

The connection registry turns a stateful problem back into a routable one: any server can find the node holding a given user's socket.

Connections, not requests

Clients hold a WebSocket to a chat server. Because the connection is sticky, the system needs a registry — user_id → server_id in Redis with a heartbeat TTL — so a message arriving on node 7 can be routed to the recipient's socket on node 12, usually via a pub/sub channel per node.

Write path

Client sends → chat server assigns a server-side sequence number → persists to the message store → acknowledges to the sender (sent) → routes to recipients online, or enqueues a push notification. Persist before you ack: acking first is how you lose messages.

Data model

Partition by conversation_id, cluster by message_id descending. This gives you 'load the last 50 messages in this conversation' as a single sequential read — the dominant query. A separate per-user inbox table tracks unread counts and last-read pointers.

4. Deep dives

This is where levels are decided. Go deep on one or two of these rather than shallow on all of them.

Ordering

Client timestamps are unusable — clocks drift and clients lie. Assign ordering server-side per conversation: a monotonic sequence from the conversation's owning partition, or Snowflake IDs (timestamp + node + counter) which are roughly time-sortable and globally unique. Clients sort by that ID, not by arrival, so late deliveries slot into the right place.

Delivery guarantees and idempotency

Networks give you at-least-once at best. Have the client generate a UUID per message and treat it as an idempotency key: a retried send with the same key is deduplicated server-side. Client-side, dedupe by message ID on render. Exactly-once delivery is a marketing term; at-least-once plus idempotency is the real implementation.

sent — server persisted it
delivered — recipient's device acked receipt
read — recipient opened the conversation and the read pointer advanced

Offline users and sync

If the recipient has no live connection, the message is already persisted, so nothing is lost — enqueue a push notification. On reconnect, the client sends its last-seen message ID and pulls everything after it. This 'sync cursor' model is much simpler and more robust than a per-user pending-message mailbox.

Group fan-out

For a small group, fan out on write to every member's connection. For very large groups (thousands of members), fan-out on write becomes a write amplification bomb — switch to fan-out on read, where members pull the conversation on open, and only push notifications to active participants. Mentioning this threshold unprompted is a strong senior signal.

Presence and typing indicators

Presence is a Redis key with a short TTL refreshed by heartbeat — no heartbeat, no presence, no cleanup job needed. Typing indicators should be best-effort, unpersisted, and rate-limited; they are the single easiest way to accidentally multiply your message volume by ten.

Scaling connections

Layer-4 load balancers with long-lived connections mean scaling is about connection count, not QPS. Deploys are the hard part: dropping 50,000 sockets at once causes a reconnect thundering herd, so drain gradually and have clients reconnect with jittered exponential backoff.

5. Tradeoffs to argue

WebSocket vs long polling

Upside: WebSockets give true bidirectional push with low overhead

Cost: Stateful servers, harder deploys, connection registry required

Fan-out on write vs read

Upside: Write fan-out makes opening a chat instant

Cost: Large groups multiply write cost; read fan-out inverts the tradeoff

Per-conversation sequence vs Snowflake IDs

Upside: Sequences give strict order within a conversation

Cost: Requires a coordination point; Snowflake is decentralised but only approximately ordered

6. Questions the interviewer will ask

A chat server dies holding 50,000 connections. Walk me through the next ten seconds.
Two users send a message in the same conversation at the same millisecond. Which one is first, and do both clients agree?
How does a user with three devices keep read state in sync?
How would end-to-end encryption change this design?

7. Mistakes that sink this answer

Ordering by client timestamp
Acknowledging the sender before the message is durably stored
Forgetting the offline path entirely
Treating a 10,000-member group the same as a 3-person chat

Practice this live

Run a mock system design interview and get scored against your target level.

Start mock interview