Pub/Sub

Imagine you are building an e-commerce app. When a customer places an order, many things must happen: send a confirmation email, reduce the stock, update the analytics dashboard, and tell the shipping team.

The simple way is for the Order Service to call each of these services directly. But this causes problems:

• Tight coupling: The Order Service must know about every other service. To add a new one (say, a loyalty points service), you must change and redeploy the Order Service.
• Slow requests: The customer waits while the Order Service calls four services one after another.
• Fragile: If the email service is down, the order request may fail, even though the order itself was saved fine.

The root problem is that the Order Service knows too much about who cares about its news. A better plan is: the Order Service just announces "an order was placed", and anyone who is interested reacts to it.

This is exactly what pub/sub is for.

What Is Pub/Sub?

Pub/Sub (publish/subscribe) is a way of sending messages where the sender does not know who will receive them.

There are three main parts:

• Publisher: the service that announces that something happened.
• Topic: a named channel, like order-events. Publishers send to it.
• Subscriber: a service that listens to a topic and reacts.

The system in the middle that receives and delivers messages is the broker. Examples are Amazon SNS, Google Cloud Pub/Sub, Kafka, and Redis Pub/Sub.

1                                      +--> Email Service
2Order Service  --->  Topic            |
3 (Publisher)         "order-events"  -+--> Inventory Service
4                                      |
5                                      +--> Analytics Service
6                                      |
7                                      +--> Shipping Service

The publisher sends one message. The broker gives every subscriber its own copy.

Why Is This Useful?

• The publisher does not know who the subscribers are, and does not care.
• You can add a new subscriber (like loyalty points) without touching the publisher.
• Each subscriber works on its own. A slow or broken one does not stop the others.

Back to the Order Example

Now the Order Service saves the order, publishes an OrderPlaced event, and replies right away. The email, stock, analytics, and shipping services each react in their own time.

Pub/Sub vs Message Queues

Both use a broker in the middle, but they solve different problems.

Queue = share the work. Pub/sub = share the news.

Using Both Together

In real systems, we often combine them. The topic fans out the message, and each subscriber has its own queue to hold its copy. Inside each subscriber, many workers read from that queue to share the work.

1                        +--> Email Queue      --> Email Workers
2Topic "order-events" ---+--> Inventory Queue  --> Inventory Workers
3                        +--> Analytics Queue  --> Analytics Workers

This is a common pattern, for example SNS + SQS on AWS.

The Life of a Message

Let us follow one event from start to end.

1Publisher        Topic / Broker            Subscriber A     Subscriber B
2   |                   |                        |                |
3   |-- publish ------->|                        |                |
4   |<-- confirm -------|                        |                |
5   |                   |-- copy 1 ------------->|                |
6   |                   |-- copy 2 ---------------------------->  |
7   |                   |<-- acknowledge --------|                |
8   |                   |<-- acknowledge ------------------------ |

Step 1: The publisher sends the event. The broker receives it and confirms. The publisher moves on.

Step 2: The broker makes a copy for every subscription. A subscription is a subscriber's link to a topic. Each subscription gets its own copy.

Step 3: Each subscriber processes its copy and acknowledges. Each subscriber tracks its own progress.

Each Subscriber Is Independent

• If Subscriber A is slow, Subscriber B is not affected.
• If Subscriber A fails, only A's copy is retried. B already finished.
• Retry limits and dead-letter queues are set per subscription, so one bad subscriber does not block the rest.

The Catch: Duplicates

Like queues, most brokers deliver at least once. If a subscriber finishes the work but crashes before acknowledging, it gets the same event again. So every subscriber must be safe to run twice, for example by checking the event ID and skipping events it has already handled.

Push vs Pull Delivery

There are two ways the broker can give messages to a subscriber.

Push: the broker sends the message to the subscriber, usually by calling an HTTP endpoint (a webhook).

Pull: the subscriber asks the broker for messages when it is ready.

1Push:   Broker  ---- message ---->  Subscriber
2 
3Pull:   Subscriber  ---- "any messages?" ---->  Broker
4        Subscriber  <---- messages ------------  Broker

Simple rule: use push for light, quick reactions. Use pull when the work is heavy and you want to control the speed.

Ephemeral vs Durable Pub/Sub

Another important question: what happens to a message if a subscriber is not there when it is published?

Ephemeral Pub/Sub

The broker delivers the message only to subscribers connected right now. It does not store it. If a subscriber is offline, it misses the message forever. Redis Pub/Sub works this way.

This is fine for things where a missed message does not matter:

• Live chat typing indicators
• "User is online" status
• Live score or price updates

Durable Pub/Sub

The broker stores messages for each subscription until the subscriber acknowledges them. If a subscriber is offline, the messages wait, and it catches up when it returns. SNS with SQS, Google Cloud Pub/Sub subscriptions, and Kafka work like this.

Kafka goes one step further. It keeps messages for a set time even after they are read, so a subscriber can replay old events.

If missing a message is not acceptable, use durable pub/sub.

Message Ordering

Like queues, pub/sub does not always keep order. Workers run at different speeds, and retries can bring an old message back after newer ones.

Order can matter. Say the events are OrderPlaced, then OrderCancelled. If a subscriber handles them in the wrong order, it may think the order is still active.

The usual fix is the same as for queues: keep order only for related messages, using a key such as the order ID.

• Kafka: the partition key
• Google Cloud Pub/Sub: the ordering key
• SNS FIFO: the message group ID

All events with the same key reach a subscriber in order. Events with different keys can be processed in parallel.

Keep order only inside the business entity that needs it (like an order ID), not across the whole topic.

Scaling Subscribers

Pub/sub scales in two directions.

1. Add more subscribers. Each new subscriber gets its own copy of every message. It does not slow down the existing ones.

2. Add more workers inside one subscriber. If one subscriber is too slow, run many instances of it that share one subscription. They split the messages between them, just like competing consumers in a queue. Kafka calls this a consumer group.

1                       Subscription A  --> A-worker 1, A-worker 2, A-worker 3
2Topic "order-events" --+
3                       Subscription B  --> B-worker 1

• Each subscription gets every message.
• Inside one subscription, each message goes to one worker.

Watch the Backlog Per Subscription

Each subscription has its own backlog. A slow subscriber's backlog grows, while the others stay empty. So monitor, for each subscription:

• Queue depth: how many messages are waiting
• Age of the oldest message: how long the oldest one has waited

Publishing Reliably: The Outbox Pattern

Here is a tricky problem. When an order is placed, the Order Service must do two things:

1. Save the order in its database.
2. Publish the OrderPlaced event to the broker.

These are two different systems, so you cannot wrap them in one transaction. Things can go wrong:

• Save works, then the service crashes before publishing. The order exists, but nobody hears about it.
• Publish works, but the save fails. Subscribers react to an order that does not exist.

The Fix: An Outbox Table

Instead of publishing directly, write the event into an outbox table in the same database transaction as the order.

1Step 1: One database transaction
2   +--------------------------------------+
3   |  INSERT order 789                    |
4   |  INSERT outbox: "OrderPlaced 789"    |   <-- both succeed or both fail
5   +--------------------------------------+
6 
7Step 2: A separate relay process
8   Outbox table  --->  Relay  --->  Broker (publish)
9                         |
10                         +--> mark the outbox row as published

1. The order and the outbox row are saved together. Either both are saved, or neither.
2. A separate relay process keeps reading new outbox rows, publishes them to the broker, and marks them as published.

Now an event is never lost, because it is saved together with the order.

The Catch: Duplicates Again

If the relay publishes an event but crashes before marking the row, it will publish it again later. So publishing is at least once, and subscribers must ignore duplicates using the event ID.

Designing Events

Good event design makes the whole system easier to live with.

Name events as facts in the past. An event says what already happened: OrderPlaced, PaymentFailed, VideoUploaded. This is different from a command like SendEmail, which tells one service to do something. Commands belong in queues. Events belong in pub/sub.

Include a few standard fields:

• Event ID: so subscribers can detect duplicates.
• Event type: so subscribers know what happened.
• Version: so the format can change safely.
• Timestamp: so subscribers know when it happened.

1{
2  "eventId": "evt-9c2b41",
3  "type": "OrderPlaced",
4  "version": 1,
5  "occurredAt": "2026-10-18T10:30:00Z",
6  "data": {
7    "orderId": "789",
8    "customerId": "42",
9    "totalAmount": 1499
10  }
11}

Decide how much data to include.

More tips:

• Do not put secrets or sensitive data in events. Many services will read them.
• Change events carefully. Add new fields instead of removing or renaming old ones, so older subscribers keep working. Use the version field when a bigger change is needed.

When to Use Pub/Sub

Use pub/sub when many services need to react to the same thing, and the publisher should not need to know about them. For example:

• Order or user events that trigger email, stock, analytics, and shipping
• Notifications sent to many devices or services
• Keeping data in sync, like updating a search index or cache when data changes
• Adding new features later, because a new subscriber needs no change to the publisher

It also helps when you want the publisher and subscribers to fail and scale separately.

Do not use pub/sub when:

• One worker should do the task, such as processing one video. Use a message queue.
• The caller needs an immediate answer, such as checking a password. Use a direct request.
• There is only one receiver and it will never change. A queue is simpler.

Summary

Pub/sub lets a publisher announce an event without knowing who listens. Every subscriber gets its own copy and works at its own pace.

• A topic is the channel. A subscription is each subscriber's link to it.
• Push is fast and simple. Pull gives the subscriber control.
• Durable pub/sub stores messages. Ephemeral pub/sub does not.
• Subscribers scale by adding more subscribers, or more workers inside one.

To use pub/sub safely, also remember:

• Handle duplicates, because delivery is usually at-least-once.
• Use ordering keys only where order matters.
• Use the outbox pattern so events are never lost between the database and the broker.
• Keep events as small, versioned facts in the past tense.
• Watch the backlog of every subscription.

Interview Answer: What Is Pub/Sub?

Pub/Sub is a messaging pattern where a publisher sends an event to a topic without knowing who will receive it, and the broker delivers a copy to every subscriber. This decouples services: the publisher does not need to know the subscribers, and new subscribers can be added without changing it.

It is different from a message queue, where each message goes to only one consumer. A queue is for handing out work, and pub/sub is for broadcasting that something happened. We often combine them, giving each subscriber its own queue.

Brokers can push messages to subscribers or let them pull, and they can be ephemeral, which drops messages for offline subscribers, or durable, which stores them. Delivery is usually at least once, so subscribers must handle duplicates. To publish reliably, we use the outbox pattern, which saves the event in the same database transaction as the data change.

We use pub/sub when many services react to the same event, and avoid it when only one worker should do a task or when the caller needs an immediate answer.