Queues vs Streams vs SSE vs WebSockets: What Each One Actually Solves

A queue, an event stream, Server-Sent Events, and a WebSocket can all move data from one place to another. That similarity is what makes the choice confusing.

They solve different problems:

• A queue hands work to something that can process it later.
• An event stream keeps a history that one or more consumers can read and replay.
• SSE keeps an HTTP response open so a server can push updates to a browser.
• A WebSocket keeps a two-way connection open so either side can send at any time.

The useful question is not simply, “Do I need real-time?” Start with four narrower questions:

1. Is a person actively waiting for the result?
2. Does data move one way or both ways?
3. What happens if a message is lost?
4. Will another consumer need to read it later?

Once those answers are clear, the choice usually becomes much less dramatic.

“Real-time” covers very different expectations

A typing indicator has to feel immediate. A dashboard can update every few seconds and still feel live. An email sent a minute after signup is asynchronous, but the user probably does not experience it as delayed.

Those cases should not share an architecture just because someone called all of them real-time.

A person watching generated text needs progressive delivery. A background worker resizing an image needs durable work ownership. An analytics service may need to replay last week’s events. A multiplayer game needs low-latency communication in both directions.

Latency matters, but it is only one part of the decision.

Message queues: hand off work and move on

A message queue is useful when the component accepting work should not also be responsible for finishing it immediately.

An API can validate a request, add a job, and respond. A worker picks up the job independently. If processing fails, the queue can retry it or move it to a dead-letter queue after repeated failures.

Common queue systems include BullMQ, RabbitMQ, AWS SQS, Celery, and Google Cloud Tasks.

Queues fit work such as:

• Sending email
• Processing uploads
• Generating reports
• Running builds
• Resizing media
• Calling a slow third-party API
• Any task that should survive beyond the request that created it

A queue is not inherently slow. A Redis-backed BullMQ worker can start a job very quickly. The important difference is ownership: the work belongs to the queue and worker, not to the original HTTP connection.

A small BullMQ example looks like this:

1import { Queue, Worker } from 'bullmq';
2import Redis from 'ioredis';
3
4const connection = new Redis({ maxRetriesPerRequest: null });
5const reportQueue = new Queue('reports', { connection });
6
7await reportQueue.add(
8  'generate-monthly-report',
9  { accountId: 'acct_123' },
10  {
11    attempts: 3,
12    backoff: { type: 'exponential', delay: 2_000 },
13  },
14);
15
16const worker = new Worker(
17  'reports',
18  async (job) => {
19    await generateMonthlyReport(job.data.accountId);
20  },
21  { connection, concurrency: 5 },
22);

The API can return after enqueueing the job. The worker can run in a different process, restart independently, and control concurrency without blocking the API server.

What a queue does not give you

A queue does not automatically tell the browser that a job is 37% complete. It also does not automatically give several independent systems a permanent copy of every message.

Those are separate concerns.

You can publish progress through SSE or WebSockets, and you can persist status in a database. For replayable multi-consumer history, an event stream is usually the closer fit.

Event streams: keep the history

In this article, “stream” means an event stream such as Kafka, Redis Streams, or NATS JetStream—not merely bytes flowing over an HTTP response.

A queue is primarily about distributing work. An event stream is primarily about retaining an ordered log of what happened.

Suppose an order service emits order.created.

Several systems may care about it:

• Inventory reserves stock.
• Analytics records the purchase.
• Fraud detection scores it.
• Notifications send a confirmation.
• An audit service stores the event for compliance.

Each consumer can read at its own pace. A new consumer can start later and replay older events, subject to the system’s retention policy.

That makes event streams useful for:

• Event-driven service integration
• Audit trails
• Analytics pipelines
• Change-data capture
• Event sourcing
• Rebuilding projections or indexes
• Fan-out to independent consumers

Ordering is scoped, not global

Kafka preserves order within a partition, not across an entire topic.

If all events for one account must stay ordered, they need a stable partition key such as accountId. That keeps related events together, but it also means a very busy account can create a hot partition.

The real design question is therefore:

What is the smallest unit for which ordering must be guaranteed?

It may be a user, account, chat room, device, document, or aggregate. Global ordering is expensive and usually unnecessary.

A stream is not a request-response API

Kafka can technically be forced into request-response patterns, but it is usually the wrong abstraction. If one service needs an immediate answer from another, HTTP or gRPC is simpler.

Use an event stream when the event itself matters beyond the lifetime of one caller.

Server-Sent Events: server-to-browser updates over HTTP

Server-Sent Events keeps an HTTP response open and sends a sequence of text events to the browser.

It is a good fit when the browser mostly listens:

• LLM output
• Build logs
• Job progress
• Notifications
• Monitoring dashboards
• Live read-only feeds

An Express endpoint can stream events like this:

1import express from 'express';
2
3const app = express();
4
5app.get('/events', (req, res) => {
6  res.setHeader('Content-Type', 'text/event-stream');
7  res.setHeader('Cache-Control', 'no-cache');
8  res.setHeader('Connection', 'keep-alive');
9  res.setHeader('X-Accel-Buffering', 'no');
10  res.flushHeaders();
11
12  const timer = setInterval(() => {
13    const event = {
14      id: crypto.randomUUID(),
15      createdAt: Date.now(),
16    };
17
18    res.write(`id: ${event.id}\n`);
19    res.write(`event: update\n`);
20    res.write(`data: ${JSON.stringify(event)}\n\n`);
21  }, 1_000);
22
23  req.on('close', () => clearInterval(timer));
24});

For a simple unauthenticated GET, the browser’s EventSource API is convenient:

1const source = new EventSource('/events');
2
3source.addEventListener('update', (event) => {
4  console.log(JSON.parse(event.data));
5});

EventSource reconnects automatically. When the server sends an id: field, the browser can include Last-Event-ID on reconnection.

That does not make replay automatic by itself. The server still needs to retain events and know how to return everything after that ID.

Why some apps parse SSE through fetch

EventSource has a deliberately small API. It only performs GET requests and gives limited control over headers.

When an application needs a POST body, custom authentication, an AbortSignal, or explicit cursor handling, it can use fetch and parse the response stream as SSE.

That is still SSE. The transport format does not require the EventSource client.

Where SSE becomes awkward

SSE is one-way. The browser sends commands through normal HTTP requests while receiving updates through the open stream.

That is often perfectly fine. A user can submit a prompt with POST /messages and receive progress from GET /runs/:id/events.

It becomes less natural when both sides need frequent, unpredictable messages on the same long-lived connection. That is where WebSockets are stronger.

WebSockets: one connection, two-way communication

A WebSocket begins as an HTTP request and then upgrades to a persistent full-duplex connection. Both the client and server can send messages whenever they need to.

That fits:

• Chat
• Multiplayer games
• Collaborative editing
• Presence and typing indicators
• Remote control surfaces
• Live sessions where client and server continuously exchange state

A minimal server using ws looks like this:

1import { WebSocketServer } from 'ws';
2
3const wss = new WebSocketServer({ port: 8080 });
4
5wss.on('connection', (socket) => {
6  socket.on('message', (raw) => {
7    const message = JSON.parse(raw.toString());
8    socket.send(JSON.stringify({ type: 'ack', messageId: message.id }));
9  });
10
11  const heartbeat = setInterval(() => socket.ping(), 30_000);
12  socket.on('close', () => clearInterval(heartbeat));
13});

The difficult part is rarely opening the connection. It is everything around it:

• Authentication and token refresh
• Reconnection with exponential backoff and jitter
• Resubscribing after reconnect
• Duplicate messages
• Missed-message recovery
• Slow-client buffers
• Presence cleanup
• Routing across multiple server instances
• Deployments that trigger reconnect storms

A WebSocket gives you a live pipe. It does not give you durable delivery, replay, or exactly-once processing.

If a message must survive a disconnect, persist it somewhere outside the socket.

A practical decision process

I usually reduce the choice to a few checks.

Is a human actively waiting?

If no one is watching, start with a queue.

If a person needs progressive feedback, add SSE or WebSockets for delivery to the UI. The durable job can still run through a queue underneath.

Which direction does data move?

• Mostly server to browser: SSE
• Browser sends occasional commands, server sends continuous updates: HTTP plus SSE
• Both sides send frequently and unpredictably: WebSockets
• Service-to-service work distribution: queue
• Multiple independent consumers need the same retained history: event stream

Can the data be lost?

• Typing indicator: often acceptable
• Build progress tick: usually recoverable from current state
• Payment instruction: never acceptable
• Audit event: should be persisted and replayable

Transport speed does not determine durability. Decide where the source of truth lives.

Does anyone need to replay it?

If a late consumer must reconstruct what happened, keep a log or event table. Redis Pub/Sub alone is not enough because messages disappear when no subscriber is listening.

Quick comparison

Where these patterns meet in a real product

Most non-trivial systems use more than one of them.

Edward, an AI web-app builder I built, is one example. The product does not treat “streaming” as one giant connection that owns the whole workflow.

A prompt creates an agent run. That run is stored and placed on a Redis-backed BullMQ queue, so the work is owned by a worker rather than by the HTTP request that accepted the prompt.

As the worker runs the agent loop, it emits structured events such as:

• text
• thinking_content
• file_start
• file_content
• install_content
• command
• build_status
• preview_url

Those events are persisted with sequence numbers. New events are also published through Redis so connected API processes can deliver them immediately. The browser receives them over SSE and keeps the latest event cursor. After a refresh or disconnect, it can reconnect and request events after that cursor.

Generation and preview compilation are also separate jobs. Edward has an agent-run queue for the model-driven session and a build queue for compiling generated projects, uploading preview assets, and scheduling backups.

The important lesson is broader than Edward:

Durable work, durable history, and live delivery are separate responsibilities.

A queue can own the work. A database or event log can own the history. SSE or WebSockets can own the current connection. Redis Pub/Sub can help processes fan out live updates without pretending to be permanent storage.

You do not need every component in every application. Separating the responsibilities makes it easier to remove what you do not need.

Common mistakes

Treating the browser connection as the job

A long-running task should not disappear because the user closed a tab or changed networks.

The browser can start or observe work. A queue or durable run record should own it.

Polling every second by default

Polling is not always bad. It is easy to cache, debug, and operate, and it may be enough for infrequent updates.

It becomes wasteful when thousands of clients repeatedly ask for a state that rarely changes, or when the experience genuinely needs progressive output. SSE is often the simplest upgrade.

Using Redis Pub/Sub as a durable queue

Redis Pub/Sub only reaches active subscribers. If a consumer is offline, the message is gone.

Use it for ephemeral notifications between processes. Use BullMQ, Redis Streams, Kafka, SQS, RabbitMQ, or persisted database state when delivery matters.

Assuming WebSockets provide reliability

TCP preserves byte order within one connection. That does not solve application-level recovery across reconnects.

You still need message IDs, acknowledgements, deduplication, and a way to retrieve anything missed while disconnected.

Reaching for Kafka because the system is “large”

Kafka is valuable when retained event history, throughput, partitioned ordering, and multiple consumer groups justify its operational cost.

It is not automatically the right choice for a task queue, a live browser connection, or a straightforward service call.

The production details that decide whether it works

Choosing the category of tool is only the beginning. Production behaviour depends on delivery semantics, idempotency, ordering, backpressure, and recovery.

Delivery semantics and idempotency

Most durable messaging systems are effectively at least once: a message may be processed more than once.

That happens when a worker finishes the side effect but crashes before acknowledging the message. The queue cannot know whether the work completed, so it retries.

Consumers should therefore be idempotent wherever possible.

For a database-backed job, a common pattern is to claim work with a conditional update:

1UPDATE jobs
2SET status = 'processing', started_at = NOW()
3WHERE id = $1 AND status = 'queued'
4RETURNING id;

If no row is returned, another worker already claimed it or it is no longer queued.

For an external payment API, use the provider’s idempotency key as well. A Redis lock alone cannot prove whether a remote charge succeeded before your process crashed.

Ordering

Ordering guarantees always have a boundary:

• Kafka: within a partition
• SQS FIFO: within a message group
• RabbitMQ: affected by queues, acknowledgements, redelivery, and consumer concurrency
• Redis Streams: insertion order in the stream, while consumer completion may differ
• SSE or WebSockets: ordered within one connection, but reconnect recovery is your responsibility

Before asking for ordering, name the entity that needs it.

Backpressure

Backpressure is what happens when producers create data faster than consumers can handle it.

Depending on the system:

• A queue grows.
• Kafka consumer lag increases.
• A WebSocket send buffer fills.
• An SSE response starts blocking or consuming memory.
• A process drops events or crashes.

You need an explicit policy:

• Slow the producer
• Limit worker concurrency
• Batch work
• Drop low-value updates
• Disconnect slow clients
• Scale consumers
• Reject new work when the backlog is unsafe

“Just buffer it” only postpones the decision.

Reconnection and replay

A reconnecting client needs answers to three questions:

1. What was the last event I successfully applied?
2. Where can I retrieve everything after it?
3. How do I avoid applying duplicates?

A monotonically increasing sequence number is often easier to reason about than timestamps. The live transport can be lossy as long as the client can recover from the durable history.

Failure isolation

Ask how each boundary fails:

• If the API restarts, do queued jobs continue?
• If Redis Pub/Sub drops an event, can the client replay it elsewhere?
• If the worker crashes mid-job, is the side effect safe to retry?
• If the browser reconnects, can it restore subscriptions and state?
• If a downstream service slows down, which backlog grows?

The goal is not to prevent every failure. It is to stop one failure from silently becoming data loss somewhere else.

Operational cost

The technically most capable tool is not always the best tool.

• SQS removes most queue operations but ties you to a managed service and usage pricing.
• BullMQ is productive when Redis is already part of the stack, but queue load now shares Redis capacity.
• RabbitMQ gives rich routing and mature queue semantics, with another cluster to operate.
• Kafka is excellent for retained high-throughput event logs, but brings partitioning and operational complexity.
• SSE works with normal HTTP infrastructure, though buffering and long-lived connection limits still matter.
• WebSockets demand more connection lifecycle and scale-out work.

Pick the simplest system that preserves the guarantees the product actually needs.

FAQ

Should I use SSE or WebSockets for LLM output?

SSE is usually enough when the server is sending generated output and the client sends prompts or cancellation through ordinary HTTP requests.

Use WebSockets when the session genuinely requires frequent two-way messages over the same connection.

Is Kafka a queue?

Kafka can distribute records among consumers in a consumer group, which can resemble a queue. Its defining model is still a retained partitioned log that consumers track with offsets.

Use it when replay and independent consumer groups matter, not merely because work needs to happen asynchronously.

Can Redis Streams replace Kafka?

For smaller systems, sometimes. Redis Streams provides persistence, consumer groups, pending entries, and replay without operating Kafka.

Kafka generally offers stronger tooling and a more natural model for very large retained event pipelines. The right answer depends on throughput, retention, operational experience, and failure requirements.

Is polling always wrong?

No. Polling every 30 seconds for a rarely changing status can be simpler and more reliable than maintaining thousands of long-lived connections.

Use push when latency or request volume makes polling materially worse.

Do I need a queue and SSE together?

Often. The queue owns the job; SSE shows its progress.

They solve different parts of the same user experience.

The rule I keep coming back to

Do not choose the tool from the word “real-time.”

Write down:

• Who owns the work?
• Who needs the result?
• How quickly do they need it?
• Can it be lost?
• Must it be replayed?
• What happens after a disconnect or retry?

Then assign each responsibility separately.

A queue is not a UI transport. SSE is not durable storage. A WebSocket is not a job runner. Redis Pub/Sub is not a database. Kafka is not a replacement for every message moving through a system.

Once those boundaries are explicit, the architecture usually gets simpler—not more complicated.