-
-
Notifications
You must be signed in to change notification settings - Fork 761
Clustering Kafka and Redis
By default, Atmosphere broadcasts messages to clients connected to the same JVM. When you scale out to multiple nodes behind a load balancer, messages broadcast on Node A won't reach clients on Node B.
Atmosphere 4.0 ships two clustering modules that solve this by relaying broadcasts through an external message broker:
| Module | Artifact | Broker | Transport |
|---|---|---|---|
| Kafka | atmosphere-kafka |
Apache Kafka | Kafka topics (one per Broadcaster) |
| Redis | atmosphere-redis |
Redis | Redis pub/sub channels |
Both modules use the same pattern: each Broadcaster publishes outgoing messages to the broker and consumes messages from other nodes, with built-in echo prevention (a node never re-delivers its own messages).
┌───────────────┐
│ Kafka / Redis │
└──────┬────────┘
┌───────┴───────┐
│ │
┌──────▼──────┐ ┌─────▼───────┐
│ Node A │ │ Node B │
│ Broadcaster │ │ Broadcaster │
│ ┌──┐ ┌──┐ │ │ ┌──┐ ┌──┐ │
│ │C1│ │C2│ │ │ │C3│ │C4│ │
│ └──┘ └──┘ │ │ └──┘ └──┘ │
└─────────────┘ └─────────────┘
When C1 sends a message, Node A's Broadcaster:
- Delivers locally to C1 and C2
- Publishes to Kafka/Redis
- Node B's Broadcaster receives it and delivers to C3 and C4
<dependency> <groupId>org.atmosphere</groupId> <artifactId>atmosphere-kafka</artifactId> <version>4.0.0</version> </dependency>
Set KafkaBroadcaster as the default Broadcaster class and provide the Kafka bootstrap servers.
Spring Boot (application.yml):
atmosphere: broadcaster-class: org.atmosphere.kafka.KafkaBroadcaster init-params: org.atmosphere.kafka.bootstrap.servers: kafka1:9092,kafka2:9092 org.atmosphere.kafka.topic.prefix: "atmosphere." org.atmosphere.kafka.group.id: my-app-node-1
Quarkus (application.properties):
quarkus.atmosphere.init-params.org.atmosphere.cpr.broadcasterClass=org.atmosphere.kafka.KafkaBroadcaster quarkus.atmosphere.init-params.org.atmosphere.kafka.bootstrap.servers=kafka1:9092,kafka2:9092
WAR (web.xml):
<init-param> <param-name>org.atmosphere.cpr.broadcasterClass</param-name> <param-value>org.atmosphere.kafka.KafkaBroadcaster</param-value> </init-param> <init-param> <param-name>org.atmosphere.kafka.bootstrap.servers</param-name> <param-value>kafka1:9092,kafka2:9092</param-value> </init-param>
| Property | Default | Description |
|---|---|---|
org.atmosphere.kafka.bootstrap.servers |
localhost:9092 |
Kafka bootstrap servers (comma-separated) |
org.atmosphere.kafka.topic.prefix |
atmosphere. |
Prefix for Kafka topic names |
org.atmosphere.kafka.group.id |
auto-generated UUID | Consumer group ID — each node must have a unique group ID so all nodes receive all messages |
- Each Broadcaster creates a Kafka topic named
{prefix}{broadcaster-id}(e.g.atmosphere._chat_lobby) - Messages are produced with a
atmosphere-node-idheader for echo prevention - A virtual thread consumer polls the topic and delivers remote messages locally
- Special characters in Broadcaster IDs are sanitized to valid Kafka topic names
Each node must use a unique group.id so every node receives every message. If two nodes share the same group ID, Kafka treats them as competing consumers and load-balances messages between them (which is not what you want for broadcasting).
By default, the group ID is a random UUID per Broadcaster — this is correct for most deployments.
<dependency> <groupId>org.atmosphere</groupId> <artifactId>atmosphere-redis</artifactId> <version>4.0.0</version> </dependency>
The Redis module uses Lettuce (6.5.x) as the Redis client.
Spring Boot (application.yml):
atmosphere: broadcaster-class: org.atmosphere.plugin.redis.RedisBroadcaster init-params: org.atmosphere.redis.url: "redis://redis-host:6379" org.atmosphere.redis.password: "secret"
Quarkus (application.properties):
quarkus.atmosphere.init-params.org.atmosphere.cpr.broadcasterClass=org.atmosphere.plugin.redis.RedisBroadcaster quarkus.atmosphere.init-params.org.atmosphere.redis.url=redis://redis-host:6379
WAR (web.xml):
<init-param> <param-name>org.atmosphere.cpr.broadcasterClass</param-name> <param-value>org.atmosphere.plugin.redis.RedisBroadcaster</param-value> </init-param> <init-param> <param-name>org.atmosphere.redis.url</param-name> <param-value>redis://redis-host:6379</param-value> </init-param>
| Property | Default | Description |
|---|---|---|
org.atmosphere.redis.url |
redis://localhost:6379 |
Redis URI (Lettuce format) |
org.atmosphere.redis.password |
(none) | Redis password (optional) |
- Each Broadcaster subscribes to a Redis pub/sub channel named after the Broadcaster ID
- Messages are published with a
nodeId||payloadenvelope for echo prevention - Two Lettuce connections per Broadcaster: one for publishing, one for subscribing
- Resources are released when the Broadcaster is destroyed
If you prefer to keep the default DefaultBroadcaster and add clustering via a filter instead of replacing the Broadcaster class, use RedisClusterBroadcastFilter:
# Spring Boot atmosphere: init-params: org.atmosphere.cpr.broadcastFilterClasses: org.atmosphere.plugin.redis.RedisClusterBroadcastFilter org.atmosphere.redis.url: "redis://redis-host:6379"
<!-- web.xml --> <init-param> <param-name>org.atmosphere.cpr.broadcastFilterClasses</param-name> <param-value>org.atmosphere.plugin.redis.RedisClusterBroadcastFilter</param-value> </init-param>
This implements the ClusterBroadcastFilter interface and intercepts broadcasts at the filter level. It uses the same Redis pub/sub mechanism and echo prevention as RedisBroadcaster.
| Kafka | Redis | |
|---|---|---|
| Latency | Higher (batch-oriented, poll-based) | Lower (push-based pub/sub) |
| Durability | Messages persisted to disk | Fire-and-forget (pub/sub messages are not persisted) |
| Scalability | Excellent — partitioned, replicated | Good — single-threaded pub/sub per channel |
| Message replay | Yes (new consumers can read history) | No (only receives messages after subscribing) |
| Operational overhead | Higher (Zookeeper/KRaft, brokers, topics) | Lower (single Redis instance or cluster) |
| Best for | High-throughput, message ordering guarantees | Low-latency, simple deployments |
Rule of thumb: Use Redis for typical real-time apps (chat, notifications). Use Kafka if you need message durability, ordering guarantees, or already have Kafka infrastructure.
Clustering is transparent to the Room API. Because Rooms are backed by Broadcasters, replacing DefaultBroadcaster with KafkaBroadcaster or RedisBroadcaster automatically clusters your rooms:
atmosphere: broadcaster-class: org.atmosphere.plugin.redis.RedisBroadcaster init-params: org.atmosphere.redis.url: "redis://redis-host:6379"
@RoomService(path = "/chat/{roomId}", maxHistory = 50) public class ChatRoom { @Message public String onMessage(String message) { return message; // broadcast across all nodes automatically } }
Both clustering solutions work with or without sticky sessions. However, sticky sessions are recommended for WebSocket connections to avoid reconnection storms when a load balancer reassigns a client to a different node.
Configure your load balancer to route by:
- WebSocket: connection-level affinity (most load balancers do this automatically)
- Long-polling: cookie or IP-based session affinity
- Understanding Broadcaster — the pub/sub bus that clustering extends
- Understanding @RoomService — declarative room handlers
- WAR Deployment — container configuration including load balancer setups