-
Slow Processing and Duplicates: You have a message that takes a long time to process. Simply increasing the visibility timeout isn't always feasible. If processing exceeds the timeout, the broker will re-deliver the message, causing your system to process the same logic twice.
-
Poison Pills in Streaming: When consuming from a stream like Kafka, a single failing message (a "poison pill") can block the entire partition. You cannot progress to the next message until the failing one is resolved, halting your application.
How can we ensure idempotent and resilient message processing in these scenarios? The answer is the Inbox Pattern.
The Solution: The Inbox Pattern
The Inbox Pattern provides a mechanism for idempotent message consumption. The core idea is to treat your database as the primary, reliable log of incoming messages.
Here's how it works:
- Upon receiving a message, the first action is to store it in a dedicated inbox table within your database (e.g., Postgres, MySQL) within the same database transaction that handles any initial business data.
- The record includes a unique identifier for the message (like a
message_id).
- Before processing a message, the system checks the inbox table to see if a message with that ID has already been processed (or is being processed).
- If it's a duplicate, the message is acknowledged and ignored.
This elegantly solves our problems:
-
For Problem #1 (Duplicates): The inbox acts as an idempotency filter. Even if the same message is delivered multiple times, it is processed only once.
-
For Problem #2 (Poison Pills): For a streaming system, you can acknowledge a problematic message to unblock the queue. Later, when the issue is fixed, you can replay the reset the stream/offset. The inbox pattern will automatically skip any messages that were successfully processed before the failure.
Trade-offs & Considerations
No pattern is a silver bullet. The Inbox Pattern introduces its own trade-offs:
-
Performance Overhead: Every message involves a database write and a check, which adds latency.
-
Complexity: It adds architectural complexity, requiring you to manage an inbox table and background processors.
-
Database as a Bottleneck: The database can become a bottleneck under very high message throughput.
-
Storage overhead: Inbox tables can grow large, requiring a implement of retention policies or archival.
Tooling & Implementation
The good news is that the Inbox Pattern is widely supported. You can likely find a framework for your programming language that implements it. If not, the core logic is straightforward to build yourself.
For .NET developers, the Brighter project provides excellent, built-in support for both the Inbox and Outbox patterns.
Conclusion
The Inbox Pattern is a powerful tool for building resilient, event-driven systems. By using a database as a protective layer, it ensures idempotent processing and helps handle failures gracefully. While it introduces some complexity, the benefit of guaranteed, duplicate-free processing is invaluable for critical business workflows. When you embrace the Outbox Pattern for reliable sending, remember to pair it with the Inbox Pattern for reliable consumption.
Reference
https://event-driven.io/en/outbox_inbox_patterns_and_delivery_guarantees_explained/
https://newsletter.systemdesignclassroom.com/p/every-outbox-needs-an-inbox