Testing async workflows in microservices is a challenge—traditional staging setups are costly and unreliable. The message isolation pattern offers a scalable solution, enabling reliable testing without duplicating infrastructure. This article breaks down how it applies across Kafka, RabbitMQ, AWS SQS, Google Pub/Sub, and NATS.
Originally posted on The New Stack, by Arjun Iyer.
The message isolation pattern works across most popular message brokers, while implementation details vary based on architecture and terminology.
When I talk to engineering leaders about testing event-driven microservices, I hear a common pain point: “Our async flows are nearly impossible to test reliably.”
Previously, we explored how OpenTelemetry and message isolation enable testing Kafka-based workflows without duplicating infrastructure. The approach resonated with many teams facing the challenge of testing asynchronous flows cost-effectively, but I’ve received numerous questions about extending this pattern to other message brokers.
The short answer is yes. The message isolation pattern works across most popular message brokers but implementation details vary based on each system’s architecture and terminology.
Before diving into specific platforms, let’s recap the core problem and our approach:
Testing microservices that communicate via message queues presents several challenges:
Our message isolation approach addresses these challenges through these key principles:
The goal is straightforward: Enable multiple developers to test their changes in isolation without interference while avoiding the cost and complexity of duplicating your entire messaging infrastructure.
Kafka remains our reference implementation for message isolation testing due to its robust consumer group model. In Kafka, multiple consumer groups can independently process the same messages from a topic, creating a natural pattern for our testing approach.
With Kafka, each producer includes a routing key in message headers, and consumer groups provide the isolation mechanism. When a developer deploys their service version to test, it joins a unique consumer group (such as my-service-group-sandbox123), receiving a copy of all messages while the baseline version continues operating in its original consumer group.
RabbitMQ’s architecture differs from Kafka because it uses exchanges and bindings as intermediaries between producers and consumers, allowing more flexible routing patterns.
In RabbitMQ, exchanges distribute messages to queues through bindings. The message isolation pattern works by creating temporary queues for sandbox services that are bound to the same exchanges as the baseline queues. For example, if your baseline service consumes from a queue bound to a topic exchange, your sandbox version would create a new temporary queue bound to the same exchange with matching routing keys.
While RabbitMQ offers broker-level filtering through binding patterns, we still need consumer-side filtering to check the sandbox routing key against a central mapping service. The consumer would extract the routing key from the message headers, call a mapping service to determine if it should process the message and then proceed accordingly.
Google Cloud Pub/Sub organizes messages around topics and subscriptions, making message isolation straightforward. A topic can have multiple subscriptions, and each subscription receives a copy of every message published to the topic – aligning perfectly with our sandbox model.
For sandbox testing, create temporary subscriptions to the same topics your baseline services use. The subscriber would implement consumer-side filtering to extract the routing key from message attributes, check against the mapping service and process messages accordingly.
AWS SQS presents a unique challenge for message isolation because it doesn’t natively support the fan-out pattern we rely on in other brokers. In SQS, messages are consumed by a single consumer; there’s no built-in mechanism for multiple consumers to receive the same message, and messages are automatically deleted after being processed.
Therefore, we need to consider alternate approaches:
Use Amazon SNS (Simple Notification Service) as the entry point to fan out messages to multiple SQS queues. Publishers send messages to an SNS topic, which delivers copies to multiple subscribed SQS queues — one for baseline, and separate ones for each sandbox environment. This approach requires no changes to existing consumers as they continue to read from their dedicated queues.
Create temporary SQS queues for testing with sandboxed producers and consumers. Deploy a new instance of your producer that sends messages to the temporary test queue and configure your test consumer to read from this new queue. Each sandbox gets its own isolated queue, producer and consumer.
For most teams, Option 1 provides the cleanest solution. Setting up an SNS topic that publishes to both your baseline SQS queue and any temporary testing queues maintains the decoupled architecture while enabling the isolation needed for testing. The overhead of message duplication is typically negligible compared to the cost of duplicating your entire infrastructure.
NATS, with its lightweight and high-performance design, offers features well-suited for sandbox testing. We can leverage NATS queue groups, which function similarly to Kafka consumer groups.
By creating sandbox-specific queue groups, test services receive copies of messages without interfering with baseline services. Your test service would join a unique queue group (such as service-sandbox123), subscribe to the same subjects as your baseline service and implement consumer-side filtering based on routing keys.
The message isolation approach becomes much more maintainable when platform teams build reusable components that handle the sandbox-specific logic.
Platform teams can create thin wrappers around standard client libraries that automatically handle sandbox filtering:
Application developers use this exactly like the standard consumer:
This approach makes the sandbox filtering completely transparent to application teams.
Platform teams can provide standardized hooks around the life cycle of sandboxes that automate resource management:
When a developer creates a new sandbox for testing:
(service-{sandbox-id}).This hook-based approach makes sandbox isolation part of the platform’s core capabilities rather than a burden on each development team.
Message isolation patterns work across all major message brokers, though implementation details vary based on each system’s architecture. The approach consistently delivers three key benefits:
At Signadot, we’ve implemented these patterns across dozens of customer environments using different messaging systems. Companies like Brex, Earnest and ShareChat have used these approaches to transform their microservice testing. If you’re interested in exploring how to apply these concepts to your specific environment, sign up and join our community Slack channel.
As microservice architectures continue to evolve, effective testing of asynchronous workflows becomes increasingly critical. The message isolation pattern provides a practical, scalable approach regardless of which message broker you’ve chosen.
Get the latest updates from Signadot
