Kafka

Kafka supports both message-level routing and topic-based isolation. The recommended approach is to use message headers to propagate a routing key (e.g., the sandbox or RouteGroup key).

Consumer Group Strategy

• Sandboxed consumers should create new, unique ConsumerGroups. This ensures that they receive a copy of all messages received by the baseline consumers, rather than competing for messages within the same group.
• Each sandboxed consumer processes only messages with routing keys that match its own routing key or the routing key of any RouteGroups it is part of.
• All other messages are processed by the baseline consumers (in the baseline ConsumerGroup).

Alternatively, use ResourcePlugins to provision sandbox-specific topics and configure the publishers and consumers to publish and consume from the new/temporary topics.

Resources

• Step-by-step Kafka tutorial
• Working demo: Selective consumption with Kafka

Summary

• Use message headers for routing key propagation.
• Sandboxed consumers use unique ConsumerGroups to receive all messages.
• Consumers filter messages by routing key.
• Optionally, create sandbox-specific topics for full isolation.