最新消息:雨落星辰是一个专注网站SEO优化、网站SEO诊断、搜索引擎研究、网络营销推广、网站策划运营及站长类的自媒体原创博客

How to guarantee message ordering while maintaining high throughput in RabbitMQ - Stack Overflow

programmeradmin3浏览0评论

I have 20000 devices which send the messages that I push to RabbitMQ, on the other side I have 1 or more worker services that process the message and reply to a custom response queue on RabbitMQ. I need to ensure strict message ordering in RabbitMQ while also achieving high throughput.

I've searched the web and these are the options that I came up with:

  1. Single queue, single consumer
    Guarantees order but limits throughput since only one consumer can process messages at a time.
  2. Single queue, single consumer with ConcurrentMessageLimit > 1
    Risks breaking order due to concurrent processing within the consumer.
  3. Queue per device, single consumer per queue
    Could work, but would need to create a lot of queues. Get's complicated if user adds a new device or removes one.
  4. RabbitMQ Super Streams with Single Active Consumer
    This option looks promising, however in the end it really looks like option 3 with added throughput benefits. Or am I missing something? As far as I understand I still need to have 20000 streams and handle the cases when device is added or remove to do the cleanup of individual streams.
    For example, if I have 3 queues as in the official example this would still mean that I have 3 active consumers but with concurrency limit = 1. Which means I can achieve high throughput by adding as many streams as necessary to suffice for my workload. Did I get it right? What if workload increases - does this mean I need to be able to add streams dynamically, is this even supported by super streams?
发布评论

评论列表(0)

  1. 暂无评论