最新消息:雨落星辰是一个专注网站SEO优化、网站SEO诊断、搜索引擎研究、网络营销推广、网站策划运营及站长类的自媒体原创博客

Transform and distribute messages from Azure Event hub to multiple event hubs - Stack Overflow

programmeradmin1浏览0评论

following scenario.

  • One event hub with incoming messages. (ingress event hub)
  • Application reads, transforms and duplicate the messages.
  • Afterwards publishing the messages to multiple other event hubs. (egress event hubs)

So one message from ingress event hub could be published to one or n egress event hubs.

Problem
We do the publishing in parallel, but at the end we have to wait, until all egress event hubs are done. That's because of the checkpointing for the ingress event hub, where we have only one checkpoint.
So, if one egress event hub have some problems (throttling, network issues,...), all other egress event hubs are also affected.

Are there any ideas to handle that?*

And btw. we have around 500 million ingress messages per day.
So things like event grid or a consumer group on ingress for each egress event hub will be very expensive. So I'm interested in a more "cheaper" variant.

Thx a lot!

发布评论

评论列表(0)

  1. 暂无评论