How do I make sure that there are no duplicate queue messages when they have a common correlation ID? How do I ignore the publishing of a queue message when there is already one on the queue with the same correlation ID?
This is needed to keep the queues clean. For example: Amazon SQS has a Message deduplication ID
Does MassTransit also have this functionality when using a postgres database?
I tried creating messages with the same correlation ID but duplicate ones are still added to the queue.
How do I make sure that there are no duplicate queue messages when they have a common correlation ID? How do I ignore the publishing of a queue message when there is already one on the queue with the same correlation ID?
This is needed to keep the queues clean. For example: Amazon SQS has a Message deduplication ID
Does MassTransit also have this functionality when using a postgres database?
I tried creating messages with the same correlation ID but duplicate ones are still added to the queue.
Share Improve this question asked Jan 22 at 15:08 BrianBrian 1511 silver badge13 bronze badges 1- I updated the answer to explain why. And how to deal with it properly. – Chris Patterson Commented Jan 25 at 14:38
1 Answer
Reset to default 2There is no deduplication ID in the SQL transport.
A message store should not be used as a database. Relying on the transport to deduplicate data is a bad idea in general. You should have an idempotent domain that can deal with duplicates.
Create a saga with a unique property (your de-duplication value) and ensure that any duplicate messages are properly handled. The saga repository (EF Core? it's a database table) then becomes your deduplicated data store – not the queue.