Queue service

We decided we will be using RabbitMQ https://www.rabbitmq.com/ an open source MIT licensed widely used messaging queue solution that can easily be plugged into the .net core solution.
RabbitMQ is highly scalable.
A queue is file based and will require allocated storage in the solution.

To ensure we do not overload the database with requests when the queues run, we’ve decided the following.

• Queuing messages will happen instantaneously in the API container.

• Dequeuing messages will happen in a different container

  • In intervals ex. every 30 seconds and batched together when writing data to the database.

• Format of the items names could be something like:

  • Internal.<cvr>.task

  • Internal.<cvr>.updateCPR

  • External.<cvr>.task

• Configuration will be in the application settings and will be used in all queues.
  They would contain

  • Dequeue interval in seconds

  • Retry time in seconds

  • Number of Retry before giving up.

  • Queue location root

• There will be a basic queue service to use in the API

• There will be a queue for each unique type of functionality we need to handle in a separate dequeuing solution in a different container.

Below is shown an image of how the architecture would look and how queue messages would flow.

Open