I am building a new product and need to have a robust queue system but I'm not sure what product to choose.
I've worked in Amazon before and usually AWS tools are the default go-to but in my own time for small project I've used things like: https://github.com/hibiken/asynq for basic tasks.
My concern with the above is that it's still in "early" development; from their README:
``` Status: The library relatively stable and is currently undergoing moderate development with less frequent breaking API changes.
Important Note: Current major version is zero (v0.x.x) to accommodate rapid development and fast iteration while getting early feedback from users (feedback on APIs are appreciated!). The public API could change without a major version update before v1.0.0 release. ```
I'm building something right now that will require something very robust; scheduling, long running workflows with several external dependencies (which may have failure and need retries and notifications, webhooks, etc)
Ideally I want to use the same tool for everything; so that would also include basic things like fan-out (like user singed-up: fan out to email service to queue welcome sequence, notify another service, make api request to another dependency, etc.)
There are so many too choose from: NATS, RabbitMQ, Asynq, AWS tools, Google Pub/Sub, etc.
There are so many options and I'm always very excited to explore tools but I need to pick one and us that across entire project for everything; since it won't be just me and having one interface/tool to learn reduces cognitive load.
What do you use normally and why?
(I use Go - if it matters)