I need to create an SQS consumer, and I’m considering the options of using a go worker or a long-running Temporal workflow to receive messages from an SQS queue. The messages in the queue are S3 event notifications, such as when a file is uploaded to S3.
In both cases, handling the message means starting another workflow that processes the file that was uploaded to S3. Thus, the SQS consumer is intended to be simple.
It seems feasible to build a long-running workflow that polls and handles messages from SQS indefinitely, but is that recommended? Temporal workflows seem to be best suited for orchestrating business tasks, instead of indefinite polling.
You don’t need a workflow to receive messages from SQS if you have just one queue. In some cases the queue subscriptions are dynamic based on some business logic. In this case adding/removing subscriptions through a special control workflow does make sense.
Unless I’m missing something, I think what you want is to attach an AWS Lambda function to your queue, and then the Lambda function would use the Temporal client API to send a signal to a workflow, or start a workflow, etc., to respond to a queue message.