I’m evaluating Temporal for a project, and wanted to clarify a couple of things, and get your opinion if it’s a good use case for Temporal.
Am I understanding this correctly:
I can create a number of Worker services, and each of those Worker services will only run Activities and Workflows that I will register for them.
So say, that I want to allocate part of my k8s cluster nodes to execute only specific types of tasks, I can just register a single workflow and activity within those specific workers?
It is also mentioned that:
If the worker receives tasks for a Workflow type it does not know, it will fail that task.
I assume, that I control if a specific Workflow-task will be delivered to a specific Worker-service using Task Queue names?
So I can avoid the situation of auto-failing tasks, just by using correct task queues?
Now to the use cases:
Video/image processing - a media file is uploaded to a server, and it needs to be processed - converted to a different format, sliced, down/up-scaled and so on. So, an operation that can be very long running. There is a limit to the concurency, due to limit of the availiable hardrware, so if all the processing nodes are busy, new processing tasks should just queue up.
It’s also mentioned that:
A Worker polls for a message only when it has spare capacity, so it never gets overloaded.
Can this be controlled somehow? Say I want to limit a worker to execute only one task at the same time.
- Async fan-out - a way to propagate, when one event results in thousands of insert/delete queries to the DB. Especially popular within the Cassandra community.
Do you think Temporal is a good fit for these two?