Separate Workers for Workflows and Activities

Community Support
, , , ,
1

Unsure if this is the right place to ask, but I’m working in a monorepo and have some questions about how temporal worker’s handle workflows and activities. Idea is that the directory structure looks something like this:

My understanding is that I need to spin up workers in each of the services, as well as workers in my temporalWorkflows service. The worker in the temporalWorkflows service will be the one executing the workflows, while the workers in each of the services will be the ones executing the activities. Basically, I’d like to guarantee that the activities are executed within the service that they’re from. I plan on configuring these by registering each activity to its own taskQueue, and the workflow to another taskQueue. I’ll also then have a worker for each task queue. Is there a simpler or alternative way of doing this that I’m not seeing, or is this the way to work with temporalWorkflows that use activities across services?

monorepo/
– volumeService/
---- volumeActivity.ts
– perimeterService/
---- perimeterActivity.ts
– temporalWorkflows/
---- workflow.ts

2

Generally, the simplest thing to do is having a single worker that can process all workflows and activities on a single task queue.

If your goal is to have separate deployment & monitoring processes for different teams, you could call each team’s worker a “service”, and that worker would have that team’s workflows and/or activities and would poll on that team’s queue.

Then the coordination between teams is knowing what their workflows’/activities’ names/signatures are and what queues they’re on.

3

There are reasons besides separate team deployments for adding task queues, for instance:

  • When you have Activities that need to run on specialized hardware, for instance with GPUs, they can go on a task queue named gpu-activities.
  • When you need to limit the concurrency of an Activity in order to not overwhelm or exceed the rate limit of a downstream service or API the Activity calls, you can put that Activity on its own task queue and set maxTaskQueueActivitiesPerSecond