Attach listeners to task queues

Community Support
1

Hello everyone!

I’m curious if there’s a way to connect listeners to task queues in Temporal. In the old workflow orchestration system I used, a pub-sub model was used for the task queues equivalent. So when an activity task was added to the queue, multiple subscribers received the message, but only the first to respond was considered; the rest were ignored. I want to do something similar with Temporal: have multiple read-only subscribers, that do not actually dequeue the tasks from the Task Queue, but are just notified about them.

Constantly polling the API for changes isn’t feasible for us.

I’m new to Temporal, so I might be missing something that does this. Interceptors do not seem to be the solution: I don’t want to have a chain sequence of interceptors, I want the no-op listener and the real worker to get the message at the same time.

Thanks in advance to whoever can help me resolve the use case!

2

What is the actual problem you are trying to solve? Temporal activities use task queues that have the semantic you want.

3

Hi @maxim! Thanks for the prompt answer.

I need to have N workers listening to the same task queue: all of them need to read the tasks being pushed, but only one of them is capable of processing the task, the other N - 1 should only be listeners.

Using the Java SDK + Spring integration, for example:

First worker:

@Component
@ActivityImpl(taskQueues = "BookTripQueue")
public class TripActivitiesImpl implements TripActivities {

    @Override
    public String bookHotel(Hotel hotel) {
        // Actually book the hotel
    }

    @Override
    public String bookCar(Car car) {
        // Actually book the car
    }
}

The second worker needs to only listen to the tasks as they are progressed, but shouldn’t work on them:

@Component
@ActivityImpl(taskQueues = "BookTripQueue")
public class TripActivitiesImplListener implements TripActivities {

    @Override
    public String bookHotel(Hotel hotel) {
        // Do something else with this information. Shouldn't actually return anything.
    }

    @Override
    public String bookCar(Car car) {
        // Do something else with this information. Shouldn't actually return anything.
    }
}

If I do this, won’t only one worker dequeue the task and run it? If the first worker does, then the second one won’t ever notice. If the second worker does, then the task won’t be completed correctly.

So my question is: how can I have a no-op worker that only listens for tasks as they are progressed in a workflow?

4

I see. Currently, Temporal does not directly support such functionality.

As a workaround, you can create a task queue per worker and invoke activity for each task queue (possibly using an interceptor). However, you would need a separate mechanism to learn about the list of workers.