Max number of jobs a worker can take from a queue

Community Support
,
1

Hello.

I want to limit the number of jobs each worker can take from a queue, so it doesn’t get overloaded by parallel heavy activities. I’m planning horizontally scale the number of workers if I need, but each one should run a maximum of 2 jobs(workflows) per worker, meanwhile, the queue grows if I don’t have enough workers.

Basically, it’s something like a basic FIFO queue.

I found a bunch of options like MaxConcurrentActivityExecutionSize - but all of them get an Activity as a unit, so what is the best strategy to use these parameters for jobs(workflow that includes all activities)?

Thanks in advance!

2

Use the Session feature.

3

Thanks, @maxim. Actually, Session was already employed, I just needed to specify:
MaxConcurrentSessionExecutionSize: 1

I have a problem that many jobs fail because of this:

13 2023-04-26 UTC 13:57:26.43 ActivityTaskTimedOut Failure Message activity ScheduleToStart timeout

12 2023-04-26 UTC 13:56:26.43 ActivityTaskScheduled Activity Type internalSessionCreationActivity

Which entity processes this internal activity? Could it happen because I use Temporalight? (though throughput extremely low)

Thanks in advance.

4

Are you setting SessionOptions.CreationTimeout?

5

Yes, I do, because the comment says:

// CreationTimeout: required, no default
//     Specifies how long session creation can take before returning an error

It’s not really clear what “session creation” means, and what is a normal timeout

6

This timeout defines how long a session should wait before timing out if no worker is available. I believe that you are hitting this timeout. Try increasing it.