How to run serverless workers that scale to 0

I’m investigating whether we can leverage Temporal Cloud for our workflows or not, and I’m running into one major issue: It seems the only integration path with the Temporal cluster, whether it be self hosted or Temporal Cloud, is to have some always on compute running worker tasks that are doing a long poll of the task queues held by the Temporal cluster. I’m not seeing any way to have the Temporal cluster call out to some endpoint, or send out some event, to indicate it has work that needs to be performed by a worker.

Our current architecture is serverless on AWS, using things such as AppSync, Event Bridge, Step Functions, Lambda, and Batch among others. Currently, all of our compute is on demand, so it scales to 0 when not in use. For example, a GraphQL Mutation kicking off a Step Function to drive some kind of asynchronous workflow, or a webhook event from an external SaaS product coming into Event Bridge and being routed to a Lambda to be handled. Scheduled tasks currently use Event Bridge Scheduler to kick things off.

Ideally I would like to have the Temporal Cloud tell me when work needs to be performed, via something like an event, so I can run workers in response to that. I see there are some metrics used for scaling workers as an option also, but I don’t see a way to get those metrics out of Temporal without also having some always on compute polling those metrics.

I saw this old topic about using Lambda as an activity worker, and potentially natively supporting AWS Lambda at some point, but I can’t find any information about that elsewhere: Is it possible to use AWS lambda as activity worker?

Is the integration pattern I’m seeking possible with Temporal Cloud?

Server release 1.25.0 added features for task queue statistics via DescribeTaskQueue api. More info on that in docs here. Let us know if you have specific questions on that.

I found that info, but it seems like I would need something to poll the metrics endpoint. It doesn’t appear that there’s a way to get Temporal Cloud to send the metrics out to something like AWS CloudWatch so I could then use those metrics to autoscale on ECS or something similar. Unless I’m missing something?

Yes, currently it would be api you’d need to poll.

Also, are there any plans to natively support AWS Lambda as a worker?

Yes, we are considering this, but not in the immediate future.

What’s the best place to monitor for an announcement about this being implemented?