Numerous Infinite-Loop Tasks Per Worker

Community Support
1

Hi everyone!

I am writing a monitoring program in Go and have been looking into numerous orchestration design patterns and stumbled upon Temporal. I am looking to determine if Temporal is a good approach for my program and would love some high-level guidance on roughly how to implement the Temporal portion of it before deciding to dive in full-force.

For the sake of simplicity, I will only refer to HTTP status monitoring for this post, but there are numerous types of monitoring I have in place, all defined within the same table in postgres using an HSTORE for the specific parameters.

My application is intended to be as follows:

  • Define the parameters for a Monitor (e.g. url, expected status code, timeout, etc) in a PSQL database table
  • Run each monitor exactly once regardless of how many instances of my program are running
  • Each monitor is extremely lightweight and should run in a single goroutine,
  • Monitors are infinite loops. They should run continuously until the go program terminates OR it is requested for it to be cancelled (which can happen at any time). Each monitoring action (e.g. HTTP request) only occurs once per minute.
  • Upon a status change, the database is updated and a NOTIFY trigger is called in the database
  • Many hundreds or even thousands of monitors should be able to run on any given instance of my program running

If I do a standalone application, all of the pieces work absolutely fine, but the issue arises with scalability. I need to be able to run multiple instances of my program for high availability so that if anything goes down, other instances will take over the monitoring responsibilities.

TL;DR:
How would I best approach running thousands of concurrent, independent, unitary, lightweight, infinitely-looping, and cancel-able goroutines using Temporal?

Thank you for any information or recommendations!

2

Polling for an external resource is a pretty common use of Temporal. There are a few ways to implement it in Temporal depending on the frequency and other requirements.

1 Like
3

Thank you! To be clear, any polling would actually be done in Golang where I have implemented the various monitors (HTTP status, HTML keyword, SSL cert expiry, ICMP status, TCP port check, etc). Temporal (or whatever solution I end up using) should be responsible only for delivering the task to workers, ensuring there is only one instance of the task running, canceling the task, and coordinating between multiple workers (i.e. one worker goes down, all others should take over the monitors that worker was originally performing).

I agree this is a relatively standard use case for orchestration, but I worry that Temporal is overkill considering how little ultimately I want it to do.

4

I don’t think there is any contradiction here. You implement an activity in Go that does all the business logic you outlined. Then you use one of the patterns I linked above to execute that activity.