Temporal as a generic purpose load testing framework

Do you think Temporal would be a good fit for a distributed load testing tool?

There are tons of existing load testing tools out there, but they are mostly HTTP only and cannot really support more complex use cases (eg. upload a file to an S3 compatible API with multi-part upload). For those use cases a generic, distributed task execution framework would be a better fit…which Temporal kinda is.

A question that immediately comes to mind is whether the “latency” that Temporal puts on the execution of workflows/tasks would not defeat the whole purpose of performance testing.

Another interesting question is whether Temporal can handle the load of the resulting data or it should be stored outside.

1 Like

We actually use Temporal workflows to perform load testing of the Temporal service itself. There are also proprietary implementations of frameworks similar to chaosmonkey that use Temporal to execute failure testing scenarios.

So I think Temporal is a perfect technology for this use case.

A question that immediately comes to mind is whether the “latency” that Temporal puts on the execution of workflows/tasks would not defeat the whole purpose of performance testing.

All the frameworks I know execute the heavy load generation from long running activities. For example, you can have an activity that in a loop performs S3 upload a preconfigured number of times. Then the workflow ensures that a configured number of such activities is executed and reports completion when done. In such usage latency of the Temporal itself doesn’t really come into play.

Another interesting question is whether Temporal can handle the load of the resulting data or it should be stored outside.

I think all the metrics and other artifacts produced by load tests should live outside and only data related to the scenario control plane should float through Temporal workflows.

I envision a system that has some sort of DSL that describes a test scenario. Then the workflow executes the scenario based on the configuration.

1 Like

All the frameworks I know execute the heavy load generation from long running activities.

I think all the metrics and other artifacts produced by load tests should live outside and only data related to the scenario control plane should float through Temporal workflows.

Makes sense. That’s what I thought as well.

I have a couple load testing scenarios for which I will try Temporal. Personally, I really like JMeter and its concept of a test plan, so I will try to come up with something similar.

One thing that would be interesting to have though is a single binary solution for the whole thing. In many cases, a distributed load testing system is the way to go, but for smaller use cases, a single executable binary is better. Having support for both would be a killer feature.

Do you think Temporal can be packaged this way? I know it can run the services in a single binary, but in this case, I guess it would have to run inside some other binary, probably communicating through internal streams emulating network connections, and storing state in something like BoltDB. Sounds like a lot of work at first, but apart from the extra persistence layer, I think it should be doable and would be a killer feature for such load testing tool.

Yes, there are many scenarios when a single binary service is needed.

For example:

  • Unit testing framework. SDKs in all languages could use a single embedded version of Temporal service for unit testing.
  • Small websites. Imagine PHP SDK + single binary service deployed as part of every WordPress website.
  • A single machine workflows. It can be used to execute various tasks instead of cron for example.
  • Desktop applications.
  • Your load testing example.

So we absolutely going to create it. No ETA yet.

1 Like

Datadog have released single binary which can be found here GitHub - DataDog/temporalite: An experimental distribution of Temporal that runs as a single process

2 Likes