Temporal seems to perfectly match some of the use cases my team has and we’re looking forward to trying it in one of our projects. I have some questions I’d like to clarify before we dive in.
Our backend services are in Kotlin and we use Kotlin’s suspend (non-blocking) calls a lot, allowing us to run a lot of IO with a pretty small thread pool. We’re looking for ways to keep it that way when we’ll start using Temporal, especially when it comes to activities.
By default, activities occupy a thread the whole time they run, however, ActivityExecutionContext.doNotCompleteOnReturn() seems to be the way to “non-blocking” activities.
Is there any expected performance penalty for running (relatively short-lived) activities via doNotCompleteOnReturn() vs. regular blocking mode?
Can activity throw an exception after it calls doNotCompleteOnReturn() instead of returning an arbitrary return value?
Am I right that async activity can still heartbeat via ActivityCompletionClient.heartbeat()?
Do activities have some sort of lifecycle one could plug into? In particular, is there a way to plug into the shutdown of a particular activity or activity worker?
For background: I’m looking for a way to build a utility method that could look like this:
class MyActivityImpl(
private val activityCompletionClient: ActivityCompletionClient
) : MyActivity {
@Override
fun method(): String = asyncActivity(activityCompletionClient) {
// calls to non-blocking suspend methods
// at the end of this block activity will be resumed via ActivityCompletionClient
}
}
Another larger question - do you consider having something like Kotin SDK at some future point? One benefit I can see is that Kotlin SDK might probably build workflows as coroutines and suspend instead of blocking a thread, so a single worker might have hundreds of thousands or even millions of suspended workflows still stored in its cache, ready to respond to query or signal, and cache size could only be limited by JVM memory and not the thread pool size.
Is there any expected performance penalty for running (relatively short-lived) activities via doNotCompleteOnReturn() vs. regular blocking mode?
I don’t think there is a performance problem with the implementation itself. The tricky part is that the worker is usually limits the number of parallel activities through WorkerOptions.maxConcurrentActivityExecutionSize. But as soon as the activity method returns the counter is decremented even if doNotCompleteOnReturn() was called. So there is no enforcement of the limit for parallelism. It is being fixed as part of this issue.
Can activity throw an exception after it calls doNotCompleteOnReturn() instead of returning an arbitrary return value?
If an activity throws an exception from its method then it is going to be considered failed. If you want to report activity failure asynchronously use ActivityCompletionClient.completeExceptionally.
Am I right that async activity can still heartbeat via ActivityCompletionClient.heartbeat() ?
Do activities have some sort of lifecycle one could plug into? In particular, is there a way to plug into the shutdown of a particular activity or activity worker?
In both synchronous and asynchronous implementations, the only way to learn about activity cancellation (or any other reasons for an activity to stop execution) is through heartbeating. If the heartbeat call throws an ActivityCompletionException then the activity is expected to stop its execion. I don’t think the worker shutdown is plugged into this mechanism yet.
Another larger question - do you consider having something like Kotin SDK at some future point? One benefit I can see is that Kotlin SDK might probably build workflows as coroutines and suspend instead of blocking a thread, so a single worker might have hundreds of thousands or even millions of suspended workflows still stored in its cache, ready to respond to query or signal, and cache size could only be limited by JVM memory and not the thread pool size.
We absolutely want to add Kotlin support for exactly the same reason you mentioned. Coroutines would allow caching a much larger number of workflows. We don’t have immediate plans to work on this due to resource constraints. If anyone is willing to contribute the Kotlin bindings I would work with them closely to ensure the correctness of the integration.
Good to know. We could manually manage the concurrency level of these background coroutines until that issue is resolved.
Just to clarify: I’m looking for a way to avoid returning an “unused” value from an activity method after I call doNotCompleteOnReturn(). That unused value is easy to create in every particular case, but more tricky if we want to have a generic return asyncActivity { ... }. Maybe there is some marker exception that won’t indicate an activity failure and could be used instead of returning an unused result?
Just to clarify: I’m looking for a way to avoid returning an “unused” value from an activity method after I call doNotCompleteOnReturn() .
I think I don’t understand the problem you are trying to solve. Why always returning some unused value doesn’t work? You are always calling doNotCompleteOnReturn.
I might be missing something obvious. When the activity code calls doNotCompleteOnReturn() it’s not immediately terminated and still has to return something, a value that won’t be directly used and will later be replaced with whatever is passed to complete()/completeExceptionally(). So the activity code looks like
fun activityMethod(): Int {
asyncActivity(activityCompletionClient) {
// this helper method calls doNotCompleteOnReturn(), schedules a coroutine, and
// calls `complete()`/`completeExceptionally()` once that coroutine completes.
}
return 0 // the "unused" value, that is specific to the return type of the method
}
I wish this could look like
fun activityMethod(): Int {
return asyncActivity(activityCompletionClient) {
// this helper method calls doNotCompleteOnReturn(), schedules a coroutine, and
// calls `complete()`/`completeExceptionally()` once that coroutine completes.
}
}
but for that to happen asyncActivity should return something that would work with any return type of the activityMethod().
I see. Do I understand correctly is that you cannot write a generic activity implementation as it requires an activity-specific return type for the return?
I’m almost done with a new change that would support dynamic activity implementation that simplifies writing generic workflow code.
The DynamicActivity implementation can implement activity of any type with a single piece of code.