Clojure SDK and async workflow

Community Support
1

I am working on a Clojure SDK for Temporal based on the Java SDK and while a straightforward mapping is quite easy it might be suboptimal as Clojure has core.async. With core.async it is possible to do async programming based on channels and threads are parked when a channel is blocked. This allows for caching a bigger amount of workflows as caching of a workflow is independent from the thread.

I am working on a proposal (for GitHub - temporalio/proposals: Temporal proposals) but I am not totally sure how to test if my ideas will work. My ideas:

  • Activity invocation return a channel that can be waited on
  • Timeouts return a channel that can be waited on
  • Whenever a thread is parked I save Java SDK thread locals and restore on resume, or instead of that I try to not have thread locals at all, by having some kind of context object (as in go-sdk)

Do these ideas make sense? If so, it might be that a Kotlin SDK and a Clojure SDK have some overlap as they could both use channels and do some saving and restoration of thread locals if needed.

2

Your ideas are absolutely spot on. We don’t want to use synchronous code that blocks threads in languages that support async natively. These languages should bind to lower level fully async API that SDK provides.

It should implement ReplayWorkflow directly. Note that ReplayWorkflowContext provides a fully asynchronous version of all APIs. For example here is the activity invocation API:

Functions.Proc1<Exception> scheduleActivityTask(
      ExecuteActivityParameters parameters, Functions.Proc2<Optional<Payloads>, Failure> callback);

It delivers a result asynchronously through the callback argument. And it can be canceled through the procedure returned.

We might need to do some refactoring to make hooking into ReplayWorkflow by different runtimes easy. But it is a pretty straightforward task.

I am working on a proposal (for GitHub - temporalio/proposals: Temporal proposals)

Great! As soon as the proposal lands I can work with you closely to ensure that implementation uses SDK in the most efficient way possible. There are quite a few non trivial gotchas, so we have to work closely together to guaranteed the quality of the implementation.

3

@erwin , I’m interested in a Temporal Clojure SDK. Have you by chance made any progress?

4

Currently I am using a couple for wrappers that create the annotations etcetera that the Java SDK expects. I have some ideas to use core.async, but as I didn’t need the performance I did not work on that at all.

1 Like
5

@gdw2 I have released a Clojure temporal library that may be of interest to you:

2 Likes