DSL workflow design

HI Team,

I’m looking for your suggestion on DSL workflow design. My challenge is how to pass data into a workflow. Considering there two cases,

  1. Manual activity which will wait for a user input.
    It’s not ideal to poll from activity.

  2. aws sqs message listener activity which will listen on a queue, filter message if condition matched then continue to next activity.
    Creating a listener in every running workflow to poll message will not work. It makes more sense to have a centralized listener to filter and dispatch message to corresponding workflow instance. Then this activity will basically just wait for a message trigger and consume the message

Since DSL is dynamic and signal is only on workflow level not inside activity, it’s difficult to define signal logic in workflow and send the signal. Can you please advise, thanks in advance.

Thanks
Gaoxin

Signal is the way to go in this case. No need to use an activity or queue. Just signal a workflow when user completes the action.

Since DSL is dynamic and signal is only on workflow level not inside activity, it’s difficult to define signal logic in workflow and send the signal. Can you please advise, thanks in advance.

I don’t understand your concern. You can always include some correlation id into the signal data and lookup a specific node in your DSL using it.

Thank you Maxim for your quick response. The DSL is generated flowchart by frontend drag & drop UI.Each activity is a draggable node in the UI. So currently manual step or others steps which need input data have to be activities for user to construct workflow and then send to temporal to execute. Also, as far as I understand, every workflow have different ID. I will need to filter all running workflow, find out the dsl and check if next activity needs signal and then send signal. Is there a better way to do this. Thank you very much.

Each activity is a draggable node in the UI. So currently manual step or other steps which need input data have to be activities for the user to construct workflow and then send to temporal to execute.

There is no need to map UI abstractions to Temporal abstractions directly. A single UI element can be implemented with any amount of code in Temporal. For example, a DSL step like “deploy my service to production” can be implemented with a pretty complex child workflow that talks to dozens of services.

I’m pretty sure that your users do not think in terms of Temporal activities. They think about business-level actions that should be performed.

So a manual step which is a single DSL workflow element can be mapped to an activity that initiates this step and a signal handling logic that completes the step. For implementation simplicity, you can always create higher-level coding abstractions (like OO classes) to encapsulate complex behaviors.

1 Like

Why currently UI element is mapped to activities is when user execute a workflow, they can see state, input/output of each element. It’s straightforward to call GetWorkflowHistory to get activity detailed information and display real time data on UI.

And in our user cases, we provide fundamental functionalities as activities mapped to UI element node, different team can drag & drop element to create their own workflow based on their business requirement.

But you are right, there is no need to map UI abstractions to Temporal abstractions directly.Backend can handle the logic for two way bindings, how to execute and feed data back to the frontend.

Will serverless workflow DSL be supported by temporalio in future (not just the dsl sample). This will allow to write the DSL in yaml/json.

Long-term plan is to make it easier to support any type of DSL on top of Temporal. As far as serverless workflow DSL, definitely planning to provide support for the entire DSL in the future. There is already some ongoing work in the serverless workflow community to provide full DSL support for Temporal. Idea is to then contribute it to the Temporal community, so stay tuned.

@tihomir W.r.t DSL support, you mean that the interpretor logic required for serverless workflow execution to run with Temporal will be part of future support?

Yes, current plan is to provide support for serverless workflow dsl exec on Temporal (Java SDK). There is a number of contributors currently, and work is ongoing. Plan is to provide an open-source library people can use to for that integration.

Hi, is there some progress? Is here a plan to support GO SDK too?

Thank you.

Some progress for Java SDK integration which will be made public hopefully very soon (same timeframe alongside next spec release, will update here). I don’t know about integration with Temporal Go SDK currently. Would help to understand the amount of interest for it first.

Hi @tihomir,

Continuing with this thread.

I’m looking at serverless workflow with Go. I’m creating a prototype but am running into an issue. If the activity that workflow is executing is defined in Serverless, we do not know what the return type of the activity is in our workflow code.

e.g.

//Get the function from serverless workflow model
	function := wfdef.GetFunction(action.FunctionRef.RefName)
	ao := workflow.ActivityOptions{
		StartToCloseTimeout: 10 * time.Second,
		TaskQueue:           "test",
	}
	ctx = workflow.WithActivityOptions(ctx, ao)
	//get activity name
	activityName := function.Operation
	var result any
	//execute activity
	err := workflow.ExecuteActivity(ctx, activityName, stateInputData.Data).Get(ctx, &result)

If I run the above code - I get an error:

payload item 0: value type: interface {}: must be a concrete type, not interface

I understand why I get this message in Go as it’s expecting a type, but if my workflow is based on serverless, we don’t know in advance what the type is. How do you approach this from a serverless perspective?

Thanks,
Jay.

we do not know what the return type of the activity is in our workflow code

When using most DSLs you are bound to JSON data, as you define the manipulation of this data inside your DSL via expression languages rather than code.
Servless Workflow specification says that workflow data is a valid JSON object and that is defined to be the result of both your activities / child workflows as they are added to the JSON workflow data (based on filter rules if defined).

You can definitely define a struct that would hold your workflow data like what we did in the Java dsl sample here.
You could also define default activity result type for all your activities that just holds the JSON result that you would then merge with your workflow data.

Hope this helps.

Hi @tihomir , any update on support for serverless workflow integration with temporal(Java SDK)?

No specific update other than looking for community help on this. If you are interested please let me know so can start sharing what we have so far.

Yes, would be happy to help !!

Even I also want to help here.

Hi @tihomir , any update or timeline we shall look forward to? Please let me know if there is help needed.

Hello @tihomir, Any update or intent to turn the demo into a full DSL workflow ?

Hi @tihomir. Is support for a serverless workflow DSL in the works?