Temporal workflow emitting "events"

Community Support
1

I’m building an app that will be shipped as a binary and needs to emit events in its workflows.
Events in this case will mean executing activities. I want to use the app in multiple environments (preferably without compiling a version per environment), so my idea is to provide activity names and a bunch of other options as configuration to the workflow.

Something like this:

events:
  created:
    activities:
      - name: my-created-event-activity
        timeout: 20m
        failOnError: true

      - name: my-other-created-event-activity
        timeout: 10m
        failOnError: false

The workflow will read this configuration and execute the appropriate activities (in parallel? sequentially? this could also be configured?)

What do you think? Is there anything missing there? Are there any existing practices for such scenarios?

Thanks!

2

So you practically creating a DSL language for your specific use case. Make sure that you load the configuration using either local activity, side effect, or mutable side effect to avoid issues with determinism if the configuration changes.

You can also create some sort of shared library that can be used by multiple workflows. This library would load the configuration and perform needed actions (by executing activities) in a generic way.

3

I will read the file once when the worker starts. Yeah, it’s a kind of DSL, but I don’t want to overcomplicate it.

It should serve as a generic hook system though.

Are there any other options that might make sense for executing activities?

4

The whole point of DSL that it is as simple as possible for a specific use case. That’s why BPMN or StepFunctions are not DSLs.

Reading file once is not going to help if your workflow reads its content directly without using side effects or local activity. As on a worker restart the file can have different content.

Are there any other options that might make sense for executing activities?

I don’t understand the question. What exactly are you trying to achieve? So far my understanding was that for certain operations you want to have pluggable activity types. Am I missing something?

5

I mean I’ll read the file, marshal it into a struct and use that struct in the workflow (either as a global or rather passed as a field to a workflow struct with an execute function).

I’d like to provide as much options for executing activities, so I guess I should probably replicate the entire ActivityOptions (or what’s it called) in the configuration. Thinking about things like task queue, timeouts, retry policies, etc.

Fail on error is kind of different, since it’s not an activity option. So I guess my question is whether there are any other options that might make sense as a config option for a generic, DSL-based activity execution tool.

But I guess I’ll build it and we’ll see. In any case, I’ll post it here.

6

I mean I’ll read the file, marshal it into a struct and use that struct in the workflow (either as a global or rather passed as a field to a workflow struct with an execute function).

My point is that passing it “as a global” is going to break determinism if change of the config might affect workflow execution.

So I guess my question is whether there are any other options that might make sense as a config option for a generic, DSL-based activity execution tool.

I think all the ActivityOptions indeed.