Determining the Best Practices for Business Logic Placement within Temporal Activities and Workflows

I’m trying to understand where to place various types of logic in my Temporal code, particularly in terms of activities and workflows. Based on the documentation, I’ve understood that all 100% external calls should be placed in activities. However, I’m not entirely clear on where to place the rest of my code, as there are various scenarios I need to consider. Here are some examples:

  1. Standard business logic code that does not generate errors and operates deterministically. This might include things like checks, calculations, or data traversal. Should this be placed in an activity for visibility and ease of execution result monitoring, or should it be left in the workflow while leveraging object-oriented programming?
  2. Non-deterministic code that doesn’t make external calls, for instance, traversing a map. It seems we could use side-effects for these kinds of operations and leave them in the workflow, but I’m not sure if this is correct.
  3. Code that doesn’t make external calls but could potentially return errors, such as processing the result of an activity and checking if specific elements are present in the response. This can also result in errors, and it’s unclear where to place it since retry logic is not necessary.
  4. The code within activity calls can also vary. I might make a single database call and then process the result in the workflow (like handling pagination), or I could handle pagination directly within the activity and return the result. What’s the best practice in this case and what should guide my decision?

In essence, if we use activities strictly for external calls, our code would look very similar to typical codebases, except for those external calls. However, this approach seems to have its downsides: less granular monitoring if we want to see what happened at each stage. On the other hand, if we put everything into activities, the entire workflow becomes a sequence of activities with the occasional sleep, signal, and checks to see if the previous activity was successful to pass its result to the next activity. This approach provides transparency, allowing us to see each step, input parameters, and responses in the Temporal Web. However, it disrupts the programming paradigm I’m used to and raises questions about the size and logical scope of activities. Should I view them as if they don’t exist and consider every function as a potential activity following the SOLID principles?

I apologize for the rambling. I tried to explain it the best I could, and I hope you understand what I’m asking. Maybe there are comprehensive open-source projects written with Temporal that I can refer to for good practices. I’m especially curious about how this could fit with the Onion Architecture paradigm if everything should be an activity. I would appreciate hearing your thoughts on this matter.

Thank you

I don’t think a single answer works in all cases. Depending on a specific use case, different tradeoffs might be necessary. I usually start from the fine-grained activities that only perform IO and put more logic into them when the default approach is not a good fit. When designing a Temporal application, you should understand the limitations. For example, you cannot pass large payloads as activity and workflow arguments and results. It might require routing activities to the same host, or breaking them into smaller chunks, etc.

As for your examples:

  1. I would keep this as part of the workflow code unless these calculations are CPU intensive or require passing large payloads from between workflows and activities.

  2. I would rewrite map traversal to be deterministic and keep it inside the workflow.

  3. I see no problem with code that returns errors deterministically to be a part of the workflow.

  4. This is very use case specific. An activity that returns a single page of results is a typical pattern.

Thank you so much for your reply!

Does this limitation apply only to the arguments for activities and workflows? As far as I understand, Temporal stores the entire state, so even if my workflow code has a very large structure, its state will still be stored. Will the same restrictions apply in this case or not?

The workflow state is not stored directly. Only inputs and outputs of activities and child workflows are stored.

Thank you very much for your explanation. Could you provide more comprehensive information on how replay works in this case? How does Temporal understand that the code is non-deterministic (has been changed)? I thought it compares events and every operation in the workflow is somehow recorded. If it doesn’t record the results and gets executed every time during replay, does this mean that I can essentially make changes without any issues (the code will be as strange as possible, just for demonstration), for example, it was:

p := 0
p++
if p >= 1 {
p++
}

And then I change it to

p := 0
if p >= 1 {
p++
}
p++

or even change a piece of code to

b := 2
a := 0
c := b + a

Could you tell me how this works or where I can read about this in more detail?

Temporal is not going to detect any of the changes you posted. It only detects changes to externally visible commands. For example, if before the change workflow was executing the sequence of activities A, B, C and after the change A, C, B, this will be detected.

It is OK to change a workflow in a way that the ordering of commands doesn’t change. For example, adding a new field to a workflow that doesn’t affect the order of execution of any commands is fine.

Thank you for your detailed responses. However, I have a couple of further questions. From your perspective, is there any issue with iterating over a map within a workflow, or generating a UUID for instance? As I understand, these changes won’t be detected by Temporal, and if a UUID isn’t applied as an activity argument, this shouldn’t create any problems. What could potentially go wrong in this case?

If I understand correctly, the code within a workflow will essentially be restarted and executed each time. So, if we happen to iterate through a map in a different order, it shouldn’t significantly impact anything. The same applies to generating a UUID.

It depends on what code is executed while the map is iterated. For example, if an activity is executed for each map value, then the code is not going to be deterministic as it changes the order of activity invocations. If the iteration generates a sum of values that is used later. Then the order of commands stays the same, and the code is deterministic.

UUIDs are tricky. If they are used as child workflow ids then the code will be non deterministic. In some cases, code that uses them might be deterministic, but I would be super careful with such changes.