If I have an activity with a state (for example in Java, a non-final instance variable X in the activity implementation) and I have this workflow:
Initially, X = null
Step 1: Run activity method A, which assigns X = someObject
Step 2: Run activity method B, which depends on X being non-null
Suppose the workflow is interrupted between Step 1 and Step 2. During replay, the workflow will recover Step 1’s result from the event history without re-executing it and enter Step 2. However, X would be null in Step 2 this time.
Does this mean that activities have to be immutable? Or are activity instance variables also persisted somewhere in the event history?
Activities aren’t replayed, so when you start an activity worker, Temporal won’t send previous activity state to the activity worker.
Typically, you’ll want to run multiple activity workers for availability and to be able to have zero downtime deployments. So in a usual setup, one activity worker might pull A from the activity queue and then a different activity worker might pull B from the queue. Thus there wouldn’t be a guarantee that the same activity worker instance that set X would be called with B.
You could set up a separate activity queue and then run just one activity worker on that queue, if you didn’t mind having downtime when the single worker needed to be restarted.
If X is serializable, you could have the activity return it to the workflow and then the workflow would call B with X; or you could save X to a database. This would be the easiest solution.
Or perhaps you might have a single activity method “run A if X hasn’t been set yet, and then run B”.
For a fully general solution where you have a long-running process that maintains state that isn’t serializable, you could run a separate process pool behind a load balancer with sticky connections. Then the activity worker would simply be forwarding the A and B requests through the load balancer to the process.