Hi. I have a question about best practice of depending on filesystem files. I want to clone a git repo in first activity (save them to a ephemeral directory) and use it in following activities but worker can be reset and directory will be removed. What is best practice in this situation?
check for directory existence in the start of every following activities (and reRun clone activity if it doesn’t exists)?
is there something like activity preconditions or intra-activity dependency? or activity files (env context) can be saved in temporal itself?
For Go see sessions that would allow you to execute multiple activities on the same worker process / host, also see file processing sample.
If you are using different SDK let me know.
Thanks. We are using Python SDK
Don’t think Python SDK samples have one for this yet, but you would use the sample concept as ones for TypeScript and Java.
Idea is that you would have an activity-specific task queue (would have specific worker that polls on it) and first activity return its task queue name so it can be set for your next activity that uses the cloned git repo (on previous host).
Thanks. I’ve changed the clone activity to check git directory existence before clone (just clone if it doesn’t exists) and will call that in next activities. Hosts are ephemeral in our case (kubernetes pods).
Maybe another idea could be to have a single activity that clones the repo and then processes it per your business logic. Im not sure how long this process takes but you could consider using heartbeating from your activity so it can be retried fast if its stuck or on worker failure, but also to be able to report “progress” from your activity as well.