Hi Team, I have a very complex business use-case for which i am trying to see how can i leverage the power of temporal and build a resilient application.
Pleaser refer to this flow : Mermaid Chart - Create complex, visual diagrams with text. A smarter way of creating diagrams.
Our biz idea supports multi-tenancy - so there are 1…N number of tenants using our app simultaneously - each of them interacts with our webapp and will provide some raw-data information.
workflow-tenant_01- we wanted to keep one long running workflow (which will be the parent workflow). this will run throughout the lifecycle of the tenant. the job of this workflow is to simple take in the raw-data from the user and pass it on to the backend microservice -
workflow-config-tenant_01is the workflow that takes in the raw-data and generates meaningful config (which is a HUGE json payload). so basically
workflow-config-tenant_01becomes the child-workflow which is triggered by the parent workflow
here on , each
workflow-config-tenant_01have 1…N Services - denoted by
workflow-config-service-tenant_01- so now this becomes the child workflow and
workflow-config-tenant_01becomes the parent workflow - and depending upon the config - these workflows will either run concurrenlty or sequentially
each service workflow
workflow-config-service-tenant_01will run a set of CRUD ops (again they may run concurrently or sequentially based on the config) - and each of them will be an
Each tenant is isolated under temporals
NOTE: the parent workflow is in python and the remaining downstream child workflow are in golang.
HELP NEEDED on the below points.
workflow-tenant_01is the only workflow that initiates the entire cycle - whats the best way to trigger an event that will start the child workflows ? should i use an http request or a signal? i use protobufs - so is rpc preferred over http ?
if more tenants gets added - how do we dynamically add new parent workflows ?
how do we run concurrent workflow-executions ? i believe workflow.Go() is for running concurrent activities and not workflows , right ?
there are workflows which are independant and can run concurrently and then there are workflows that are dependant on each-other and must be run sequentially - whats the best way to manage workflow execution considering both these scenarios ?
serviceis a standalone mircoservice (k8s deployment) - so this has a worker running , and a workflow which is waiting to be triggered by the immediate parent workflow. since this single
servicedeployment is used to handle the req’s for all the tenants - whats the best way to deal with such a situation where in a single k8s-deployment of a
service(say SERVICE_A) - that has a single
worker(which can handle millions of workflows as per docs) must run each
tenantschild workflow trigger concurrently
This would mean that we would handling millions of workflows even for a very few tenants (say 10 in number), so what are the limitations/set-backs/etc for all the above points ??
ps: Please note that all this NOT YET implemented - we’re currently in idea evaluation phase - and are working on our POC