How do I get new workflow to add to existing task queue?

I have multiple workflows. The first one is very simple and fast. It gathers information from multiple sources and creates a list of files that need to be process. After it creates the list of files to process it starts a child workflow with a different task queue that loops through the list and then creates another child workflow with another task queue is created depending on the file type while does the actual processing of the file. The first workflow finishes so I can execute another one.

This works great, but the problem starts when I go and run the original workflow that gathers the list of files multiple times. It hangs and never runs at the process file workflow when I want it too add too the file processing task queue that already running so the workers keep processing files with out me having wait until they finish processing the first batch and then start the original workflow.

I tried a bunch of things to try to get it too work, but I can’t seem to get it too work. Not sure if it even possible.

It hangs and never runs at the process file workflow when I want it too add too the file processing task queue that already running

If I understand this correctly your main workflow starts the “looping” child workflow async (with parent close policy set to abandon), is this correct? If so your child workflow keep its execution when the main parent completes.

My guess here is that when you start your main workflow again it maybe tries to start the child workflow with the same id as the one currently running processing the first batch. With Temporal you can have only one running exec with a particular workflow id at one time (per namespace).
Consider maybe having unique identifier you assign to each “gather” batch of operations and use it as part of the child workflow id.

Which SDK are you using? Also if you can show pseudo code that include the ChildWorkflowOptions you set think would help.