Defining a general LLM tool calling workflow with agent-specific tool payload transformations

Community Support
1

My team is building a distributed platform for teams to build and deploy agents. We are running into a design challenge in the case when an agent needs to define transformation logic for a specific tool’s inputs and/or outputs.

Let me focus on response truncation as a specific example of transforming the tool response.

Say we have the following:

  1. A shared ToolLoopWorkflow that specific agents can execute as a child workflow, passing it a prompt and a list of available tool definitions. This workflow will sequentially call an LLM and then the selected tools, in a loop.
  2. A shared tool, getReallyLongData, that returns a big JSON object that’s too big to put into an LLM. The workflow runs the tool by executed an activity (already defined on a remote machine).
  3. A DataAnalystAgentWorkflow that calls ToolLoopWorkflow as a child workflow, passing getReallyLongData as one of the available tools.

Since the response of getReallyLongData is too big to put directly into an LLM, my DataAnalystAgentWorkflow wants to inject a custom truncate_response() function that takes the output of getReallyLongData and returns a truncated string representation that can be fed into the LLM. That way, when ToolLoopWorkflow calls getReallyLongData, the response is run through truncate_response() before being passed back to the LLM.

The problem? I can’t pass truncate_response() to the workflow because it’s code, and I can’t inject it into the getReallyLongData tool activity’s logic because it’s a shared tool that’s defined in another project.

Is there something wrong with our approach? We’ve looked at defining hooks with Interceptors, but are not confident that’s how they’re intended to be used. Do we have to get rid of ToolLoopWorkflow and duplicate that logic for every workflow agent that needs that functionality?

Any pointers appreciated!

2

I think you have a few options.

and I can’t inject it into the getReallyLongData tool activity’s logic because it’s a shared tool that’s defined in another project.

Not entirely clear to me how your project is defining tools, but this would probably be the ideal solution as it would reduce payload sizes in history coming back from the activity. In agents frameworks I have worked with, it should be possible to redefine the tool using getReallyLongData since tools are generally just functions. But maybe creating that new tool is more complicated than typical. This method would also alleviate any problem with the size of the data hitting temporal history limit sizes and give you the ability to use something like a claim check pattern if needed.

If you can’t do that, I see two options: remove the tool calling workflow, or predefine the potential truncation functions so that the actual code doesn’t need to be passed across the workflows.

On removing the tool calling workflow, duplicating the tool calling logic shouldn’t really be necessary. The root workflows should be able to share the logic without being separate workflows by executing the same functions in themselves. This is how our integrations with AI frameworks have worked so far: Runner.run or equivalent is executed inside the root workflow rather than having a separate one.