Aggregating many JSON files into Parquet

You can use ContinueAsNew to avoid the history size limit issue. Can still potentially Signal the Workflow doing the aggregation from other Workflows and periodically call ContinueAsNew to prevent a large history. Here is an example of how to use ContinueAsNew.