Batch Workflow best practice

Community Support
1

Hi! My service get about 200 000 record from external system and i need to process this one(sub process). At the end i need to create report with detail (problem record). I plan to save this batch in database before start temporal workflow(input for workflow ->batchId) and get it in workflow.I fund batch example,but i think it is not correct, because i can get non deterministic error(record is not sort,batch size can change).May be i have more better decision?For example get list inside activity?

  List<SingleRecord> records = recordLoader.getRecords(pageSize, offset);
    // Starts a child per record asynchrnously.
    List<Promise<Void>> results = new ArrayList<>(records.size());
    for (SingleRecord record : records) {
      // Uses human friendly child id.
      String childId = Workflow.getInfo().getWorkflowId() + "/" + record.getId();
      RecordProcessorWorkflow processor =
          Workflow.newChildWorkflowStub(
              RecordProcessorWorkflow.class,
              ChildWorkflowOptions.newBuilder().setWorkflowId(childId).build());
      Promise<Void> result = Async.procedure(processor::processRecord, record);
      results.add(result);
    }
    // Waits for all children to complete.
    Promise.allOf(results).get();
2

Sorry, did you find a bug in the batch sample? I believe it doesn’t have any problem with determinism.

3

Thanks for answer! In batch sample dataset is always static. In my case -start workflow-crash in itaration cycle-support change dataset(delete one set)-get determinism.

4

The sample will work fine with non static dataset as it iterates with data only through activities. As activity result are always recorded in the event history any crashes do not lead to determinism problems.

5

Thanks!Create process! A create the same ,but forget about sort(