Querying History Data

Proto: https://github.com/temporalio/api/blob/master/temporal/api/workflowservice/v1/service.proto
Example: How can I get a workflow state/status from an external application in java?


Hi, Is there any documentation on the params. specifically interested in how to process the different pages. Does it just return a page and i’m suppose to use the next token to grab the next? or do you have any example of processing the data.

since the UI view doesn’t really show the parent to child to child relationship clearly, i wanted to just pull the data and just render the nodes myself. Wondered if there is an easy way to do that with getting the data in one shot or will it require multiple queries

For example, on the UI I notice that you are able to filter by child, workflow, activity types. When i look at the interface, I’m only seeing the ability to filter by

Also it’s not clear how you identify what event type belong to child. Do you just filter to see if “child” is in the event type?

Example, is child workflow enum value 29 - 36, and activity is enum value 10-16, and everything else falls under workflow?,

Also how do you get the timeElapsedDisplay . From the json in the UI, it’s not clear how you get this value from the API other then subtracting the event time from the first event.

Thanks,
Derek

I would recommend against history parsing approach to render information in the UI. Have you considered maintaining a list of child workflows in the parent and returning the relevant information through a query?

Another option is to enable ElasticSearch for list queries. These queries support predicates which would allow listing all children of a given workflow.

  1. Why do you recommend against using the history? It looks like the temporal UI is using the history endpoints to render the UI for summary as well and not just history. When i look at the request for temporal web, it looks like for the summary it’s pulling information from ‘’/api/namespaces/:namespace/workflows/:workflowId/:runId’’ and which make two api calls; one to describeWorkflow and the other to getWorkflowExecutionHistoryAsync. It then just aggregates the data from the history based on workflowId. Interestingly it’s not clear why that is done since it looks like all the data from describeworkflow can be derived from the history data.

  2. What happens if the workflow is complete, wouldn’t the data be lost at that point and you won’t have any query endpoint to hit? I thought you could only query the workflow if it was still active?

  3. When you say elastic search, are you referring to setting up Temporal service with an elastic search db correct? Right now we have it setup with a sql datastore.

Thanks
Derek

  1. Why do you recommend against using the history? It looks like the temporal UI is using the history endpoints to render the UI for summary as well and not just history. When i look at the request for temporal web, it looks like for the summary it’s pulling information from ‘’/api/namespaces/:namespace/workflows/:workflowId/:runId’’ and which make two api calls; one to describeWorkflow and the other to getWorkflowExecutionHistoryAsync . It then just aggregates the data from the history based on workflowId. Interestingly it’s not clear why that is done since it looks like all the data from describeworkflow can be derived from the history data.

It should not be done this way. Describe output should be enough for the summary page. @Ruslan could you clarify?

What happens if the workflow is complete, wouldn’t the data be lost at that point and you won’t have any query endpoint to hit? I thought you could only query the workflow if it was still active?

You can query closed workflows without any problem up to the retention period. But after the retention period, the execution history is not available as well.

When you say elastic search, are you referring to setting up Temporal service with an elastic search db correct? Right now we have it setup with a sql datastore.

No, ES used for visibility queries only. SQL or Cassandra are still required for all other DB operations.

What would be nice if the describe output also had the workflow input and output? This would help see what input is actually being passed in.

@Ruslan Here is the snippet of code that I see in the temporal web code for the summary that relies on the history API to get additional data.

I see. Note that it does it only for the archived workflows which don’t provide any information through describe. IMHO it is not really good design. I would rather include “describe” output into the archived file.

Just to clarify, are you saying that the history data (archive) doesn’t provide any of the information in describe.

Is there any plans to move the describe output into the history data (achieve)?

I looked at the code more closely. It reads only the first event which I think is OK. I’m against parsing the whole history to extract some information. Assuming that history can be up to 200k events and 100mb in size it can be potential performance issue.

@darewreck
As for the elapsed time calculation: it is done on the client/Web side using this helper https://github.com/temporalio/web/blob/d1d9772a1482b84d5a1a7c96f13e30c7f78471c9/client/routes/workflow/helpers/get-time-elapsed-display.js

History events filtering by type is done on the Web side as well in the view layer: https://github.com/temporalio/web/blob/44a83dae53a9af237b1b48c786c5a3a725c9f248/client/routes/workflow/history.vue#L355 . This implementation will likely change due to performance reasons especially for large histories

@darewreck

I have an item tracking visualization of parent to child relationship on Web UI.
For now i have some other items as top priorities to address, then as i start working on parent-child relationship it can be quite valuable to hear your input over the proposed UX and see whether it serves your needs :raised_hands:

If you are interested to review and give feedback, feel free to dm me here

This topic was automatically closed after 2 days. New replies are no longer allowed.