My temporal cluster have two server nodes.I guess this promblem only appear in multi server nodes scene.
I use samples-go/cron to test the two server nodes temporal cluster.I find some workflow executions didn’t update their status in executions_visibility table after they completed.In the webUI,they still diplayed running.
Whats the Temporal server version you are using? Can you give more info on your multi-cluster setup?
I assume you configure standard visibility only. If you have server metrics enabled and are scraping them could you share your visibility latencies graph?
histogram_quantile(0.95, sum(rate(task_latency_bucket{operation=~"VisibilityTask.*", service_name="history"}[1m])) by (operation, le))
also check the execution status for this via tctl using:
The server version is 1.21.0.
Here is the multi-cluster setup, I use some variables to instead the real ip.Olny the ${nodeIP} is different between cluster nodes.