Hi all — I'm fairly new to Temporal and evaluating it to automate internal business workflows. We need to visualise runs for business/admin users in our own frontend (they
won't use the Temporal Web UI) — both live progress (including human-in-the-loop steps waiting on approval) and a historical audit trail that outlives the retention period.
From forum threads (e.g. "writing update info to custom db") I understand the recommended approach at scale is to write progress to our own database and have the UI poll
that, rather than querying Temporal's own store/Visibility for this purpose.
My current plan is to capture this generically with interceptors so workflows don't need per-workflow instrumentation:
- a workflow interceptor (inbound execute) recording workflow start/complete/fail, plus propagating a root/run id via headers
- an activity interceptor on the activity worker writing directly to our DB per attempt (start, error, attempt number, completion) — no extra activities scheduled, so no
history-event cost
**My questions:**
1. Is this two-interceptor split (workflow lifecycle workflow-side, per-attempt detail activity-side) the right shape, or is there a more idiomatic mechanism I'm missing
(sinks, Visibility, history export)?
2. At some point it feels like I'm re-implementing Temporal's event history in my own DB. Where do experienced teams draw the line on what to project vs. what to leave in
Temporal (and fetch on demand)?
3. Known gaps: terminated workflows don't run interceptors, and an activity that's scheduled but never picked up writes nothing. Is a periodic reconciliation sweep
(Describe/List stale rows) the standard answer?
Any pointers to real-world examples of business-facing dashboards built on a Temporal projection would be much appreciated. Thanks!
DON’T DO THIS.
At aguru.ai, we made the same mistake; we projected the data from Temporal to our PSQL and we had a lot of inconsistency and data drifting issues, and it was not reliable.
We moved to a “single source of truth” approach by reading all data from temporal. While the workflow is running, we emit data to the temporal history, and then read it from there.
About the “retention period” limitation, we used the “archiver” feature provided by Temporal.
You will also face some performance issues with long-running workflows that have a very large history.
That sound interesting. Can you elaborate on what you mean by “we emit data to the temporal history”? Are you adding extra history entries or extra context to the history? What mechanism do you use to read from the history? Are you plugging in your own UI to display history data directly?