Hi, could someone please provide a concise description of event-store vs. visibility-store, to include relationship to ElasticSearch. I’m familiar with how to query the event-store, but the visibility-store (the description of) is . . . opaque. For example, you can attach search-attributes to the workflow, but are they only queryable from ElasticSearch? Also, within a workflow, how does one attach visibility meta to the workflow execution?
Most importantly: if you’re adding visibility meta to the visibility-store, what actually gets written? Does it include the event data? Is there a mechanism to build aggregations of meta entries? Imagine you have a workflow with 10,000 parts, where each part represents an S3 multi-part upload. Going further, you could attach heartbeat (meta?) to each activity as it progresses, updating # bytes uploaded. Displayed on Temporal UI, it’s too much information. We’ll likely build a separate UI (which is fine). But, you need the ability to reduce/aggregate. Is this something the visibility “engine” supports? Or, is the idea that you build your own event-stream out of Temporal, and aggregate that way?
Many thanks!