How is ES wired up with Kafka?

Community Support
, ,
1

Hi can someone please explain how things are wired up internally ? Especially temporal server, Kafka, Elastic Search,prometheus?

from the config its clear that temporal server connects to kafka as a producer possibly. how is ES wired up to Kafka

i don’t see any configuration in ES to consume messages from Kafka.
Also in es i see a table gets created with name ‘temporal-visibility-dev’ but there is no data in it.

When i go to kafka node and start a local consumer, i do see messages flowing in when i submit a workflow…

Can please let me know how to make es consume messages from kafa.

1 Like
7

Hey madhu,

I think what you’re asking for is very reasonable. That being said we are very focused on getting a stable V1 release out the door and documenting this stuff will take some serious time. I have already taken down what you asked here as a task and will do my best to update you when we have the docs ready. In the meantime there is a video which may help:

8

Thanks a lot @ryland

1 Like
9

Np. Will make sure to followup when we have more.