Batching requests to Elasticsearch

RyanThomas · October 19, 2020, 8:10pm

When Temporal is handling a high volume of workflows per second, does batch together documents for multiple workflow executions into one request to Elasticsearch?

Wenquan_Xing · October 19, 2020, 8:21pm

Yes:

github.com

temporalio/temporal/blob/e299cdd3dae36776312c656c5d34602fdf3f9bc2/service/worker/indexer/esProcessor.go#L102-LL111


	params := &es.BulkProcessorParameters{
		Name:          processorName,
		NumOfWorkers:  config.ESProcessorNumOfWorkers(),
		BulkActions:   config.ESProcessorBulkActions(),
		BulkSize:      config.ESProcessorBulkSize(),
		FlushInterval: config.ESProcessorFlushInterval(),
		Backoff:       elastic.NewExponentialBackoff(esProcessorInitialRetryInterval, esProcessorMaxRetryInterval),
		BeforeFunc:    p.bulkBeforeAction,
		AfterFunc:     p.bulkAfterAction,
	}

Topic		Replies	Views
Action batching Community Support	3	1177	August 23, 2022
Beginner Question : Using Elastic Search Community Support elasticsearch	5	3313	August 16, 2021
Elasticsearch for visibility store Community Support elasticsearch	0	560	October 24, 2023
Elasticsearch error for visibilitye Community Support elasticsearch	12	3324	May 3, 2021
Operation not support. Please use on ElasticSearch Community Support java-sdk	5	1684	June 23, 2022

Batching requests to Elasticsearch

Related topics