Hello,
Our environment was using temporal 0.29 and we are trying to update to v1.3.0. ( docker-compose with mysql )
The sequence for the update was as follows :
1 Stop all workers
2 Stop server with docker-compose version 0.29
3 update database schema do version 1.1
4 started server with docker-compose version 1.3.0
unfortunately the server doesn’t stay up :
temporal_1 | {“level”:“info”,“ts”:“2020-11-16T12:50:27.607Z”,“msg”:“Service resources started”,“service”:“worker”,“address”:“172.19.0.2:7239”,“logging-call-at”:“resourceImpl.go:380”}
temporal_1 | {“level”:“info”,“ts”:“2020-11-16T12:50:27.616Z”,“msg”:“Get dynamic config”,“name”:“worker.executionsScannerEnabled”,“value”:“false”,“default-value”:“false”,“logging-call-at”:“config.go:79”}
temporal_1 | {“level”:“info”,“ts”:“2020-11-16T12:50:27.616Z”,“msg”:“Get dynamic config”,“name”:“worker.taskQueueScannerEnabled”,“value”:“true”,“default-value”:“true”,“logging-call-at”:“config.go:79”}
temporal_1 | panic: runtime error: invalid memory address or nil pointer dereference
temporal_1 | [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1bca5b6]
temporal_1 |
temporal_1 | goroutine 369 [running]:
temporal_1 | go.temporal.io/server/service/history.(*workflowExecutionContextImpl).loadWorkflowExecution(0xc000f209c0, 0x3285ea0, 0xbfe4bb44e1da2ea1, 0x1514b56f, 0x3285ea0)
temporal_1 | /temporal/service/history/workflowExecutionContext.go:342 +0x436
temporal_1 | go.temporal.io/server/service/history.loadMutableStateForTimerTask(0x2520920, 0xc000f209c0, 0xc0001c6c40, 0x2506b80, 0xc000421380, 0x2506a60, 0xc0005e1d20, 0x2520920, 0xc000f209c0, 0xc000f2a360, …)
temporal_1 | /temporal/service/history/nDCTaskUtil.go:126 +0x4c
temporal_1 | go.temporal.io/server/service/history.(*timerQueueActiveTaskExecutor).executeActivityTimeoutTask(0xc0005e1d30, 0xc0001c6c40, 0x0, 0x0)
temporal_1 | /temporal/service/history/timerQueueActiveTaskExecutor.go:176 +0x16c
temporal_1 | go.temporal.io/server/service/history.(*timerQueueActiveTaskExecutor).execute(0xc0005e1d30, 0x250cac0, 0xc0001c6c40, 0x3285e01, 0x21d946ce, 0xed7446c93)
temporal_1 | /temporal/service/history/timerQueueActiveTaskExecutor.go:96 +0xd9
temporal_1 | go.temporal.io/server/service/history.(*timerQueueActiveProcessorImpl).process(0xc00015ee00, 0xc000f11e60, 0xed7446c93, 0x0, 0x0)
temporal_1 | /temporal/service/history/timerQueueActiveProcessor.go:303 +0x8e
temporal_1 | go.temporal.io/server/service/history.(*taskProcessor).processTaskOnce(0xc0001b3a40, 0xc000dbc780, 0xc000f11e60, 0x0, 0x0, 0x0, 0x0)
temporal_1 | /temporal/service/history/taskProcessor.go:258 +0xa2
temporal_1 | go.temporal.io/server/service/history.(*taskProcessor).processTaskAndAck.func1(0x1fef040, 0xc000f14c80)
temporal_1 | /temporal/service/history/taskProcessor.go:211 +0x75
temporal_1 | go.temporal.io/server/common/backoff.Retry(0xc000969ee0, 0x24b97c0, 0xc000630330, 0xc000969ed0, 0x0, 0xc000dc0f88)
temporal_1 | /temporal/common/backoff/retry.go:103 +0xf1
temporal_1 | go.temporal.io/server/service/history.(*taskProcessor).processTaskAndAck(0xc0001b3a40, 0xc000dbc780, 0xc000f11e60)
temporal_1 | /temporal/service/history/taskProcessor.go:238 +0x1b8
temporal_1 | go.temporal.io/server/service/history.(*taskProcessor).taskWorker(0xc0001b3a40, 0xc000dbc780)
temporal_1 | /temporal/service/history/taskProcessor.go:161 +0x8e
temporal_1 | created by go.temporal.io/server/service/history.(*taskProcessor).start
temporal_1 | /temporal/service/history/taskProcessor.go:135 +0x4b
Tried to use version 1.2.1 instead and seems to work ok.
What is odd is that we had a separated environment that we followed that same update procedure and all worked well.
Any thoughts on what is wrong? we need a version 1.3.0 for the metrics with prometheus.
Thanks again for all the help.