I understand that heartbeats are throttled to the server. I’m trying to understand activity heartbeat interactions with activity errors.
Are unsent heartbeats sent when the associated activity errors? From what I observe, the answer is “no, heartbeats are only sent according to a local timer”. Could someone confirm/deny this behaviour?
Yes. I will open an issue to update Activities | Temporal Documentation with this info. In addition to throttling, we send the heartbeat when the activity returns an error so that resumption can have those details available.
Mmm, is there a way I could turn on the worker’s logging to see when a heartbeat is sent to the server? I might have a bug in my code because my heartbeat data doesn’t always contain what I would expect. Thanks!
Heartbeat recording to the actual server is asynchronous without a way to intercept. But you can intercept every call to heartbeat from your activity with an activity outbound interceptor. However, we do not have a way to intercept when it is actually sent to the server. Technically a gRPC interceptor could be written, but that is low level and discouraged.