Hi everyone, I am building a Temporal workflow designed to handle periodic third-party integration tasks, including fetching and parsing bill status updates from external web endpoints. As part of an automated activity step, the worker issues an outbound HTTP GET request to pull account metrics and html payloads directly from here.
However, during activity execution, the HTTP client call within the activity function frequently hangs or exceeds the configured ScheduleToClose timeout limit. While accessing the target portal directly in a standard web browser returns a response normally, the headless request running inside the worker activity thread stalls out and triggers repeated activity retries until the retry policy gives up.
I am also noticing a couple of related workflow state and worker performance issues with this activity setup. When the outbound HTTP request blocks, the activity worker thread remains occupied, causing the local task queue to back up and delaying execution for adjacent workflow tasks. Additionally, because the external endpoint occasionally returns chunked data or connection resets under load, Temporal logs continuous ActivityTaskTimedOut and connection reset errors, which clutters our server history and leads to workflow execution failures if non-retryable error handling isn’t explicitly defined.
Has anyone encountered similar network blocking or timeout issues when running third-party web scraping or HTTP polling within Temporal activities? I am trying to figure out if there are specific best practices for setting StartToClose timeouts, passing custom HTTP client context cancellation deadlines, or decoupling outbound web requests into asynchronous webhook patterns to prevent worker pool starvation.