We introduced a new polling activity and explicitly set a large ScheduleToCloseTimeout (e.g., 1500 minutes), but the Temporal UI kept showing a 30-minute timeout and the activity is getting failed at first try only. Our workflow configuration was the cause: WorkflowTaskTimeout and WorkflowExecutionTimeout (30 minutes). Because the workflow task timeout acts as an upper bound on activity timeouts, Temporal capped the activity’s ScheduleToCloseTimeout at 30 minutes, ignoring the value we configured.
Given that this activity polls until the workflow times out—and our production WorkflowExecutionTimeout is much higher—what is the recommended way to choose an appropriate ScheduleToCloseTimeout for long-running polling activities? Should it align with the workflow execution timeout, or is there a better pattern for determining the ideal value?