How to handle Poller exceptions

I have this up and running:

WorkerFactory factory = WorkerFactory.newInstance(client);
WorkerOptions o = WorkerOptions.newBuilder()
                .setXXX()
                ...
                .build();
 Worker worker = factory.newWorker(temporalConfiguration.getTaskQueue(), o);

Now I want to handle exceptions thrown by the ThreadPoolExecutor backed by Poller class.

PollerOptions pollerOptions = PollerOptions.newBuilder()
                .setUncaughtExceptionHandler(handler)
                .build();

My question is, how can I set that PollerOptions while creating the Worker?
On WorkerOptions builder I can’t see any setter for PollerOptions.
I have tried on Temporal versions 1.17.0 and 1.19.0.

Regards,