You want to saturate the worker first. Unless your activities are memory bound increasing the number of polling threads and concurrent activity and workflow tasks will increase CPU usage.
When the worker is CPU bound then you can either change the hardware to the one with more CPU capacity or add more workers to the pool. There is no generic recipe to which one is better for all the use cases.