Can you restrict worker client to have access to only one specific queue?

I’m considering using Temporal to distribute calls to my clients, where I have agents (workers) running. I want to direct calls to specific server workers based on dedicated queues, similar to the example: Worker-Specific Task Queues

I have one concern: Can I configure authorization so that a client only has access to a single queue?

And additional question how many tenants I can expect to support in this way before I reach any known limits?