Hi, Temporal community!
I’m trying to implement payload encryption with Temporal and looking for some help.
What’s unusual about our use-case is that we maintain many task queues (one per customer) and we would like to use a different encryption key per customer to ensure not only Temporal Cloud can’t access our data, but there’s also no data leaking between customers.
I’m looking at the example here samples-python/encryption/worker.py at main · temporalio/samples-python · GitHub and, unfortunately, it doesn’t seem to me like I have access to the queue name or any other workflow metadata at the codec level.
Can anyone recommend a more or less robust way of passing that metadata or maybe a different (better) approach to my original problem? Thanks!
Since it’s per task queue, you can simply create different clients with different data converters for each worker. They can even share a connection, e.g.
my_common_client = await Client.connect(...)
# Setup client for customer 1
config_for_customer1 = my_common_client.config()
config_for_customer1["data_converter"] = dataclasses.replace(
temporalio.converter.default(),
payload_codec=MyEncryptionCodec("my-customer1-key")
)
client_for_customer1 = Client(**config_for_customer1)
# Setup client for customer 2
config_for_customer2 = my_common_client.config()
config_for_customer2["data_converter"] = dataclasses.replace(
temporalio.converter.default(),
payload_codec=MyEncryptionCodec("my-customer2-key")
)
client_for_customer2 = Client(**config_for_customer2)
And then use those clients with the workers. This is the best way since your keys are different at the entire task queue level. There are technically more advanced ways if the keys need to differentiate based on actual object on the same task queue (usually involves a custom payload converter that sets something in metadata for use by the codec).
Oh, I actually like this approach a lot 
Thanks for the suggestion!