Thanks for all the help @yux
Finally, we figured out that ther was no issue with the configurations.
It appears that most of my workflows keeps sleeping/waiting for an event.
and By design temporal do not replicate sleeping workflows… So one way of force replicating is to send a fake signal to all open workflows . This forces the replication to happen immediately.
I use a shell script in tctl to figure out all open workflow and send them a signal.
Please beware this is just a one time activity… any newly created workflow and any workflow which gets a signal in due course go get replicated on their own
Here is the shell script i use
#remove the command file to send signals (from previous run)
rm -rf force_replicate.sh
#create a fresh command file with all open workflows -> generates approproate tctl command to send fake singals to workflows
tctl -ns mynamespace wf la --op | awk '{$1="echo -n \"" ; $2=" --> \" ;tctl -ns \"mynamespace\" wf s --name fakesignal --wid ";print $1 $3 $2 $3}' |awk 'NR>1' > force_replicate.sh
# prove execute permission
chmod +x force_replicate.sh
# send fake signals now..
./force_replicate.sh