# Child workflow only initiated but not getting executed

**URL:** <https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351>\
**Category:** Community Support\
**Tags:** go-sdk\
**Created:** [March 13, 2024, 6:55am UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351 "2024-03-13T06:55:07Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![tsanydzul](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/tsanydzul/32/5036_2.png) [@tsanydzul](https://community.temporal.io/u/tsanydzul)\
**Post date:** [March 13, 2024, 6:55am UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/1 "2024-03-13T06:55:07Z")

</div>

Hello, i got an issue where its intermittent, sometimes when there is a parent workflow tried to initiate a child workflow it got stuck for some reason

and something in common for every stuck CWF is the event of the CWF is only getting initiated but no event for executed, while in success execution i can see Cwf getting initiated and getting started

Stuck CWF

 ![Cwf stuck, not getting executed](https://us1.discourse-cdn.com/flex016/uploads/temporal/original/2X/5/5b317cd0ad46e3d345e97d06d801eb1502e611fd.jpeg)

Success CWF

 ![success cwf](https://us1.discourse-cdn.com/flex016/uploads/temporal/original/2X/7/7197218db8e5ed9097a70ca5380ab69bc60b7d14.jpeg)

on the stacktrace of stuck cwf i found this

 ![image](https://us1.discourse-cdn.com/flex016/uploads/temporal/original/2X/b/b17c06002eef4c390b0fe05df817675624ce146e.png)

any suggestion how i trace the issue ? or how i can terminate cwf when its note getting executed yet ? since its stuck and cannot reset since the cwf still running

---

<div class="post-metadata">

**Author:** ![tsanydzul](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/tsanydzul/32/5036_2.png) [@tsanydzul](https://community.temporal.io/u/tsanydzul)\
**Post date:** [March 15, 2024, 3:14am UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/2 "2024-03-15T03:14:08Z")

</div>

When i tried to describe, the wf transaction got shard error

 ![image](https://us1.discourse-cdn.com/flex016/uploads/temporal/original/2X/4/45e4fbb587a2c565767dba7e85753b7b334d773d.png)

---

<div class="post-metadata">

**Author:** ![maxim](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/maxim/32/8_2.png) [@maxim](https://community.temporal.io/u/maxim)\
**Post date:** [March 15, 2024, 6:42pm UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/3 "2024-03-15T18:42:56Z")

</div>

If you are getting this consistently, then your cluster/database might be broken.

---

<div class="post-metadata">

**Author:** ![Ahmad\_Ricky\_Nazarrud](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/ahmad_ricky_nazarrud/32/1864_2.png) [@Ahmad\_Ricky\_Nazarrud](https://community.temporal.io/u/Ahmad_Ricky_Nazarrud)\
**Post date:** [March 20, 2024, 4:40pm UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/4 "2024-03-20T16:40:50Z")

</div>

do you have suggestion on how should we do to fix it

---

<div class="post-metadata">

**Author:** ![maxim](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/maxim/32/8_2.png) [@maxim](https://community.temporal.io/u/maxim)\
**Post date:** [March 21, 2024, 12:59am UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/5 "2024-03-21T00:59:22Z")

</div>

You should probably reinstall the cluster and make sure that the DB is fully consistent. Do you run the DB with some type of async replication?

---

<div class="post-metadata">

**Author:** ![Ahmad\_Ricky\_Nazarrud](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/ahmad_ricky_nazarrud/32/1864_2.png) [@Ahmad\_Ricky\_Nazarrud](https://community.temporal.io/u/Ahmad_Ricky_Nazarrud)\
**Post date:** [March 25, 2024, 6:50am UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/6 "2024-03-25T06:50:05Z")

</div>

I dont think we have any specific type of async replication.

What do you mean by reinstall the cluster? do we need to drop/truncate certain table or how?  
and also does reinstall the cluster, will impact the existing workflow that running?

---

<div class="post-metadata">

**Author:** ![maxim](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/maxim/32/8_2.png) [@maxim](https://community.temporal.io/u/maxim)\
**Post date:** [March 25, 2024, 6:00pm UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/7 "2024-03-25T18:00:20Z")

</div>

I’m not sure if reinstalling will help if your DB loses records. Usually, this happens when some sort of replication is used, and it is not fully consistent. As I know nothing about your specific setup it is hard to give any recommendation.

---

<div class="post-metadata">

**Author:** ![Ahmad\_Ricky\_Nazarrud](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/ahmad_ricky_nazarrud/32/1864_2.png) [@Ahmad\_Ricky\_Nazarrud](https://community.temporal.io/u/Ahmad_Ricky_Nazarrud)\
**Post date:** [March 25, 2024, 6:45pm UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/8 "2024-03-25T18:45:15Z")

</div>

Here is the illustration of how we setup the cassandra right now

 ![image](https://us1.discourse-cdn.com/flex016/uploads/temporal/original/2X/4/424aaeb7637a2fb8a81855327bb0ef0bfd3a8a52.png)

we already try temporal tctl command “tctl admin shard close\_shard”, but it seems doesnt solve the problem.  
we already try to exec cassandra command such as nodetool removenode, cleanup, repair. but the problem is still there.

we capture log from temporal history service:  
{“level”:“error”,“ts”:“2024-02-02T02:54:57.744Z”,“msg”:“Persistent store operation failure”,“service”:“history”,“shard-id”:32,“address”:“10.59.68.74:7234”,“shard-item”:“0xc00277ea00”,“store-operation”:“update-shard”,“error”:“Failed to update shard. previous\_range\_id: 30415, columns: (range\_id=30414)”,“shard-range-id”:30416,“previous-shard-range-id”:30415,“logging-call-at”:“context\_impl.go:732”}

---

<div class="post-metadata">

**Author:** ![Ahmad\_Ricky\_Nazarrud](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/ahmad_ricky_nazarrud/32/1864_2.png) [@Ahmad\_Ricky\_Nazarrud](https://community.temporal.io/u/Ahmad_Ricky_Nazarrud)\
**Post date:** [March 26, 2024, 5:55am UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/9 "2024-03-26T05:55:38Z")

</div>

@maxim @tihomir

---

<div class="post-metadata">

**Author:** ![maxim](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/maxim/32/8_2.png) [@maxim](https://community.temporal.io/u/maxim)\
**Post date:** [March 27, 2024, 3:27am UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/10 "2024-03-27T03:27:31Z")

</div>

Sorry, I’m not sure why your Cassandra is inconsistent.

---

<div class="post-metadata">

**Author:** ![Ahmad\_Ricky\_Nazarrud](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/ahmad_ricky_nazarrud/32/1864_2.png) [@Ahmad\_Ricky\_Nazarrud](https://community.temporal.io/u/Ahmad_Ricky_Nazarrud)\
**Post date:** [March 27, 2024, 11:06am UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/11 "2024-03-27T11:06:29Z")

</div>

do you have any other suggestion for us what needs to be checked? or what should we do?

---

<div class="post-metadata">

**Author:** ![maxim](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/maxim/32/8_2.png) [@maxim](https://community.temporal.io/u/maxim)\
**Post date:** [March 27, 2024, 4:11pm UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/12 "2024-03-27T16:11:03Z")

</div>

I recommend finding Cassandra experts who can troubleshoot inconsistent behavior.

Another option is to use [Temporal Coud](https://temporal.io/cloud) and never care again about Temporal cluster and persistence management.

---

<div class="post-metadata">

**Author:** ![Ahmad\_Ricky\_Nazarrud](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/ahmad_ricky_nazarrud/32/1864_2.png) [@Ahmad\_Ricky\_Nazarrud](https://community.temporal.io/u/Ahmad_Ricky_Nazarrud)\
**Post date:** [March 27, 2024, 8:36pm UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/13 "2024-03-27T20:36:02Z")

</div>

If lets say i setup a new cluster, is there any way we migrate the existing data from existing to the new cluster? and not impacting the current workflow process/state

---

<div class="post-metadata">

**Author:** ![maxim](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/maxim/32/8_2.png) [@maxim](https://community.temporal.io/u/maxim)\
**Post date:** [March 28, 2024, 5:03pm UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/14 "2024-03-28T17:03:00Z")

</div>

Yes, you can migrate data using [multi-cluster](https://docs.temporal.io/self-hosted-guide/multi-cluster-replication) replication. But as your current cluster is corrupted, I’m not sure if this is going to work.

---

<div class="post-metadata">

**Author:** ![Ahmad\_Ricky\_Nazarrud](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/ahmad_ricky_nazarrud/32/1864_2.png) [@Ahmad\_Ricky\_Nazarrud](https://community.temporal.io/u/Ahmad_Ricky_Nazarrud)\
**Post date:** [March 28, 2024, 5:33pm UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/15 "2024-03-28T17:33:58Z")

</div>

do you know what this tctl command does? and when we need to use this command

tctl admin shard close\_shard

---

<div class="post-metadata">

**Author:** ![Ahmad\_Ricky\_Nazarrud](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/ahmad_ricky_nazarrud/32/1864_2.png) [@Ahmad\_Ricky\_Nazarrud](https://community.temporal.io/u/Ahmad_Ricky_Nazarrud)\
**Post date:** [March 28, 2024, 7:44pm UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/16 "2024-03-28T19:44:14Z")

</div>

![image](https://us1.discourse-cdn.com/flex016/uploads/temporal/original/2X/4/4fc101e525265fd6f932d3bc2025523d1e09f7ed.jpeg)  
Hi @maxim , @tihomir ,  
I try to execute tctl command to describe shard\_id but there is one shard\_id that the update\_time is not updated like the other shard.

`tctl admin shard describe`

1. do you know why the shard\_id is not updated?
2. if i want to block transaction to come to this shard, can i use command `tctl admin shard close_shard` ? this mean the new transaction after i close the shard should be directed to other shard.
3. or is there any other way for me to isolate that shard

---

<div class="post-metadata">

**Author:** ![Ahmad\_Ricky\_Nazarrud](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/ahmad_ricky_nazarrud/32/1864_2.png) [@Ahmad\_Ricky\_Nazarrud](https://community.temporal.io/u/Ahmad_Ricky_Nazarrud)\
**Post date:** [April 1, 2024, 3:56am UTC](https://community.temporal.io/t/child-workflow-only-initiated-but-not-getting-executed/11351/17 "2024-04-01T03:56:11Z")

</div>

Hi @maxim , @tihomir  
do you mind to give me clearance on this?
