Why OpenTracing context doesn't flow into async activity / child workflow execution?

Community Support
,
1

Hello,
I am testing open tracing integration with Temporal (java-sdk). Everything looks good except when I use async execution of activities or child workflows - in that case I get multiple, not related separate traces (with root spans for main WF and for each async invocation…).
It seems the reason is that when any outbound tracing interceptor executes (either for child WF or for activity), it relies on the fact that there’s the active parent span - which is null when activity is executed asynchronously (this is what I noticed during debugging) - quite expected when trace context is stored in thread local…
I configure interceptors for both worker & workflow client.
Do I miss any additional configuration or this is just not implemented / supported yet?..
Thanks!

1 Like
2

Hi @Maxim_Chuvilyaev, can you please open an issue for this in our Java SDK? Adding a small reproducible that shows the issue you are running into would help a lot if possible as this indeed sounds like a possible bug.

3

Hi @tihomir ,
Thanks for you answer, I will open a ticket in java-sdk github. The setup that I have cannot be really used as a reproduction :wink: (real project, written in kotlin). TBH, I think that any WF that invokes a child WF or (at least 2) activities asynchronously should reproduce the issue.
But I will try to amend some existing tests in temporal-opentracing, and provide the code.

UPDATE: I reproduced the issue using existing tests in temporal-opentracing and created a new issue in github:

4

@Maxim_Chuvilyaev
Thank you for reporting the issue and for providing steps to reproduce!

5

I am seeing a similar issue where I use Async.procedure to start a non-activity method. My class uses an @Autowired instance of org.springframework.cloud.sleuth.Tracer. In the method started by Async.procedure, this.tracer.currentSpan() returns null, although the logs still show the expected trace, span and parentIds. However, if I want to create a child span, I need to pass the span into the method started by Async.procedure. In this example, I added comments showing the values of the various tracing fields:

public void workflowMethod() {
...
  // "parentId":"e6e043d543b82662","spanId":"7eb8dde76a0dc167","traceId":"e6e043d543b82662"
  Async.procedure(this::doWork, this.tracer.currentSpan);
...
}

public void doWork(Span parentSpan) {
        // tracer.currentSpan() returns null here
        // "parentId":"e6e043d543b82662","spanId":"7eb8dde76a0dc167","traceId":"e6e043d543b82662"
        try (Tracer.SpanInScope ws = tracer.withSpan(tracer.nextSpan())){
            // "spanId":"193f0876f1ae6171","traceId":"193f0876f1ae6171" -- parentId is lost completely
            ...
        }
        Span span = tracer.nextSpan(parentSpan);
        try (Tracer.SpanInScope ws = tracer.withSpan(span)) {
          // This section has the values want
          // "parentId":"7eb8dde76a0dc167","spanId":"a0aea87f2b406565","traceId":"e6e043d543b82662"
          ...
        }
}

Is this the same issue as Open tracing span context not propagated when activity or child workflow invoked asynchronously · Issue #537 · temporalio/sdk-java · GitHub, or is it something different? I wanted to ask in case a new issue should be filed for this.

6

Hey Nathan,

It’s a known issue, but I just made this for visibility:

Thank you for the report.