Tctl show not outputting valid json

Community Support
1

I’m trying to build a simple python script that will extract some data from failed workflows such as the reason for the failure, etc.

To test it, I have a workflow that simply executes this:

if (true) throw ApplicationFailure.newNonRetryableFailure("ERROR", "dehk2");

However, when I try to dump the ExecutionHistory of this workflow, the stack trace is not “valid” json.

{
    "EventId":5,
    "EventTime":"2022-08-28 12":"34":44.829213835 +0000 UTC,
    "EventType":"WorkflowExecutionFailed",
    "Version":0,
    "TaskId":59768879,
    "Attributes":{
        "WorkflowExecutionFailedEventAttributes":{
            "Failure":{
                "Message":"ERROR",
                "Source":"JavaSDK",
                "StackTrace":"io.temporal.failure.ApplicationFailure.newNonRetryableFailureWithCause(ApplicationFailure.java":"127)
io.temporal.failure.ApplicationFailure.newNonRetryableFailure(ApplicationFailure.java":"108)
com.saifxhatem.temporal.workflows.impl.WorkflowImpl.workflowMethod(WorkflowImpl.java":"46)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java":"62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java":"43)
java.lang.reflect.Method.invoke(Method.java":"498)
io.temporal.internal.sync.POJOWorkflowImplementationFactory$POJOWorkflowImplementation$RootWorkflowInboundCallsInterceptor.execute(POJOWorkflowImplementationFactory.java":"321)
io.temporal.internal.sync.POJOWorkflowImplementationFactory$POJOWorkflowImplementation.execute(POJOWorkflowImplementationFactory.java":"295)
io.temporal.internal.sync.WorkflowExecuteRunnable.run(WorkflowExecuteRunnable.java":"53)
io.temporal.internal.sync.SyncWorkflow.lambda$start$0(SyncWorkflow.java":"131)
io.temporal.internal.sync.CancellationScopeImpl.run(CancellationScopeImpl.java":"101)
io.temporal.internal.sync.WorkflowThreadImpl$RunnableWrapper.run(WorkflowThreadImpl.java":"110)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java":"511)
java.util.concurrent.FutureTask.run(FutureTask.java":"266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java":"1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java":"624)
java.lang.Thread.run(Thread.java":748),
                "FailureInfo":{
                    "ApplicationFailureInfo":{
                        "Type":dehk2,
                        "NonRetryable":true
                    }
                }
            },
            "RetryState":"RetryPolicyNotSet",
            "WorkflowTaskCompletedEventId":4
        }
    }
}

Any attempts to parse it in python will throw an error because of the stacktrace line.

Interestingly enough, if I write it to a file, the file gets some \n thrown in there, so it turns into valid json.

However, the machine that will execute the script will not be the machine where tctl runs, so I do not want to deal with the hassle of moving files, so I would prefer to read it in memory.

Is there any way to get valid json from tctl show?

3

Is there any way to get valid json from tctl show?

With Java SDK you could get the history json (pretty printed) via api, for example:

public static String getWorkflowExecutionHistoryAsJson(String wfId, String wfRunId) {
  GetWorkflowExecutionHistoryRequest request =
          GetWorkflowExecutionHistoryRequest.newBuilder()
                  .setNamespace("default")
                  .setExecution(WorkflowExecution.newBuilder()
                          .setWorkflowId(wfId)
                          .setRunId(wfRunId)
                          .build())
                  .build();
  return new WorkflowExecutionHistory(
          service.blockingStub().getWorkflowExecutionHistory(request).getHistory()).toJson(true);
}

Looking at tctl the only way it will encode the raw workflow history to json is to specify the --output_filename prop currently. Opened issue here to address this.

For now maybe what can help is to do create a file but not pass the file and pipe the output to cat, for example:

tctl wf show -w <wfid> -r <wfrunid> --output_filename wf.txt | cat wf.txt

4

Yeah, I noticed writing to file would output valid json. This has really thrown a wrench in my plans :confused:

Thank you for your help.

5

Just to add, you could also use grpcurl and Temporal api proto files so you could do something like:

git clone https://github.com/temporalio/api

cd api

grpcurl -plaintext --import-path ./ -proto temporal/api/workflowservice/v1/service.proto  \
-d '{"namespace": "<namespace_name>", "execution": {"workflow_id": "<wfid>", "run_id": "<runid>"}} \ 
<frontend-host>:<frontend-port> temporal.api.workflowservice.v1.WorkflowService/GetWorkflowExecutionHistory

This should give you valid JSON as well. Maybe this helps.

To see all grpc services exposed by frontend can do:

grpcurl -plaintext <frontend-host>:<frontend-port> list

1 Like
6

I’ll try to use this but I have a question.

According to grpcurl, you can use list to describe the services themselves.

e.g. if I use ./grpcurl -plaintext localhost:7233 list

This is the output:

grpc.health.v1.Health
grpc.reflection.v1alpha.ServerReflection
temporal.api.workflowservice.v1.WorkflowService
temporal.server.api.adminservice.v1.AdminService

If I try to use list on the WorkflowService: temporal.api.workflowservice.v1.WorkflowService

Failed to list methods for service "temporal.api.workflowservice.v1.WorkflowService": Symbol not found: temporal.api.workflowservice.v1.WorkflowService

However, if I try to ust list on one of the (I’m assuming) default grpc services

./grpcurl -plaintext localhost:7233 list grpc.health.v1.Health

grpc.health.v1.Health.Check
grpc.health.v1.Health.Watch

Not sure if I’m doing something wrong or if this is an bug.

7

Still checking on why list/describe operations are not exposed for temporal services. Will update when know more.

Both of these commands work when using protos from api repo, for example:

grpcurl -plaintext --import-path ./ -proto temporal/api/workflowservice/v1/service.proto list temporal.api.workflowservice.v1.WorkflowService

If that helps for now.

8

Ok found existing issue here that explains why reflection api is not yet implemented for this, as well as great idea to try using evans.

1 Like
9

Thank you so much for your help.