Microservice Workflow Design

Community Support
1

Hey guys,

Really enjoying temporal and appreciate all your hard work!

I have a seemingly simple question and I suspect maybe I’m just not thinking about it correctly. Perhaps I could get some guidance as the best way to design a workflow between microservices.

Here is the sequence of events that need to happen. We own all the services. We’ve been doing this all with Kafka plus a home grown solution we’ve been struggling to find dev time to manage and add features to. But even in the best of circumstances, we run into inconsistency/unreliability issues.

  • Receive source payload - Service A
  • Transform payload - Service A
  • Save source payload - Service A
  • Preprocess input from payload - Service B
  • Create an item - Service B
  • Update source payload - Service A

I’m interested in the best way to think about this.
Should this be one workflow or two?
If two should we use a child workflow to accomplish the activities on service B?

I would expect the implementation of the Service B activities would need to live in Service B, but how do I invoke those activities from Service A?

Are there code samples for this type of multiple microservices situation?

2

You can have a single workflow that makes all the service calls. Make sure the service calls are idempotent, and use try/catch to repair state. For example:

const { reserve, unreserve, charge, refund, sendPackage } = proxyActivities<
  typeof activities
>({
  startToCloseTimeout: '1 minute',
})

export async function processOrder(order: Order): Promise<void> {
  await reserve(order)

  try {
    await charge(order)
    try {
      await sendPackage(order)
    } catch (e) {
      await refund(order)
      throw e
    }
  } catch (e) {
    await unreserve(order)
    throw e
  }
}

More info—first 10 minutes of this video: https://twitter.com/lorendsr/status/1544806504443695104

3

I would add that activities can be hosted by different services. So the same workflow is going to work without control flow changes. The only difference would be that activities running in service A and B should be scheduled on different task queues.

4

Thank you. Great job on the talk by the way!

2 Likes
5

@maxim thx for chiming in!

Question about your comment. If I understand what you are saying, this would be ideal and actually what I expected. I’m just unsure how to implement it.

Continuing with Loren’s example code – let’s say the ‘charge’ activity exists in a Payment Service separate from this Order Service.

If I wanted the ‘charge’ activity to be in the payment service, do I need to implement a call to the payment service from the Order Service to charge the order? Or is there a way to tell temporal of the charge activity and invoke it from the Order Service. Currently when I try to do that I get the following error:

Activity function 'charge' is not registered on this Worker, ...

Secondarily, typescript complains because it can’t find ‘charge’ anywhere on the Order Service.

What am I not getting?

6

If you have existing services with APIs, the simplest thing to do is call the APIs from the Activities—one Activity per API call. In this scenario, you have a single Worker, and it has all the Workflows and Activities, and everything’s on a single task queue.

Max’s system has 2 or 3 different types of Workers:

  • Workflow Worker on task queue my-workflows
  • Worker with all payment-related Activities on task queue payments
  • Worker with all order-related Activities on task queue orders

It would look like:

const { charge, refund } = proxyActivities<
  typeof paymentActivities
>({
  startToCloseTimeout: '1 minute',
  taskQueue: 'payments'
})

const { sendPackage } = proxyActivities<
  typeof orderActivities
>({
  startToCloseTimeout: '1 minute',
  taskQueue: 'orders'
})

export async function processOrder(order: Order): Promise<void> {
  await charge(order)
  try {
    await sendPackage(order)
  } catch (e) {
    await refund(order)
    throw e
  }
}
7

This post explains the differences between “call service from an activity” or “service directly implementing an activity” approaches.

8

:tada: :sunglasses: :clap:

Thank you. I finally got this working.

This is what I was missing:

const { charge, refund } = proxyActivities<
  typeof paymentActivities
>({
  startToCloseTimeout: '1 minute',
  taskQueue: 'payments'  <----   VERY IMPORTANT
})

const { sendPackage } = proxyActivities<
  typeof orderActivities
>({
  startToCloseTimeout: '1 minute',
  taskQueue: 'orders'  <----   VERY IMPORTANT
})

In this example if “charge” and “refund” are in a different service typescript will complain. For now I’m ignoring it.

The other thing that I had to understand that wasn’t quite clear to me is that, temporal allows you to do this entire workflow without needing kafka or any other message bus to send data to “charge” or “refund” in the payment service. Simply by calling the payment activities from the order service, temporal will send the data to the other microservice. :exploding_head: Incredible! This feature simplifies things so much!

@maxim @loren thank you for the help.