Sharing discovery: Setup temporal with nx monorepo and nest.js

Setup temporal with nx monorepo and nest.js

Hello everyone! I hope this post find you well.

Context

I wanted to give a little bit of context here, you can skip to later sections if this doesn’t interest you

I’ve being in a path to learn more about distributed systems and I came to find temporal.io. It has been a lot of trial and errors to understand and get a grasp on this new type thinking with workflows.

Since I’ve never had contact with temporal, I took the natural frictionless path to start an empty project and begin writing code to help with the mental model and concepts around the workflow architecture.

Everything was going well so I decided to bring temporal into my existing project that currently uses Typescript with nest.js in a monorepo structure using nx. All hell broke loose and my terminal looked like a christmas tree with only red lights :joy:

On the following sections I’ll explain what I had to do to integrate temporal into my existing project!

The problem

Temporal requires that the workflows must be bundled to be used in a worker. In a regular node.js application this would be easily achieved by adding another file into the building process, but with a monorepo project buildings and pipelines get a little more delicate and needs attention.

My build process attempted to bundle the workflow file and caused a lot of issues with the way temporal needs the workflow to be bundled and throwed errors like:

  • [ERROR] Field 'browser' doesn't contain a valid alias configuration...
  • [ERROR] Module not found: Error: Can't resolve './apps/temporal-worker/src/workflows/index.ts
  • [ERROR] Failed to activate workflow[...]error: TypeError: 'sayName' is not a function

The solution

I’m assuming you have worked with nest.js and nx or other monorepo system.

The solutions is quite simple, just add a new stage in the nx build execution.

Project Structure

Here’s an image of my project structure. This temporal-worker is inside apps.

Screenshot_2022-12-08_at_17.46.43

The workflows folder contains all the workflows in my application and an index.ts to export the functions to be bundled.

Screenshot_2022-12-08_at_18.09.17

Actions needed:

1. Implement function to bundle the workflows:

// File: utils/create-workflow-bundle.js
const { bundleWorkflowCode } = require('@temporalio/worker');
const { writeFile } = require('fs/promises');
const path = require('path');

async function bundle() {
	// Recommended bundle function from temporal sdk
  const { code } = await bundleWorkflowCode({
		// the path here must be the file that exports all workflows
    workflowsPath: require.resolve('../workflows/index.ts')
  });
		// the path here will be the location of the bundled workflow code
		// and will be used for the worker to run the workflows
  const codePath = path.join(__dirname, '../workflows/workflow-bundle.js');

  await writeFile(codePath, code);
  console.log(`Bundle written to ${codePath}`);
}
bundle();

2. Add bundle stage in the build process

// File: project.json

"bundle-workflow": {
  "executor": "nx:run-commands",
  "dependsOn": [],
  "options": {
    "commands": [
      "rm -f apps/temporal-worker/src/workflows/workflow-bundle.js",
      "node apps/temporal-worker/src/utils/create-workflow-bundle.js"
    ],
    "parallel": false
  },
  "configurations": {}
}

The paths used in commands are unique to my project. So if you find yourself struggling to understand what file they relate to, just go back at the beginning of the post and check the project structure.

3. Add dependency to other build stages:

// File: project.json

"build": {
      "executor": "@nrwl/webpack:webpack",
      "dependsOn": [
        "^build",
        "bundle-workflow"
      ],
...
}

:pushpin: Full project.json file :link: here

4. Use the recommended worker structure

// File: worker.ts
import { Worker } from '@temporalio/worker';
import * as activities from './activities';

export async function runWorker() {
  // Step 1: Register Workflows and Activities with the Worker and connect to
  // the Temporal server.

  const worker = await Worker.create({
    workflowBundle: {
      codePath: require.resolve('./workflows/workflow-bundle.js')
    },
    activities,
    taskQueue: 'crawl-queue'
  });

  // Step 2: Start accepting tasks on the `hello-world` queue
  await worker.run();
}
7 Likes

I would vote for adding this example to GitHub - temporalio/samples-typescript

To help others figuring out how Temporal can play nice with monorepo & nx

1 Like

:+1: We’d be happy to look at a PR for an nx sample! Currently have yarn workspaces sample:

and have PRs for turborepo:

and NestJS:

Hey @loren I’ll be glad to be helpful! Is there a place where I can get the guidelines for opening a PR?

Hi, are you looking for info in Contributing section here or something else?

1 Like

How do you start the worker? Do you have a full example in github demonstrating this work?