Using temporal with nx monorepo

Community Support
1

Hello,
We are trying to use temporal inside an Nx monorepo with pnpm but encoutering issues that seem to be node_modules related.
We are using a webpack to package all worker related code with the worker being the entry point. When trying to start the worker, getting the following error:

Module not found: Error: Can't resolve './<omitted>/src/temporal/workflows.ts' in '/<omitted>/src/temporal' 
2022-07-06T07:46:20.137Z [ERROR] resolve './<omitted>/src/temporal/workflows.ts' in '/<omitted>/src/temporal' 
... 
Field 'browser' doesn't contain a valid alias configuration

Also tried passing in a workflowBundle which allowed the worker to start but when I try to start a new workflow, the error just happens then.

Created a minimal reproduction repo:

To see the issue simply run:

npm i 
nx serve-temporal temporal
2

Why use webpack on all worker code?

The worker bundles workflow code using webpack, but in all our samples, the worker and activity code aren’t bundled.

3

The way that nx creates the project.json configuration is using webpack to package the app code.
We placed the worker in the same project as the web app and so created a similar build configuration.
Are you saying that this is the wrong way to go about it? Do the worker and activities (and workflows?) need to live in a separate location and be bundled differently? I would love to understand more about this.

Thank you.

4

Instead of the webpack executor, can you use this?

You should be able to compile TS to JS and then run the worker with node. The equivalent in the hello-world sample would be:

# compiles src/*.ts to lib/*.js
npm run build
# runs worker
node lib/worker.js

The Worker and Activities are loaded into that Node process, and the code at ./workflows is bundled for you and run in vms when needed:

5

You can’t bundle the worker because it has binary (compiled) dependencies

6

Using tsc worked for me, thank you.
I am still having trouble using this in my original project which includes multiple projects with dependencies, but it does not seem to be a temporal issue anymore.

7

Just a follow up:
I was eventually able to run temporal on top of an nx monorepo, but it was very hard and had to use tips from here:

And also use the bare “run-commands” executor to run the worker.

1 Like
8

Hey @abadyan, I tried following the link you sent as a way of working around the issue, but I didn’t succeed, can you give more details about how you setup project.json and how to execute the worker after it’s built?

9

Just in case someone in the future needs, here’s how you can use bundling with ESM and Nx:

In your tsconfig.json:

{
  "extends": "@tsconfig/node18/tsconfig.json",
  "version": "5.0.4",
  "compilerOptions": {
    "strict": true,
    "module": "ESNext",
    "moduleResolution": "node",
    "target": "ESNext",
    "rootDir": "./src",
    "outDir": "./lib",
    "baseUrl": "."
  },
  "include": ["src/**/*.ts"]
}

The code to bundle has this general format:

import { bundleWorkflowCode } from '@temporalio/worker';
import { writeFile } from 'node:fs/promises';
import path from 'node:path';
import { createRequire } from 'node:module';
import { fileURLToPath } from 'node:url';

const require = createRequire(import.meta.url);
const __dirname = fileURLToPath(new URL('.', import.meta.url))

async function bundle() {
  const { code } = await bundleWorkflowCode({
    workflowsPath: require.resolve('../src/workflows/workflows'),
  });

  const codePath = path.join(__dirname, '../lib/workflow-bundle.js');
  await writeFile(codePath, code);
  console.log(`Bundle written to ${codePath}`);
}

bundle().catch((err) => {
  console.error(err);
  process.exit(1);
});

The easiest way was to just recreate require to be honest… I’m open to ideas here if anyone wants to share!

In the worker, here’s how you can have bundling for production but loading workflows from a file in development:

import { Worker } from '@temporalio/worker';
import { URL, fileURLToPath } from 'node:url';
import * as activities from './activities.js';
import path from 'node:path';

function workflowOptions() {
  if (process.env.STAGE === 'production') {
    const workflowBundlePathURL = new URL('./workflow-bundle.js', import.meta.url)
    return {
      workflowBundle: {
        codePath: fileURLToPath(workflowBundlePathURL)
      }
    }
  }

  const workflowsPathUrl = new URL(`./workflows/workflows${path.extname(import.meta.url)}`, import.meta.url)
  return {
    workflowsPath: fileURLToPath(workflowsPathUrl)
  }
}

async function run() {
  const worker = await Worker.create({
    ...workflowOptions(),
    activities,
    taskQueue: 'hello-world',
  });
  await worker.run();
}

The package.json scripts look like this:

{
    "test": "jest",
    "build": "tsc --build && npm run bundle",
    "bundle": "node --experimental-specifier-resolution=node --loader ts-node/esm scripts/build-workflow-bundle.ts",
    "build.watch": "tsc --build --watch",
    "worker": "nodemon --exec 'node --experimental-specifier-resolution=node --loader ts-node/esm' src/worker.ts",
}

Assuming this is under packages/workflows and that Nx is configured as a package-based monorepo, you can just run nx run workflows:build.

4 Likes
10

Do you have a full example in git of this working?

11

Hi Isaac!

Unfortunately, no. The above might still work, but I abandoned ESM because it was just too much of a hassle.

Hopefully, we’ll all be able to just use One Thing™️ in the future.

I can spin up an Nx + Temporal worker in Typescript example git repo, been using it for basically every single project in my company so far and it works quite well. I’ll link it here once I set it up.

But basic idea:

  • create nx workspace
    • heavily suggest using the integrated setup. I can elaborate why if anyone cares.
  • create an @nx/node app, put it in apps/<your-app
  • create an @nx/js library called workflows, put it in libs/workflows
  • inside your apps/<your-app>/src/main.ts, use require.resolve('@your-nx-workspace-name/workflows') as the workflowsPath

This leaves out bundling but in my git repo I’ll add the whole 9 yards :smile:

Breaking up workflows as library is not necessary, you can put everything in the worker app if you want to. I just feel it creates a healthy separation, and allows you to test workflows separately, while also leveraging Nx’s task caching capabilities.

12

Thanks so much @bnpnt - this has already helped. I look forward to your example repo. I also need to go back and understand what was different about this setup from my original one which couldn’t resolve the workflowsPath.

13

Here it is! Let me know if you have any questions. I also check slack way more often!

2 Likes