How to setup a bazel workspace with `rules_js` for a monorepo with multiple packages?

I want to setup the bazel build system to build a monorepo with multiple JavaScript packages with the new rules_js rule set. The documentation of rules_js says that it supports "nested npm packages in a monorepo" via "workspaces", but I don't understand how to wire everything up so that:

each package in the monorepo can have different external dependencies
packages in the monorepo can also depend on each other

I tried using @npm//$DEPENDENCY in the deps of rules like js_library to refer to my npm dependencies (like with the older rules_nodejs ruleset), but I just get errors about non-existing targets.

Solution

TL;DR

Set up a pnpm workspace and use npm_link_all_packages multiple times to set up separate node_modules folders in your bin tree. Then refer to the dependencies in these node_modules folders by :node_modules/$DEPENDENCY in your deps.

How does rules_js create node_modules folders?

rules_js uses the npm_link_all_packages rule to set up node_modules folders based on a pnpm-lock.yaml lockfile. This is similar to the pnpm install command of the pnpm package manager. The difference is that pnpm install creates the node_modules folders in the source tree and npm_link_all_packages creates them in the bin tree.

Since you want that each package can have different external dependencies, they each need to have their own node_modules. pnpm supports a single lock file for multiple packages if you set up a pnpm workspace. The resulting lock file can be used by rules_js to set up a Bazel workspace with multiple packages that each get their own node_modules folder in the bin tree. The npm_link_all_packages rule will automatically set up the correct node_modules folder based on the Bazel package name and the pnpm-lock.yaml.

So you have to set up a pnpm workspace and use npm_link_all_packages multiple times to set up separate node_modules folders in your bin tree.

How exactly do I set this up?

To achieve this, you can put the following pieces in your workspace root:

pnpm-workspace.yaml to configure the list of pnpm workspace packages
pnpm-lock.yaml to record how all all direct and indirect npm dependencies are resolved
WORKSPACE.bazel to setup rules_js
package.json to avoid confusing pnpm, and in case you want global scripts or settings for your IDE or similar
BUILD.bazel to create the virtual store with npm_link_all_packages

And in each pnpm workspace package $PACKAGE you put:

$PACKAGE/package.json to configure the direct npm dependencies of $PACKAGE
$PACKAGE/BUILD.bazel to create the node_modules folder in the bin tree with npm_link_all_packages, and for your js_library or similar

So every pnpm workspace package is also a bazel package (since it has a BUILD.bazel).

How can I refer to my npm dependencies in `deps`?

In the deps of js_library and similar, you can point to the dependencies in the node_modules folders generated by npm_link_all_packages. Typically, the js_library is in the same Bazel package as the npm_link_all_package, so you can just use :node_modules/$DEPENDENCY. If the js_library is in a sub-package of $PACKAGE, you can use //$PACKAGE:node_modules/$DEPENDENCY instead. Then in your JavaScript files, you can import from "$DEPENDENCY" or require("$DEPENDENCY"). This will work out at runtime because the node_modules folder will be at an appropriate place in the runfiles for node to find it.

How can the packages in my workspace refer to each other?

If you want one of your packages $PACKAGE to depend on another one of your packages $OTHER, you put a "$OTHER": "workspace:*" into the $PACKAGE/package.json as usual with pnpm. You also have to make sure that the default target of $OTHER (for example, //some/path/to/other:other) is an npm_package rule. Then you can use :node_modules/$OTHER or //$PACKAGE:node_modules/$OTHER in the deps of a js_library or similar to refer to $OTHER just like you would refer to an npm dependency.

How can I create the `pnpm-lock.yaml`?

Note that this is a working pnpm setup with some additional BUILD.bazel files, so you can:

use pnpm install --lockfile-only to create the pnpm-lock.yaml
use pnpm install to create the pnpm-lock.yaml and also create node_modules folders in the source tree. These folders are not needed for rules_js but can be helpful for your IDE.