Search code examples
javascriptnode.jspnpm

Is this an inconsistency in the pnpm docs?

Background

Reading the pnpm docs, we find the following paragraph in the Node-Modules configuration options with pnpm article:

By default, pnpm v5 will create a semi-strict node_modules. Semi-strict means that your application will only be able to require packages that are added as dependencies to package.json (with a few exceptions). However, your dependencies will be able to access any packages.

The default configuration looks like this:

All packages are hoisted to node_modules/.pnpm/node_modules: hoist-pattern[]=*

All types are hoisted to the root in order to make TypeScript happy: public-hoist-pattern[]=*types*

All ESLint-related packages are hoisted to the root as well: public-hoist-pattern[]=*eslint*

However, in all other articles I have read, including Symlinked node_modules structure, it is stated that the folder structure is as follows:

enter image description here

In other words, there is no node_modules folder directly below node_modules/.pnpm, as stated in the first article above (see second bold section).

Question

  1. Is that an error in the docs? If so, which of the two is correct?
  2. If there is no node_modules below .pnpm, how can it be true that indirect dependencies can access any module in the virtual store (node_modules/.pnpm)? For example, in the above image, I do not see how qar can access bar if qar tries to import bar without listing it in its dependencies. node will first search the innermost node_modules, then the next one up is the outermost node_modules, which only contains the app's direct dependencies.
Solution

  • The reason all dependencies can access all other packages by default in pnpm, is that pnpm places a node_modules inside the virtual store, which symlinks to all the packages in the virtual store:

    node_modules/.pnpm/node_modules/<symlinksToAllPackagesInVirtualStore>

    In other words, the image in the question above is missing a node_modules folder.

    It is hoist=true/false that controls whether or not to create that folder. If hoist=false, dependencies can only access their own direct dependencies (similar to the default for the end user application).

    LINKS

    COMMENTS

    The concepts linked above could have used better explanations in the docs.

    First of all, hoisting means "moving" a package, either by moving the actual files or by creating a symlink to them.

    Secondly, there are two types of hoisting done in pnpm.

    The first type, controlled by either public-hoist-pattern, shamefully-hoist, or node-linker, refers to whether packages are moved up to the root node_modules. node-linker: hoisted will move all packages up, and thus create a flat node_modules structure, similar to what npm and yarn does. shamefully-hoist is seemingly similar, except it symlinks those packages in the root node_modules to the virtual store, instead of actually moving them. At least I think that is what it does. The documentation is lacking here.

    The second type of hoisting, is hoisting into a node_modules subfolder of the virtual store (node_modules/.pnpm/node_modules). In this case, no packages are moved, they still stay in the virtual store (node_modules/.pnpm), but symlinks to them are placed in that node_modules subfolder. This is what enables all dependencies to access all other packages installed.

    As mentioned, pnpm only allows such "unlisted" access by default for dependencies of the end user application. The end-user application itself can only access its direct dependencies. That is one of the benefits of pnpm (in npm and yarn, every package can access every other package, due to the flat node_modules they create). It is pnpm's folder structure and symlinking that enables this behavior. Read the blog articles for more insight.

    NOTE

    If a pnpm maintainer sees this, I suggest updating the docs with a proper folder structure image. It is not easily understood, the way it now stands.