I am trying to dockerize a node.js application (called "Hub") on a docker for windows with Linux-based containers, but when building/running the container cannot access a dependency. The application is part of a microservice-architecture, where I have several local common-modules that are shared between the various services. The Hub-service does not use any volumes.
In this particular case, the affected service makes use of an authentication-module and a helper-module. The authentication-module includes the helper-module as well. This seems to be the issue.
The error I receive is:
hub-1 | Error: Cannot find module 'helper'
hub-1 | Require stack:
hub-1 | - /src/apps/Common/Authentication/index.js
hub-1 | - /src/apps/Hub/WS/eventhandler.js
hub-1 | - /src/apps/Hub/WS/initws.js
hub-1 | - /src/apps/Hub/index.js
hub-1 | requireStack: [
hub-1 | '/src/apps/Common/Authentication/index.js',
hub-1 | '/src/apps/Hub/WS/eventhandler.js',
hub-1 | '/src/apps/Hub/WS/initws.js',
hub-1 | '/src/apps/Hub/index.js'
hub-1 | ]
hub-1 | }
The folder structure is following the suggestion from: Setting up docker nodejs application with local npm dependencies
It looks like this:
Main Folder
|- <Dockerfiles for each service>-> "Dockerfile_Hub"
|- compose.yaml (Docker-compose)
|-apps
|-Hub
|- ...several other services
|-Common
|-Authentication
|-Helper
|- ... several other common modules
In the dockerfile I copy ALL commonly used modules to the container and then run the npm install command. I have another service, where this works flawlessly, it seems to have something to do with file paths in interdepedencies between common-modules (Authentication-module has a dependency on the helper-module as well).
Package.json-files:
{
"name": "hub",
"version": "1.0.0",
"description": "Hub Service",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "ISC",
"dependencies": {
"authentication": "file:../Common/Authentication",
"axios": "^1.6.8",
"config": "file:../Common/Config",
"crypto": "^1.0.1",
"express": "^4.19.1",
"express-session": "^1.18.0",
"fs": "^0.0.1-security",
"helper": "file:../Common/Helper",
"http": "^0.0.1-security",
"jsonwebtoken": "^9.0.2",
"keycloak-connect": "^24.0.2",
"logger": "file:../Common/Logger",
"node-jose": "^2.2.0",
"socket.io": "^4.7.5",
"swagger-jsdoc": "^6.2.8",
"swagger-ui-express": "^5.0.0"
}
}
{
"name": "authentication",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "ISC",
"dependencies": {
"config": "file:../Config",
"helper": "file:../Helper",
"jsonwebtoken": "^9.0.2",
"jswt": "^1.4.4",
"logger": "file:../Logger"
}
}
Docker files (as suggested in the mentioned article):
ARG NODE_VERSION=20.12.0
FROM node:${NODE_VERSION}-alpine
ENV NODE_ENV development
WORKDIR /usr/src/app
RUN mkdir -p /src
COPY ./apps/Common /src/apps/Common
COPY ./apps/Hub /src/apps/Hub
# Download dependencies as a separate step to take advantage of Docker's caching.
# Leverage a cache mount to /root/.npm to speed up subsequent builds.
# Leverage a bind mounts to package.json and package-lock.json to avoid having to copy them into
# into this layer.
RUN --mount=type=bind,source=/apps/Hub/package.json,target=package.json \
--mount=type=bind,source=/apps/Hub/package-lock.json,target=package-lock.json \
--mount=type=cache,target=/root/.npm \
npm ci --omit=dev
RUN cd /src/apps/Hub && npm install
# Run the application as a non-root user.
USER node
# Expose the port that the application listens on.
EXPOSE 8000
# Run the application.
CMD node /src/apps/Hub/index.js
Docker-compose:
version: "3"
#Hub
services:
hub:
build:
context: .
dockerfile: Dockerfile_Hub
environment:
NODE_ENV: development
APP_PATH: /src/apps/Hub
PINO_LOG_LEVEL: debug
ports:
- 3000:3000
The build-process runs without any errors:
[+] Building 0.0s (0/0) docker:default
[+] Building 0.0s (0/0) docker:defaultr reading preface from client //./pipe/docker_engine: file has already been closed
[+] Building 3.2s (15/15) FINISHED
=> [hub internal] load build definition from Dockerfile_Hub
=> => transferring dockerfile: 1.38kB
=> [hub] resolve image config for docker.io/docker/dockerfile:1
=> CACHED [hub] docker-image://docker.io/docker/dockerfile:1@sha256:a57df69d0ea827fb7266491f2813635de6f17269be881f696fbfdf2d83dda33e
=> [hub internal] load metadata for docker.io/library/node:20.12.0-alpine
=> [hub internal] load .dockerignore
=> => transferring context: 2B
=> [hub stage-0 1/8] FROM docker.io/library/node:20.12.0-alpine@sha256:ef3f47741e161900ddd07addcaca7e76534a9205e4cd73b2ed091ba339004a75
=> [hub internal] load build context
=> => transferring context: 478.86kB
=> CACHED [hub stage-0 2/8] WORKDIR /usr/src/app
=> CACHED [hub stage-0 3/8] RUN mkdir -p /src
=> CACHED [hub stage-0 4/8] COPY ./apps/Common /src/apps/Common
=> CACHED [hub stage-0 5/8] COPY ./apps/Hub /src/apps/Hub
=> CACHED [hub stage-0 6/8] RUN --mount=type=bind,source=/apps/Hub/package.json,target=package.json --mount=type=bind,source=/apps/Hub/package-
=> CACHED [hub stage-0 7/8] RUN cd /src/apps/Hub && npm install
=> [hub stage-0 8/8] WORKDIR /src/apps/Hub
=> [hub] exporting to image
=> => exporting layers
=> => writing image sha256:4f2ce3b43d06485ec72da1c107df361e6b0a11d44785beea3baff85ff60170a3
=> => naming to docker.io/library/finslice-hub
[+] Running 2/2
✔ Container crazy_rosalind Removed
✔ Container finslice-hub-1 Recreated
Attaching to hub-1
hub-1 | node:internal/modules/cjs/loader:1146
hub-1 | throw err;
hub-1 | ^
hub-1 |
hub-1 | Error: Cannot find module 'helper'
hub-1 | Require stack:
hub-1 | - /src/apps/Common/Authentication/index.js
hub-1 | - /src/apps/Hub/WS/eventhandler.js
hub-1 | - /src/apps/Hub/WS/initws.js
hub-1 | - /src/apps/Hub/index.js
hub-1 | requireStack: [
hub-1 | '/src/apps/Common/Authentication/index.js',
hub-1 | '/src/apps/Hub/WS/eventhandler.js',
hub-1 | '/src/apps/Hub/WS/initws.js',
hub-1 | '/src/apps/Hub/index.js'
hub-1 | ]
hub-1 | }
hub-1 |
hub-1 | Node.js v20.12.0
hub-1 exited with code 1
If I access the containers file system I see that all files/folders are properly copied.
It has probably something to do with the way I set the WORKDIR / CD-commands in the docker-file. I tried changing this to (last 2 lines of dockerfile):
WORKDIR /src/apps/Hub CMD node index.js
with the same result.
Running the application just via node.js outside of docker works without any issues.
I have finally solved this. After trying out Raeisi's tipp and moving to .dockerignore files instead of only selectively copying the files I took a closer look at the file system in a copy of the container.
What happens is: npm install for internal depedencies (i.e. npm install ../Common/Authentication etc.) only creates symlinks to the folder, but does not actually copies the module into the folder. Would probably work, if I worked on a Linux-machine, but on a windows-machine the symlinks refer to a windows-style directory (i.e. d:\myApp\myModule), which of course will not work in a linux based container.
What solved the issue is this article:
Install npm local module directory without symlinks?
simple solution: "use npm install --install-links ../module" instead of "npm install ../module". This will copy the entire source as if it was an external package.
