When running builds on our TeamCity agent (centos 7), any npm command (version 7.24.2) run with the teamcity user generates the following error :
[Step 2/25] npm cache clear --force
[npm cache clear --force] npm WARN using --force Recommended protections disabled.
[npm cache clear --force] npm ERR! code EACCES
[npm cache clear --force] npm ERR! syscall unlink
[npm cache clear --force] npm ERR! path /data/teamcity-agent/npm-cache/_cacache/index-v5/34/70188cc2e95f746adf32e21191fb2cad2a56f8c7c78dcf4a0d1587143321
[npm cache clear --force] npm ERR! errno -13
[npm cache clear --force] npm ERR!
[npm cache clear --force] npm ERR! Your cache folder contains root-owned files, due to a bug in
[npm cache clear --force] npm ERR! previous versions of npm which has since been addressed.
[npm cache clear --force] npm ERR!
[npm cache clear --force] npm ERR! To permanently fix this problem, please run:
[npm cache clear --force] npm ERR! sudo chown -R 2158:993 "/data/teamcity-agent/npm-cache"
[npm cache clear --force]
[npm cache clear --force] npm ERR! A complete log of this run can be found in:
[npm cache clear --force] npm ERR! /data/teamcity-agent/npm-cache/_logs/2022-05-03T10_51_17_066Z-debug.log
[Step 2/25] Process exited with code 243
[Step 2/25] Process exited with code 243
[Step 2/25] Step npm cache clear (Node.js NPM) failed
But when digging into the error, it's not true that the user teamcity does not own all the files in the npm cache, it does, all files are owned by the correct user. It's that some of the directories don't have the execute bit set. For example:
> cd /data/teamcity-agent/npm-cache
> ls -alR | grep drw-
drw-------. 2 teamcity teamcity 138 Apr 29 01:52 34
drw-------. 2 teamcity teamcity 138 Apr 29 09:54 cc
You can see out of the hundreds of other directories in the cache two have only got the owners read and write bits set, all the others look like this:
> cd /data/teamcity-agent/npm-cache/_cacache/content-v2/sha512
> ls -al
drwxr-xr-x. 257 teamcity teamcity 8192 Apr 28 12:10 .
drwxr-xr-x. 4 teamcity teamcity 32 Apr 28 12:10 ..
drwxr-xr-x. 7 teamcity teamcity 56 Apr 29 01:52 00
drwxr-xr-x. 11 teamcity teamcity 96 Apr 28 12:10 01
.....
If the owners execute flag is not set on the directories then the teamcity agent can't modify the contents of the directory and hence all the npm commands fail. The quick fix for the issue is to run:
> chmod -R +755 /data/teamcity-agent/npm-cache
But after time the issue comes back in a different area of the npm cache. This leads to the actual question here:
What is causing some directories in the npm cache to get created without the execute flag set?
I have checked that there is only one version of npm installed on the agent. Incidentally, the TeamCity agent is running on centos 7 (CentOS Linux release 7.9.2009 (Core))
We never did get to the bottom of what was causing this issue but we have implemented a fix that has stopped it from happening.
In the end we moved the npm cache from /data/teamcity-agent/npm-cache to /tmp/teamcity-agent/npm-cache via the file at /usr/etc/npmrc which is now set to the following:
$ cat /usr/etc/npmrc
registry=https://npm-private-registry.aaa.bbb/
always-auth=true
_auth="................"
email=aaa@bbb.ccc
cache=/tmp/teamcity-agent/npm-cache
When you list the npm config looks like this:
$ npm config list
; "global" config from /usr/etc/npmrc
_auth = (protected)
always-auth = true
cache = "/tmp/teamcity-agent/npm-cache"
; email = "aaa@bbb.ccc" ; overridden by user
registry = "https://npm-private-registry.aaa.bbb/"
; "user" config from /home/centos/.npmrc
...
...
The only difference between the two locations is that /data/teamcity-agent is owned by the teamcity user and /tmp/teamcity-agent is owned by root.