npmyamlcontinuous-integrationazure-pipelines
Azure Pipeline caching missing despite identical fingerprints
I'm running a nodejs server that is built & deployed by Azure pipelines and Azure releases.
I'm wanting to cache the node_modules folder because it's become quite large and project build time has become quite long.
According the pipeline run logs the caches were originally hitting, but now the caches are missing. This is despite the fact that the cache fingerprint that is saved in one run is the exact same fingerprint that is used in the following pipeline run. Examples below:
First run:
2024-01-24T20:53:03.5987539Z ##[section]Starting: Retrieve Cached/Cache node_modules folder
2024-01-24T20:53:03.6098219Z ==============================================================================
2024-01-24T20:53:03.6098361Z Task : Cache
2024-01-24T20:53:03.6098415Z Description : Cache files between runs
2024-01-24T20:53:03.6098484Z Version : 2.198.0
2024-01-24T20:53:03.6098538Z Author : Microsoft Corporation
2024-01-24T20:53:03.6098616Z Help : https://aka.ms/pipeline-caching-docs
2024-01-24T20:53:03.6098703Z ==============================================================================
2024-01-24T20:53:03.9263162Z Resolving key:
2024-01-24T20:53:03.9423821Z - npm [string]
2024-01-24T20:53:03.9428589Z - "Windows_NT" [string]
2024-01-24T20:53:03.9802688Z - server/package.json [file] --> 690F2B6CC31DC2C79B3B5126ADDAC15623C79390833318FF8474C21513E5DC7C
2024-01-24T20:53:03.9862457Z Resolved to: npm|"Windows_NT"|2/prfzR8J2uSc/k530A+6uo4rEq8s19zTSw0YbIifCI=
2024-01-24T20:53:04.0296527Z Using default max parallelism.
2024-01-24T20:53:04.5141679Z Using default max parallelism.
2024-01-24T20:53:04.5150994Z Max dedup parallelism: 192
2024-01-24T20:53:04.5151690Z DomainId: 0
2024-01-24T20:53:04.6243522Z ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session 0d07dd69-dd14-4b2e-8f1e-2ef4b5bb7256
2024-01-24T20:53:04.6267589Z Hashtype: Dedup64K
2024-01-24T20:53:04.6498404Z Getting a pipeline cache artifact with one of the following fingerprints:
2024-01-24T20:53:04.6498978Z Fingerprint: `npm|"Windows_NT"|2/prfzR8J2uSc/k530A+6uo4rEq8s19zTSw0YbIifCI=`
2024-01-24T20:53:04.9307178Z There is a cache miss.
2024-01-24T20:53:08.6481764Z Process exit code: 0
2024-01-24T20:53:08.6635727Z DedupManifestArtifactClient will correlate http requests with X-TFS-Session _______-____-________-____
2024-01-24T20:53:10.6908307Z 1 files processed.
2024-01-24T20:53:10.6917079Z Processed 1 files from C:\Users\VssAdministrator\AppData\Local\Temp successfully.
2024-01-24T20:53:13.6997885Z Uploading 1 files from directory C:\Users\VssAdministrator\AppData\Local\Temp.
2024-01-24T20:53:13.6998684Z Uploaded 57,636,821 out of 296,608,768 bytes.
2024-01-24T20:53:18.7158705Z Uploading 1 files from directory C:\Users\VssAdministrator\AppData\Local\Temp.
2024-01-24T20:53:18.7159362Z Uploaded 151,437,141 out of 296,608,768 bytes.
2024-01-24T20:53:19.5412305Z Uploaded 175,004,692 out of 296,608,985 bytes
2024-01-24T20:53:19.5417492Z Content upload is done!
2024-01-24T20:53:19.5421190Z
2024-01-24T20:53:19.5444177Z Content upload statistics:
2024-01-24T20:53:19.5445329Z Total Content: 593.2 MB
2024-01-24T20:53:19.5449168Z Physical Content Uploaded: 48.6 MB
2024-01-24T20:53:19.5449484Z Logical Content Uploaded: 175.0 MB
2024-01-24T20:53:19.5449721Z Compression Saved: 126.4 MB
2024-01-24T20:53:19.5449967Z Deduplication Saved: 418.2 MB
2024-01-24T20:53:19.5450193Z Number of Chunks Uploaded: 1,947
2024-01-24T20:53:19.5450421Z Total Number of Chunks: 13,366
2024-01-24T20:53:19.5450583Z
2024-01-24T20:53:19.5450872Z Creating a pipeline cache artifact with the following fingerprint: `npm|"Windows_NT"|2/prfzR8J2uSc/k530A+6uo4rEq8s19zTSw0YbIifCI=`
2024-01-24T20:53:19.5451180Z Cache item created.
2024-01-24T20:53:19.5451491Z Saved under the following scope: 26;_______-____-________-____;refs/tags/Release_v3.6.4;_______-____-________-____
2024-01-24T20:53:19.6026897Z Saved item.
2024-01-24T20:53:19.6027169Z ApplicationInsightsTelemetrySender correlated 4 events with X-TFS-Session _______-____-________-____
2024-01-24T20:53:19.6028744Z ##[section]Finishing: Retrieve Cached/Cache node_modules folder
Second run:
2024-01-24T20:53:03.5987539Z ##[section]Starting: Retrieve Cached/Cache node_modules folder
2024-01-24T20:53:03.6098219Z ==============================================================================
2024-01-24T20:53:03.6098361Z Task : Cache
2024-01-24T20:53:03.6098415Z Description : Cache files between runs
2024-01-24T20:53:03.6098484Z Version : 2.198.0
2024-01-24T20:53:03.6098538Z Author : Microsoft Corporation
2024-01-24T20:53:03.6098616Z Help : https://aka.ms/pipeline-caching-docs
2024-01-24T20:53:03.6098703Z ==============================================================================
2024-01-24T20:53:03.9263162Z Resolving key:
2024-01-24T20:53:03.9423821Z - npm [string]
2024-01-24T20:53:03.9428589Z - "Windows_NT" [string]
2024-01-24T20:53:03.9802688Z - server/package.json [file] --> 690F2B6CC31DC2C79B3B5126ADDAC15623C79390833318FF8474C21513E5DC7C
2024-01-24T20:53:03.9862457Z Resolved to: npm|"Windows_NT"|2/prfzR8J2uSc/k530A+6uo4rEq8s19zTSw0YbIifCI=
2024-01-24T20:53:04.0296527Z Using default max parallelism.
2024-01-24T20:53:04.5141679Z Using default max parallelism.
2024-01-24T20:53:04.5150994Z Max dedup parallelism: 192
2024-01-24T20:53:04.5151690Z DomainId: 0
2024-01-24T20:53:04.6243522Z ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session 0d07dd69-dd14-4b2e-8f1e-2ef4b5bb7256
2024-01-24T20:53:04.6267589Z Hashtype: Dedup64K
2024-01-24T20:53:04.6498404Z Getting a pipeline cache artifact with one of the following fingerprints:
2024-01-24T20:53:04.6498978Z Fingerprint: `npm|"Windows_NT"|2/prfzR8J2uSc/k530A+6uo4rEq8s19zTSw0YbIifCI=`
2024-01-24T20:53:04.9307178Z There is a cache miss.
2024-01-24T20:53:08.6481764Z Process exit code: 0
2024-01-24T20:53:08.6635727Z DedupManifestArtifactClient will correlate http requests with X-TFS-Session _______-____-________-____
2024-01-24T20:53:10.6908307Z 1 files processed.
2024-01-24T20:53:10.6917079Z Processed 1 files from C:\Users\VssAdministrator\AppData\Local\Temp successfully.
2024-01-24T20:53:13.6997885Z Uploading 1 files from directory C:\Users\VssAdministrator\AppData\Local\Temp.
2024-01-24T20:53:13.6998684Z Uploaded 57,636,821 out of 296,608,768 bytes.
2024-01-24T20:53:18.7158705Z Uploading 1 files from directory C:\Users\VssAdministrator\AppData\Local\Temp.
2024-01-24T20:53:18.7159362Z Uploaded 151,437,141 out of 296,608,768 bytes.
2024-01-24T20:53:19.5412305Z Uploaded 175,004,692 out of 296,608,985 bytes
2024-01-24T20:53:19.5417492Z Content upload is done!
2024-01-24T20:53:19.5421190Z
2024-01-24T20:53:19.5444177Z Content upload statistics:
2024-01-24T20:53:19.5445329Z Total Content: 593.2 MB
2024-01-24T20:53:19.5449168Z Physical Content Uploaded: 48.6 MB
2024-01-24T20:53:19.5449484Z Logical Content Uploaded: 175.0 MB
2024-01-24T20:53:19.5449721Z Compression Saved: 126.4 MB
2024-01-24T20:53:19.5449967Z Deduplication Saved: 418.2 MB
2024-01-24T20:53:19.5450193Z Number of Chunks Uploaded: 1,947
2024-01-24T20:53:19.5450421Z Total Number of Chunks: 13,366
2024-01-24T20:53:19.5450583Z
2024-01-24T20:53:19.5450872Z Creating a pipeline cache artifact with the following fingerprint: `npm|"Windows_NT"|2/prfzR8J2uSc/k530A+6uo4rEq8s19zTSw0YbIifCI=`
2024-01-24T20:53:19.5451180Z Cache item created.
2024-01-24T20:53:19.5451491Z Saved under the following scope: 26;_______-____-________-____;refs/tags/Release_v3.6.5;_______-____-________-____
2024-01-24T20:53:19.6026897Z Saved item.
2024-01-24T20:53:19.6027169Z ApplicationInsightsTelemetrySender correlated 4 events with X-TFS-Session 0d07dd69-dd14-4b2e-8f1e-2ef4b5bb7256
2024-01-24T20:53:19.6028744Z ##[section]Finishing: Retrieve Cached/Cache node_modules folder
To me these logs suggest that the first run saved the cached item under npm|"Windows_NT"|2/prfzR8J2uSc/k530A+6uo4rEq8s19zTSw0YbIifCI=, and the second run attempted to find the cached item under the fingerprint
npm|"Windows_NT"|2/prfzR8J2uSc/k530A+6uo4rEq8s19zTSw0YbIifCI=, but was unable to do so for some reason?
My yml code for the cache activity and npm install activity is as follows:
- task: Cache@2
inputs:
key: 'npm | "$(Agent.OS)" | server/package.json'
path: 'server/node_modules'
cacheHitVar: 'prodServerNPMCache'
displayName: Retrieve Cached/Cache node_modules folder
- task: Npm@1
displayName: Server - npm install
inputs:
workingDir: server
verbose: true
Any help or ideas would be appreciated, thank you!
Solution
Check the Pipeline log of the two Pipeline Runs. The Package.json files hash value are the same:690F2B6CC31DC2C79B3B5126ADDAC15623C79390833318FF8474C21513E5DC7C and the Fingerprint has the same value.
This means that the content of the package.json file has not changed during the two Pipeline runs.
Refer to this doc: Cache isolation and security
To ensure isolation between caches from different pipelines and different branches, every cache belongs to a logical container called a scope. Scopes provide a security boundary that ensures a job from one pipeline cannot access the caches from a different pipeline.
In this case, when two Pipeline runs are running in two different Pipelines, the cache will be re-created in the second Run with the same Fingerprint.
You can check if the two pipeline runs are running in the different pipeline definition.
On the other hand, from the Pipeline log timestamp, the two pipeline Runs are running at the same time.
You need to make sure that the first Pipeline run completed the Post-job: Cache task(in this case, the Npm cache will create completely), then you can run the second Pipeline run to read the Pipeline Cache.
Test to run the Pipeline in the same definition, it can work as expected.
For example:
- I want to install the "n" package and I get an error
- n <version> command does not activate specified version
- Change n install location
- How to install a specific version of Node on Ubuntu/Debian?
- Different node version for different projects, is there a way of telling node which version to use?
- Install Node.js to install n to install Node.js?
- How to select the latest node.js v6 version using n?
- n-install: ERROR: GNU Make not found, which is required for operation
- How to downgrade Node version with n
- how switch to previous version in n (Node version manager)?
- Automatically use the right version of Node for a package
- internal/modules/cjs/loader.js:905 -> throw err;
- Why doesn't "n" downgrade my node version on a Mac?
- Node version manager
- n failed to install/switch node in Linux?
- vue command not found on Mac
- How to uninstall n and all node versions installed by n
- Angular CLI on HTTPS - can't install CI as root
- n (node version manager): cannot create directory
- npm module n emits errors
- How to update npm permanently?
- Cannot change nodejs version using n
- upgrade nodejs to stable version
- How should I install and use multiple versions of Node on the same production machine?