I want to download pdf with nodeJS, so I created a function with node-downloader-helper module (see below). This function is called multiples time to download a list of pdf.
function downloadPDF(url, dirname, index) {
const options = {
method: "GET", // Request Method Verb
headers: {}, // Custom HTTP Header ex: Authorization, User-Agent
fileName: `document${index}.pdf`, // Custom filename when saved
forceResume: false, // If the server does not return the "accept-ranges" header, can be force if it does support it
removeOnStop: true, // remove the file when is stopped (default:true)
removeOnFail: true, // remove the file when fail (default:true)
httpRequestOptions: {}, // Override the http request options
httpsRequestOptions: {
timeout: 10000, // Timeout in milliseconds
}, // Override the https request options, ex: to add SSL Certs
timeout: 10000, // Timeout in ms, 0 to disable (default:0)
};
const dl = new DownloaderHelper(encodeURI(url), dirname, options);
dl.on('end', () => console.log('Download Completed'));
dl.on('error', (err) => console.log('Download Failed', err));
dl.start().catch(err => console.error(err));
}
When I run my program I get this error : (and sometimes with server completely crash). I think it’s about a timeout error but I tried different methods but none worked.
Error: Response status was 408
at ClientRequest.<anonymous> (/home/ubuntu/CPE-Colle/node_modules/node-downloader-helper/dist/index.js:1:7055)
at Object.onceWrapper (node:events:642:26)
at ClientRequest.emit (node:events:527:28)
at HTTPParser.parserOnIncomingClient [as onIncoming] (node:_http_client:631:27)
at HTTPParser.parserOnHeadersComplete (node:_http_common:117:17)
at TLSSocket.socketOnData (node:_http_client:494:22)
at TLSSocket.emit (node:events:527:28)
at addChunk (node:internal/streams/readable:324:12)
at readableAddChunk (node:internal/streams/readable:297:9)
at Readable.push (node:internal/streams/readable:234:10) {
status: 408,
body: ''
}
I think found the solution, my puppeteer browser that was fetching pdf links wasn't closing so after a while my server crashed.