Search code examples
node.jsfilesystemsfile-descriptor

Node.js - Properly closing fd with multiple streams


I am creating an append/rw file (depending if the download is resumed or not)

this.syncFile = await fs.promises.open(this.syncPath, 'a+');
this.syncFile = await fs.promises.open(this.syncPath, 'r+');

later I create a write stream for that FD

let writeStream = fs.createWriteStream(null, {fd:this.syncFile.fd, autoClose:false, highWaterMark: 1024*1024});

after downloading the remaining data I may or may not create a readStream for the same FD to calculate checksum of full file

let readStream = fs.createReadStream(null, {fd:this.syncFile.fd, autoClose:false, highWaterMark: 1024*1024, start:0});

for finalizing I would like to close the FD.

fs.closeSync(this.syncFile.fd);

I have tried numerous combinations with close() or dispose() on the readStream/writeStream I keep getting uncaught exceptions

[Window Title]
Error
[Content]
Uncaught Exception:
Error: EBADF: Closing file descriptor 5 on garbage collection failed, close

or console warnings about closing file descriptor on gc always two in a row sometimes followed by

Error: EBADF: bad file descriptor, write

Why is this happening? The read/write streams have option autoClose:true Also, before, when I only had a single writeStream some combination that I cannot re-create worked.

What is the correct way of closing all the streams and the FD in this scenario? I cannot wait for the GC to close the file descriptor because it holds an exclusive lock on the file, a file whose attributes I am changing (child_process.execSync('attrib -h +r "' + this.finalPath + '"');) Also the file is unusable by other programs until GC is triggered.

If I only do

delete this.syncFile; I get a beatiful spam of

(node:25180) Warning: Closing file descriptor 10 on garbage collection
(node:25180) Warning: Closing file descriptor 10 on garbage collection
(node:25180) Warning: Closing file descriptor 8 on garbage collection
(node:25180) Warning: Closing file descriptor 8 on garbage collection
(node:25180) Warning: Closing file descriptor 7 on garbage collection
(node:25180) Warning: Closing file descriptor 7 on garbage collection
(node:25180) Warning: Closing file descriptor 9 on garbage collection
(node:25180) Warning: Closing file descriptor 9 on garbage collection

But no exceptions, handled or otherwise, but this is still not good enough because of the issues listed above. My only option seems to be to force a GC collection?


Solution

  • If you're opening the file with fs.promises.open(), then you're getting the newer FileHandle object (not a regular file descriptor) and you will need to call close on that object as in fileHandle.close(), not fs.close(fd).