Some issue in node.js fs.createWriteStream

Question

This code seems behave Ok, but when I try read the file (someFile.json) in the last line simply does not work. :-( Please some clue. Although I know the issue is complex.

var fs = require('fs');

var request = require("request")


var dataFile = 'data/someFile.json'
var url = "https://api.someUrl"
var file = fs.createWriteStream(dataFile);
request(url).pipe(file);

file.on('finish',function(){
    console.log('file download to ',dataFile)
    file.destroy();
    //file.closeSync;
    file.on('close', function(){ console.log('File Closed ')})
})


var datos = fs.readFileSync(dataFile, 'utf8');

Answer 1

You are trying to read the file BEFORE it has been written. The writeStream you are writing to is asynchronous. That means it completes some indeterminate time in the future. Meanwhile, the rest of your code continues to run and thus you try to read it before the file has been finished.

In the code you have, here's the sequence of events.

1. Create writeStream
2. Initiate http request()
3. Hook future data arriving from the request() to your stream with .pipe().
4. Register finish event handler on the stream.
5. Call fs.readFileSync() to read the file
6. Data arrives from request() and is sent to the write stream where it may be buffered.
7. More data arrives from request() and is sent to the write stream where it may be buffered.
8. Request finishes, telling the writeStream to flush its data to disk and close itself.
9. The finish event occurs.
10. You register a close event handler.
11. The close event occurs.

---

You can likely use stream notifications to do what you want, but to just test out the file, you can put the fs.readFileSync() in the completion handle for the close event on the write stream.

const fs = require('fs');
const request = require("request")

const dataFile = 'data/someFile.json'
const url = "https://api.someUrl"
const file = fs.createWriteStream(dataFile);
request(url).pipe(file);

file.on('finish',function(){
    console.log('file download to ',dataFile)
}).on('close', function(){ 
   console.log('File Closed ');
   // file is available for reading now
   var datos = fs.readFileSync(dataFile, 'utf8');    
   console.log(datos);
});

Also, you do not need to .destroy() the stream. By default, it will be set to autoClose which will close automatically when the stream you piped to it is done. And, you do not want your .on('close', ...) event handler inside your finish error handler.

In this new suggested code, here's the sequence of events:

1. Create writeStream
2. Initiate http request()
3. Hook future data arriving from the request() to your stream with .pipe().
4. Register finish event handler on the stream.
5. Register close event handler.
6. Data arrives from request() and is sent to the write stream where it may be buffered.
7. More data arrives from request() and is sent to the write stream where it may be buffered.
8. Request finishes, telling the writeStream to flush its data to disk and close itself.
9. The finish event occurs.
10. The close event occurs.
11. Call fs.readFileSync() to read the file