vinyl-fs (gulp file stream) not finishing .dest write before firing callback or promise

Question

Frustratingly trying to get a vinyl-fs (gulpfs) stream to finish writing ALL the .dest files before launching code to upload a copy of the destination folder to an S3 bucket. That copy operation keeps missing a directory that the stream has yet to write.

The issue is how vinly-fs plays with callbacks/promises (does it?).

I've put together some testing code to experiment but when that code is run the promise is resolved immediately (after the writes start?) even though I FORCED the stream to wait for 5 seconds after the .dest command before firing the promise resolve. My resolve function currently fires and then 5 seconds later I'm back to my prompt. Totally stuck on this. I don't want to live without vinyl-fs but it just doesn't seem to play well no matter what I try and I need this to work as a sequence (write ALL files out before uploading them).

Below is my testing code. I set this up so you can easily put dump the code into a file, set some source directory and try for yourself. I looked at gulpjs issues and on stackoverflow and some others are having the same problem but I have seen no definitive answer/solution.

test.js

var source = './builds/**/*.*';

var p = foo(source);

p.then( bar, oopsBar );

//
// vinyl-fs based module task
//
function foo(src) {

  var fs = require('vinyl-fs');
  var wait = require('gulp-wait')

  done = new Promise( function(resolve,reject){
      fs.src(src)
        .pipe(fs.dest('./temp'))
        .pipe(wait(5000))
        .on('finish',resolve());

    });
    // return a promise
    return done;
}

function bar() {
    console.log('Now uploading to S3 bucket');
}

function oopsBar() {
    console.log('Something went wrong');
}

BTW you might be wondering why I just don't use gulp.task and return the promise and fire my upload as a separate gulp task. I currently minimize my gulp.tasks to cli started tasks. Eventually like running this code just use wow "node test" instead and remove the global gulp dependency and gulpfile.js all together. Vinyl-fs is a great module!, but I've decided it's best not to buy into Gulp/Grunt/Brunch/Jake thing.

Answer 1

Seems like you wanted

.on('finish', resolve)

instead of calling resolve() right away. Also, readable streams don't emit a finish event, they emit an end event, so you 'd have to use

.on('end', resolve)