I'm supporting a build process that uses gulp and trying to troubleshoot a problem. To troubleshoot this, I'd like to know if the code that makes multiple calls to gulp.src().pipe().dest() in a single gulp task is legal and correctly written.
More exactly, I want to know if gulp.src().pipe().dest() completes prior to the next javascript line that makes another call to gulp.src().pipe().dest().
The code would look like this:
gulp.src(getSrcFiles())
.pipe(dostuff)
.pipe(dostuff2)
.dest('temp');
// will the above copy complete before I try doing substitutions
gulp.src('temp/**')
.pipe(mySubstitutions)
.dest('temp')
gulp.src('temp/**')
.dest('dist')
The question is will the first pipeline of gulp.src().pipe().pipe().dest() complete prior to the next line which performs some variable substitutions on the files copied in the first pipeline?
I didn't see the answer in the gulp documentation at https://gulpjs.org/API.html#gulp-dest-path-options.
Some other questions I found that did not answer my question were:
Q.js Promises in Gulp Loop (not answered, but similar issue being raised).
gulp run sequence after promise resolve (similar question not answered)
How does gulp treat the Promise of a then()? (similar question, crickets, no answer:-))
More questions but no answers... I have a gut feeling I'm going to answer this one myself after I figure it out, and then I'll be able to answer some of the other questions.
A great question and answer that will help me understand gulp and how it ensures ordering of tasks is What does Gulp "done" method do?. The gulp npm run-sequence docs were also very useful, see https://www.npmjs.com/package/run-sequence.
UPDATE: I found someone else who reported this as a problem.
This is the answer I was looking for:
Will gulp.src().pipe().dest() completes prior to the next javascript line that makes another call to gulp.src().pipe().dest(). No. A stream pipeline is started but not necessarily completed when the next javascript line is run.
Can you use multiple gulp.src().pipe().dest() in a single gulp task? Yes, but you must write the gulp task in a special way to handle this (shown below).
The following code shows how to write a gulp task that makes several gulp.src().dest() copies. The way it works is to convert the gulp.src().pipe() stream into a promise, and then use Promise.all() to wait for all of the streams to complete before saying that the gulp task is complete.
gulp.task('task', function (doneWithTaskCb) {
return Promise.all([
new Promise(function(resolve, reject) {
gulp.src(src + '/*.md')
.pipe(plugin())
.on('error', reject)
.pipe(gulp.dest(dist))
.on('end', resolve)
}),
new Promise(function(resolve, reject) {
gulp.src(src + '/*.md')
.pipe(plugin())
.on('error', reject)
.pipe(gulp.dest(dist))
.on('end', resolve)
})
]).then(function () {
doneWithTaskCb();
});
});
I didn't come up with it. The credit goes to TySound at https://github.com/gulpjs/gulp/issues/1181#issuecomment-126694791.
The important code is this pattern:
new Promise(function(resolve, reject) {
gulp.src()
.pipe()
.pipe(gulp.dest(dist))
.end('end', resolve);
}
That code creates a promise that when it resolves the pipeline is finished. So in essence, it converts the "stream" into a promise.
The code that calls doneWithTaskCb() is important because that is how gulp knows that the current task is complete.
Promise.all( [ new Promise(), new Promise() ] )
.then(function() {
doneWithTaskCb();
});