how to create a readable stream from a remote url in nodejs?

Question

on nodejs documentation, the streams section says I can do fs.createReadStream(url || path). But, when I actually do that It tells me Error: ENOENT: no such file or directory. I just want to pipe the video from a readable to a writable stream, But I'm stuck on creating a readable one.

my code:

const express = require('express')
const fs = require('fs')
const url = 'https://www.example.com/path/to/mp4Video.mp4'
const port = 3000

app.get('/video', (req, res) => {
    const readable = fs.createReadStream(url)
})
app.listen(port, () => {
    console.log('listening on port ' + port)
})

the ERROR:

listening on port 3000
events.js:291
      throw er; // Unhandled 'error' event
      ^

Error: ENOENT: no such file or directory, open 'https://www.example.com/path/to/mp4Video.mp4'
Emitted 'error' event on ReadStream instance at:
    at internal/fs/streams.js:136:12
    at FSReqCallback.oncomplete (fs.js:156:23) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: 'https://www.example.com/path/to/mp4Video.mp4'
}

PS: https://www.example.com/path/to/mp4Video.mp4 IS NOT THE ACTUAL URL

Answer 1

fs.createReadStream() does not work with http URLs only file:// URLs or filename paths. Unfortunately, this is not described in the fs doc, but if you look at the source code for fs.createReadStream() and follow what it calls you can find that it ends up calling fileURULtoPath(url) which will throw if it's not a file: URL.

function fileURLToPath(path) {
  if (typeof path === 'string')
    path = new URL(path);
  else if (!isURLInstance(path))
    throw new ERR_INVALID_ARG_TYPE('path', ['string', 'URL'], path);
  if (path.protocol !== 'file:')
    throw new ERR_INVALID_URL_SCHEME('file');
  return isWindows ? getPathFromURLWin32(path) : getPathFromURLPosix(path);
}

It would suggest using the got() library to get yourself a readstream from a URL:

const got = require('got');
const mp4Url = 'https://www.example.com/path/to/mp4Video.mp4';

app.get('/video', (req, res) => {
    got.stream(mp4Url).pipe(res);
});

More examples described in this article: How to stream file downloads in Nodejs with Got.

---

You can also use the plain http/https modules to get the readstream, but I find got() to be generally useful at a higher level for lots of http request things so that's what I use. But, here's code with the https module.

const https = require('https');
const mp4Url = 'https://www.example.com/path/to/mp4Video.mp4';

app.get("/", (req, res) => {
    https.get(mp4Url, (stream) => {
        stream.pipe(res);
    });
});

More advanced error handling could be added to both cases.