How to make a resumable upload using NodeJS (NestJS) to Google Cloud Storage (GCS)?

Question

I'm having trouble making a resumable upload service in NestJS to GCS.

The scenario is, the client uploads a file from the frontend and on the backend it is sent directly to GCS without temporarily storing it on the BE server.

Here's the code snippet I'm using.

try {
  const filePath = path.join(directory, nameWithExtension);
  const file = this.bucket.file(filePath);
  const passthroughStream = new stream.PassThrough();
  passthroughStream.write(image.buffer);
  passthroughStream.end();

  const streamFileUpload = async () => {
    passthroughStream
      .pipe(file.createWriteStream({ resumable: true, gzip: true, public: true }))
      .on('finish', () => console.log(`resumable upload succeed`));
    return filePath;
  };

  const res = await streamFileUpload().catch((error) => {
    throw new Error(`${logPrefix} Error uploading ${filePath} ${error.message}`);
  });
  return `${process.env.GOOGLE_STORAGE_ENDPOINT}/${this.bucket.name}/${res}`;
} catch (error) {
  throw new Error(`${logPrefix} Error uploading ${error.message}`);
}

on createWriteStream I included the option resumable: true but it doesn't seem to work as expected.

I'm curious about this https://cloud.google.com/storage/docs/performing-resumable-uploads but still don't quite get it.

Any suggestion is very much appreciated, Thanks!

Update 2021/10/06

I changed the code to be like below:

async resumableUpload(directory: string, image: MultipartFile, nameWithExtension: string): Promise<string> {
  const logPrefix = 'GoogleStorageService.resumableUpload:';
  const filePath = path.join(directory, nameWithExtension);
  const { buffer } = image;
  const blob = this.bucket.file(filePath);
  const promiseUpload = new Promise((resolve, reject) => {
    const blobStream = blob.createWriteStream({
      resumable: true,
      gzip: true,
      public: true,
    });
    blobStream
      .on('error', () => {
        reject(`${logPrefix} Unable to upload image, something went wrong`);
      })
      .on('finish', async () => {
        const publicUrl = new URL(process.env.GOOGLE_STORAGE_ENDPOINT || '');
        publicUrl.pathname = path.join(this.bucket.name, filePath);
        resolve(publicUrl.toString());
      })
      .end(buffer);
  });
  const response = promiseUpload
    .then((res: string) => res)
    .catch((err: Error) => {
      throw new Error(`${logPrefix} Error uploading ${err.message}`);
    });
  return response;
}

And it worked out well.

But if there any better way please don't hesitate to give the best advice for this question. Thanks

Answer 1

Posting @Wisnu solution as a community wiki for better visibility. After Wisnu edited the code which is below, the resumable upload service started working.

async resumableUpload(directory: string, image: MultipartFile, nameWithExtension: string): Promise<string> {
  const logPrefix = 'GoogleStorageService.resumableUpload:';
  const filePath = path.join(directory, nameWithExtension);
  const { buffer } = image;
  const blob = this.bucket.file(filePath);
  const promiseUpload = new Promise((resolve, reject) => {
    const blobStream = blob.createWriteStream({
      resumable: true,
      gzip: true,
      public: true,
    });
    blobStream
      .on('error', () => {
        reject(`${logPrefix} Unable to upload image, something went wrong`);
      })
      .on('finish', async () => {
        const publicUrl = new URL(process.env.GOOGLE_STORAGE_ENDPOINT || '');
        publicUrl.pathname = path.join(this.bucket.name, filePath);
        resolve(publicUrl.toString());
      })
      .end(buffer);
  });
  const response = promiseUpload
    .then((res: string) => res)
    .catch((err: Error) => {
      throw new Error(`${logPrefix} Error uploading ${err.message}`);
    });
  return response;
}