Search code examples
javascriptgoogle-cloud-storage

Set metadata while uploading a new object using stream


I'm using stream to upload my files to GCS.

Everything is working but I can't figure out how to add custom metadata to the newly created object.

I can achive the desire result by setting the metadata AFTER the object has been created, as shown in google docs (https://github.com/googleapis/nodejs-storage/blob/main/samples/fileSetMetadata.js) , but this means adding update policy to my user and for me it seems pretty strange to update the object instead of doing everything in one single operation.

Here is my code I'm using to upload file to GCS:

// 1 get bucket (assume this is working, it returns my bucket)
const bucket = gcsGetBucket(storageBucket || GCS_BUCKET_DEFAULT);
if (!bucket) throw new Error('Bucket not found or not initialized');
const newFile: File = bucket.file("testFile);
**newFile.metadata = { test1: 'xyz1' };**

// 2 function used to upload using passthrough stream
const uploadStream = () =>
  new Promise((resolve, reject) => {

    const passthroughStream = new Stream.PassThrough();
    passthroughStream.write(fileToUpload);
    passthroughStream.end();
    passthroughStream
      .pipe(
        newFile.createWriteStream({
          resumable: false,
          gzip: true,
          contentType: fileMimeType,
          **metadata: { test2: 'xyz2' },**
        }),
      )
      .on('error', (err) => {
        console.warn('error!');
        reject(err);
      })
      .on('finish', () => {
        resolve(true);
      });
  });


// 3 upload file
const uploadResult = await uploadStream();

// 4 set response
res.success = true;

My attempts write nothing to object's metadata.


Solution

  • As you mentioned it works by setting the metadata directly to the file object, it will directly modifies the metadata of the file object

    You can refer to this answer provided by Max888

    when you want to set custom metadata as well as contentType etc, you need to wrap the whole thing as a meta data key value like this:

    await file.save(html, {
          metadata: {
            contentType: "fileMimeType",
            resumable: false,
            gzip: true,
            metadata: { test2: 'xyz2' },
          },
        });