Search code examples
node.jslambdazip

How to unzip a zip with multiple files into my lambda node.js function tmp directory preserving the original filenames


I have a zip file in S3 that contains several txt files and an image.

I am needing to grab that file from S3 and then unzip all of the files with their original filenames into the tmp directory in lambda using node.js

I am not super familiar with node and while the S3 part of getting the file works fine, I frankensteined the zip part together from the interwebs and was wondering if someone could help me get it right.

const zlib = require('zlib');
const fs = require('fs');
    
try { 
  const s3Object = await s3
    .getObject({
      Bucket: 'mybucket',
      Key: `zip/${myzipfilename}`
    })
    .promise();

  console.log("s3 zip fetched");
                
  // write file to tmp          
  writeFileSync(`/tmp/${myzipfilename}`, s3Object.Body);
        
  //unzip files
  const fileContents = fs.createReadStream(`/tmp/${myzipfilename}`);

  //I am quite sure this part is incorrect and is currently giving me an error
  const writeStream = fs.createWriteStream(`./tmp/${filename.slice(0, -3)}`);

  const unzip = zlib.createGunzip();
  fileContents.pipe(unzip).pipe(writeStream);            
}

End result within the lambda tmp directory would be something like:

/tmp/myoriginaltxtfilename.txt
/tmp/myoriginaltxtfilename2.txt
/tmp/myoriginaltxtfilename3.txt
/tmp/myoriginalimagefilename.png

I don't need to rezip anything.


Solution

  • You have a couple of issues in your code. First of all, at this line:

    const writeStream = fs.createWriteStream(`./tmp/${filename.slice(0, -3)}`);
    

    filename is not defined.

    Second, you're using nodejs zlib to extract a .zip file which contains multiple files which won't work. zlib module is only for streams and buffers that represent singular resources not zip archives. You could use node-stream-zip instead.

    Let's say you've successfully downloaded the zip file from S3 and saved in /tmp directory. Using node-stream-zip extracting the files from the zip file without unzipping it would look something like this:

    const StreamZip = require('node-stream-zip');
    const zip = new StreamZip({
        file: `/tmp/${myzipfilename}`,
        storeEntries: true
    });
    
    zip.on('ready', () => {
      console.log('All entries read: ' + zip.entriesCount);
    });
    
    zip.on('entry', (entry) => {
      if ('/' === entry.name[entry.name.length - 1]) {
        console.log('[DIR]', entry.name);
        return;
      }
    
      console.log('[FILE]', entry.name);
    
      zip.stream(entry.name, (err, stream) => {
        if (err) {
          console.log('Error: ', err.toString());
          return;
        }
    
        stream.on('error', (err) => {
          console.log('[ERROR]', err);
          return;
        });
    
        stream.pipe(fs.createWriteStream(`/tmp/${entry.name}`));
      });
    });