Search code examples
ruby-on-railsrubyamazon-s3ziprubyzip

Rubyzip: Export zip file directly to S3 without writing tmpfile to disk?


I have this code, which writes a zip file to disk, reads it back, uploads to s3, then deletes the file:

compressed_file = some_temp_path

Zip::ZipOutputStream.open(compressed_file) do |zos|
  some_file_list.each do |file|
    zos.put_next_entry(file.some_title)
    zos.print IO.read(file.path)
  end
end # Write zip file

s3 = Aws::S3.new(S3_KEY, S3_SECRET)
bucket = Aws::S3::Bucket.create(s3, S3_BUCKET)
bucket.put("#{BUCKET_PATH}/archive.zip", IO.read(compressed_file), {}, 'authenticated-read')

File.delete(compressed_file)

This code works already but what I want is to not create the zip file anymore, to save a few steps. I was wondering if there is a way to export the zipfile data directly to s3 without having to first create a tmpfile, read it back, then delete it?


Solution

  • I think I just found the answer to my question.

    It's Zip::ZipOutputStream.write_buffer. I'll check this out and update this answer when I get it working.

    Update

    It does work. My code is like this now:

    compressed_filestream = Zip::ZipOutputStream.write_buffer do |zos|
      some_file_list.each do |file|
        zos.put_next_entry(file.some_title)
        zos.print IO.read(file.path)
      end
    end # Outputs zipfile as StringIO
    
    s3 = Aws::S3.new(S3_KEY, S3_SECRET)
    bucket = Aws::S3::Bucket.create(s3, S3_BUCKET)
    
    compressed_filestream.rewind
    bucket.put("#{BUCKET_PATH}/archive.zip", compressed_filestream.read, {}, 'authenticated-read')
    

    The write_buffer returns a StringIO and needs to rewind the stream first before reading it. Now I don't need to create and delete the tmpfile.

    I'm just wondering now if write_buffer would be more memory extensive or heavier than open? Or is it the other way around?