Search code examples
pythonamazon-web-servicesamazon-s3gziptarfile

Append a folder to gzip in memory using python


I have a tar.gz file downloaded from s3, I load it in memory and I want to add a folder and eventually write it into another s3.
I've been trying different approaches:

from io import BytesIO
import gzip
buffer = BytesIO(zip_obj.get()["Body"].read())
im_memory_tar = tarfile.open(buffer, mode='a')

The above rises the error: ReadError: invalid header .

With the below approach:

im_memory_tar = tarfile.open(fileobj=buffer, mode='a')
im_memory_tar.add(name='code_1', arcname='code') 

The content seems to be overwritten.
Do you know a good solution to append a folder into a tar.gz file?
Thanks.


Solution

  • very well explained in question how-to-append-a-file-to-a-tar-file-use-python-tarfile-module

    Note that 'a:gz' or 'a:bz2' is not possible. If mode is not suitable to open a certain (compressed) file for reading, ReadError is raised. Use mode 'r' to avoid this. If a compression method is not supported, CompressionError is raised.