Search code examples
pythonzippicklearchive

How to archive binary data

Use zipfile to archive data dictionary:

import os, sys, cPickle, zipfile 
data = {1: 'one'}
pfile = r'c:\temp\myPickle.p'
cPickle.dump( data, open( pfile, "wb" ))

zfilename = r'c:\temp\myArchive.zip'
zfile = zipfile.ZipFile(zfilename, "w", zipfile.ZIP_DEFLATED)
zfile.write(pfile, os.path.basename(pfile))
zfile.close()

The approach results two files on a local drive: one is pickle and another is zip. To get pickled data zip file needs to be un-archived:

fh = open(zfilename, 'rb')
z = zipfile.ZipFile(fh)
for each in z.namelist():
    z.extract(each, r'c:\temp')
fh.close()

How to simplify the process?

Solution

  • Use gzip instead. It compresses the file as you write it so there is no need for intermediate files.

    # (python 2) import cPickle as pickle
import pickle
import gzip

data = {1: 'one'}

pfile = r'test.pkl.gz'
pickle.dump(data, gzip.open(pfile, "w"), pickle.HIGHEST_PROTOCOL)
print pickle.load(gzip.open(pfile))