Jpeg Image : zlib.error : Error -3 while decompressing data : incorrect header check

Question

I have a jpg image that is in memory and therefore is in bytes. I then compress it with the following:

zlib.compress(pickle.dumps(self.fileInstance))

I compress it to store it into a MongoDB database. This image is one of the elements in the record that I save. I save the with the following.

list_search_result = list(search_result)
json_search_result = json.dumps(list_search_result, cls=Encoder, indent=2)

I got an error with bytes and Json, so I simply convert the bytes image to a string with the following.

from bson.objectid import ObjectId
import json 
import datetime
class Encoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, ObjectId):
            return str(obj)
        elif isinstance(obj, datetime.datetime):
            return str(obj)
        elif isinstance(obj,bytes):
            return str(obj)
        return super(Encoder, self).default(obj)

Now, I want to recover the image from the json file. I thought that doing the same steps in the opposite order would work but no.

So here is what I do to store:

image -> pickle -> compress -> str -> json

I thought this would work:

json -> bytes -> decompress -> depickle -> image

I receive the zlib.error : Error -3 during the following:

  image = pickle.load(zlib.decompress(attachment[1].encode()))

  image = io.BytesIO(image)
  dt = Image.open(image)

EDIT:

Okay so I was toying around, and I think the issue might be the .encode(). I start with b" ". After str(b" "), I get "b' '". If I do the .encode(), I get b"b' '". How do I deal with this?

Answer 1

str() is useful to display something - it creates text readable for human.

It may show that you had bytes ("b''") or it may show string like \xe0 for values which can't be converted to chars. But it doesn't have to create text useful for keeping in database.

Many databases have field to keep bytes and then you could keep image as bytes (without converting to pickle which may only add more bytes, and without compressing because images already use some compression)

If you have to keep (or send by internet) file as string then better convert it to base64. And this method used by some APIs to send image in JSON.

---

Convert image to base64

import base64

fh = open('lenna.png', 'rb')
data = fh.read()
fh.close()

data = base64.b64encode(data).decode()

print(text[:100])  # text

# ... save in database ...

Convert base64 to image

# ... read from database ...

data = base64.b64decode(text.encode())

print(data[:100])  # bytes

fh = open('lenna.png', 'wb')
fh.write(data)
fh.close()

Result:

# text

iVBORw0KGgoAAAANSUhEUgAAAgAAAAIACAIAAAB7GkOtAAAAA3NCSVQICAjb4U/gAAAgAElEQVR4nOzbXa5tS5Il5DHMzH3OtfY+

# bytes 

b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x02\x00\x00\x00\x02\x00\x08\x02\x00\x00\x00{\x1aC\xad\x00\x00\x00\x03sBIT\x08\x08\x08\xdb\xe1O\xe0\x00\x00 \x00IDATx\x9c\xec\xdb]\xaemK\x92%\xe41\xcc\xcc}\xce\xb5\xf6>\xe7\xde\x88\xc8\xa2\x1e\xea\x85\x16 !$\x10\x88?\x05/t\x06\x95\xc8\xe2\x95\x06\xd0'

Tested on image lenna.png (Wikipedia: Lenna)