Using webp images with Pillow 2.2.2 or 2.3.0

Question

I was using pillow version 2.2.2 to convert webp image to jpeg image. The webp images are stored in a memory buffer. I found that when I try tom open the webp image it cause a memory leak with large number of images become a real problem.

def webp_to_jpeg(raw_img):
 image =  Image.open(StringIO.StringIO(raw_img))
 buffer = StringIO.StringIO()
 image.save(buffer, "JPEG")
 return string_buffer.getvalue()

This memory leak only happen when I work with webp images. I try to update pillow to 2.3.0 however when I did that I was not able to read webp images at all and I got the following exception "WEBP unknown extension"

Answer 1

It's a webp decoder bug in PILLOW (see here). It's still leaking memory in version 2.4.0

The only workaround I've found is based on python-webm. This one is leaking memory too, but you can fix it:

In encode.py, import the libc free() function :

from ctypes import CDLL, c_void_p
libc = CDLL(find_library("c"))
libc.free.argtypes = (c_void_p,)
libc.free.restype = None

Then modify _decode() to free the buffer allocated in the webp decoder .dll:

def _decode(data, decode_func, pixel_sz):
    bitmap = None
    width = c_int(-1)
    height = c_int(-1)
    size = len(data)

    bitmap_p = decode_func(str(data), size, width, height)
    if bitmap_p is not None:
        # Copy decoded data into a buffer
        width = width.value
        height = height.value
        size = width * height * pixel_sz
        bitmap = create_string_buffer(size)

        memmove(bitmap, bitmap_p, size)

        #Free the wepb decoder buffer!
        libc.free(bitmap_p)

    return (bytearray(bitmap), width, height)

To convert a RGB webp image :

from webm import decode

def RGBwebp_to_jpeg(raw_img):
    result = decode.DecodeRGB(raw_img)
    if result is not None:
        image = Image.frombuffer('RGB', (result.width, result.height), str(result.bitmap),'raw', 'RGB', 0, 1)

        buffer = StringIO.StringIO()
        image.save(buffer, "JPEG")
        return buffer.getvalue()