Search code examples
pythonimagevips

Strange behavior in pyvips, impossible to write some images

I'm currently trying to make pyvips work for a project where I need to manipulate images of a "big but still sensible" size, between 1920x1080 and 40000x40000.

The installation worked well, but these particular 2 lines sometime work, sometime don't.

img = pyvips.Image.new_from_file('global_maps/MapBigBig.png')
img.write_to_file('global_maps/MapTest.png')

It seems that for the biggest images, I get the following error message when writing back the image (the loading works fine):

pyvips.error.Error: unable to call VipsForeignSavePngFile

pngload: arithmetic overflow

vips2png: unable to write to target global_maps/MapFishermansRowHexTest.png

I say it seems, because the following lines work perfectly well (with a size of 100 000 x 100 000, far bigger than the problematic images):

size = 100000
test = pyvips.Image.black(size, size, bands=3)
test.write_to_file('global_maps/Test.png')

I could not find an answer anywhere, do you have any idea what I'm doing wrong ?

EDIT: Here is a link to an image that does not work (it weights 102 Mo).

This image was created using pyvips and a 40 time smaller image, this way:

img = pyvips.Image.new_from_file('global_maps/MapNormal.png')
out = img.resize(40, kernel='linear')
out.write_to_file('global_maps/MapBigBig.png')

And it can be read using paint3D or gimp.

Solution

  • I found your error message in libspng:

    https://github.com/randy408/libspng/blob/master/spng/spng.c#L5989

    It looks like it's being triggered if the decompressed image size would go over your process pointer size. If I try a 32-bit libvips on Windows I see:

    $ ~/w32/vips-dev-8.12/bin/vips.exe copy MapFishermansRowHexBigBig.png x2.png
pngload: arithmetic overflow
vips2png: unable to write to target x2.png

    But a 64-bit libvips on Windows works fine:

    $ ~/vips-dev-8.12/bin/vips.exe copy MapFishermansRowHexBigBig.png x.png
$

    So I think switching to a 64-bit libvips build would probably fix your problem. You'll need a 64-bit python too, of course.

    I also think this is probably a libspng bug (or misfeature?) since you can read >4gb images on a 32-bit machine as long as you don't try to read them all in one go (libvips reads in chunks, so it should be fine). I'll open an issue on the libspng repo.