I am using libvips to rotate the images. I am using a VM that have 3002 MB Ram and 512MB temp storage.The AWS Lambda Machine.
The command I running to rotate images is
vips rot original.jpg rotated.jpg d90
It throwing the following error
Exit Code: 1, Error Output: ERROR: wbuffer_write: write failed unix error: No space left on device
The jpg image is arround 10Mb.
Here's how libvips will rotate your jpg image.
90 degree rotate requires random access to the image pixels, but JPEG images can only be read strictly top-to-bottom, so as a first step, libvips has to unpack the JPG to a random access format. It uses vips (.v) format for this, which is pretty much a C array with a small header.
For images under 100mb (you can change this value, see below) decompressed, it will unpack to a memory buffer. For images over 100mb decompressed, it will unpack to a temporary file in /tmp
(you can change this, see below).
Next, it does the rotate to the output image. It can do this as a single streaming operation, so it will typically need enough memory for 256 scanlines on the input image, and 256 on the output, so around another 30mb or so in this case, plus some more working area for each thread.
In your specific case, the input image is being decompressed to a temporary file of 30,000 x 10,000 x 3 bytes, or about 900mb. This is way over the 512mb you have in /tmp
, so the operation fails.
The simplest solution is to force the loader to load via a memory buffer. If I try:
$ vipsheader x.jpg
x.jpg: 30000x10000 uchar, 3 bands, srgb, jpegload
$ time vips rot x.jpg y.jpg d90 --vips-progress --vips-leak
vips temp-3: 10000 x 30000 pixels, 8 threads, 128 x 128 tiles, 256 lines in buffer
vips x.jpg: 30000 x 10000 pixels, 8 threads, 30000 x 16 tiles, 256 lines in buffer
vips x.jpg: done in 0.972s
vips temp-3: done in 4.52s
memory: high-water mark 150.43 MB
real 0m4.647s
user 0m5.078s
sys 0m8.418s
The leak
and progress
flags make vips report some stats. You can see the initial decompress to the temporary file is talking 0.97s, the rotate to the output is 4.5s, it needs 150mb of pixel buffers and 900mb of disc.
If I raise the threshold, I see:
$ time VIPS_DISC_THRESHOLD=1gb vips rot x.jpg y.jpg d90 --vips-progress --vips-leak
vips temp-3: 10000 x 30000 pixels, 8 threads, 128 x 128 tiles, 256 lines in buffer
vips x.jpg: 30000 x 10000 pixels, 8 threads, 30000 x 16 tiles, 256 lines in buffer
vips x.jpg: done in 0.87s
vips temp-3: done in 1.98s
memory: high-water mark 964.79 MB
real 0m2.039s
user 0m3.842s
sys 0m0.443s
Now the second rotate phase is only 2s since it's just reading memory, but memory use has gone up to around 1gb.
This system is introduced in the libvips docs here:
http://jcupitt.github.io/libvips/API/current/How-it-opens-files.md.html