Search code examples
multithreadinggraphicsmagick

Can graphicsmagick batch process on more than 2 threads?

If I create 6 graphicsmagick batch files for converting 35k images, this is what I see in htop:

htop

Why aren't more threads being used? I'm guessing that both of those threads are even on the same core (4 core intel with hyperthreading). I can't find a graphicsmagick config about this online. Do I blame my OS for poorly scheduling?

The only related option in the gm man page, -limit <type> <value>, is a resource limit per image, while I am looking for a way to increase the number of threads used for multiple images, not for a single image.

It is true that the only thing graphicsmagick says about parallelism is about OpenMP (which seems to be about multi-threaded single image processing). So maybe there is no support for what I am trying to do. My question might be more general then: "if I launch multiple instances of gm, why do they all run on the same thread?" I'm not sure if this an OS question or a gm question.

Each line in the batch files is:

convert in/file1.jpeg -fuzz 10% -trim -scale 112x112^ -gravity center -extent 112x112 -quality 100 out/file2.jpeg

I run the batch file with: gm batch -echo on -feedback on data/convert/simple_crop_batchfile_2.txt

I am on GraphicsMagick 1.3.18 2013-03-10 Q8 and Ubuntu 14.10, which when I upgrade with apt-get tells me: Calculating upgrade... graphicsmagick is already the newest version

My story here does show the pointlessness of using multiple batch files (although there is a 30% speedup in overall processing time using 2 batch files concurrently over 1)

Solution

  • Turns out I can blame this on the CPU, the picture in the question of core utilization comes from an Intel Xeon X5365 @ 3.00GHz processor. Here is a picture of just 4 concurrent processes on an Intel Xeon E5-2620 v2 @ 2.10GHz:

    parallel cpu utilization

    The OS and software versions are the same on the two machines (as well as the same exact task w/ the same exact data), the only difference is the CPU. In this case the later CPU is over 2x as fast (for the case of 4 batchfiles).