R Magick process multiple files

Question

so hello everyone.

I'am facing a problem with magick and I don't get it...

I wrote a function to load, rescalce and save an image

    convert <- function(infile,outpath, topspace=200){
  require(magick)
  if(!file.exists(infile)) return("file does not exist")

  print(paste0("Load ", infile, " ..."))
  read <- image_read(infile, strip= TRUE)
  basename <- basename(infile)
  basename2 <- gsub(".png", "", basename)

  print(paste0("Resize ", basename, " ..."))
  read4300 <- image_resize(read, "4300x4300")
  read4000 <- image_resize(read4300, "4000x4000")

  print(paste0("Extent ", basename, " ..."))
  read5400 <- image_extent(read4300, paste0("4500x", 5400-2*topspace), gravity = "North")
  read5400 <- image_extent(read5400, "4500x5400", gravity = "Center")
  read4050 <- image_extent(read4000, "4500x4050")
  print(paste0("Speicher ", basename, " ..."))

  image_write(read4000 , paste0(outpath,"\\",basename2,"4000.png"))
  image_write(read5400 , paste0(outpath,"\\",basename2,"5400.png"))
  image_write(read4050 , paste0(outpath,"\\",basename2,"4050.png"))
}

This funtion works fine. I want to apply this function on multiple files in a loop, but everytime it gets extremely slow with the third image (in any order). It gets extremely slow evertime while resizing image3.png. If i try just image3.png it works fine. If I change the order (3-2-1) it the problem occurs with image1.png. So the files are arround 1MB, processing time of the first two images is arround 15 seconds, the third one needs about 5-10 minutes...

I tried to add

read <- NULL
  read4300 <- NULL
  read4000 <- NULL
  read5400 <- NULL
  read4050 <- NULL
  remove(list = ls())

in the convert function, but this does not help. I'am running out of space or what is the problem and how can i fix it?

Answer 1

This sounds like a memory issue.

This might be caused by a limitation of usable resources by the ImageMagick security policy. See https://github.com/ImageMagick/ImageMagick/issues/396 (Your code would not work for me due to this issue, but you seem to be on Windows, so things might differ there.)

To see if this could be an issue in your case, try running identify -list resource from the command line.

If it is rather an issue with memory management within R, you could add a gc(full=TRUE) command at the end of your convert function to see if that could reclaim any resources by explicitly invoking garbage collection.

You could try this slightly modified function on your files and see if that works for you.

convert_img <- function(infile, outpath, topspace=200){
    require(magick)
    if(!file.exists(infile)) return("file does not exist")
    
    print(paste0("Load ", infile, " ..."))
    read0 <- image_read(infile, strip = TRUE)
    basename <- basename(infile)
    basename2 <- tools::file_path_sans_ext(basename)
    
    print(paste0("Resize ", basename, " ..."))
    read4300 <- image_resize(read0, "4300x4300")
    read4000 <- image_resize(read0, "4000x4000")
    
    print(paste0("Extent ", basename, " ..."))
    read5400 <- image_extent(read4300, paste0("4500x", 5400-2*topspace), gravity = "North")
    read5400 <- image_extent(read5400, "4500x5400", gravity = "Center")
    read4050 <- image_extent(read4000, "4500x4050")
    print(paste0("Speicher ", basename, " ..."))
    
    image_write(read4000 , file.path(outpath, paste0(basename2, "4000.png")))
    image_write(read5400 , file.path(outpath, paste0(basename2,"5400.png")))
    image_write(read4050 , file.path(outpath, paste0(basename2,"4050.png")))
    gc(full=TRUE)
}

# my test examples ran fine and R memory usage stayed below 2 GB:
files <- list.files(path="~/Downloads", pattern = ".png", full.names = TRUE)
sapply(files, convert_img, outpath="~/outfiles/")