Search code examples
rloopsrasterr-raster

Increment Loop in Batches


I was hoping someone could help me with a loop function I'm working with. I have tried searching Google and Stack Overflow extensively but, as I don't know the exact search terminology, I fear I am missing some results. With that in mind, I apologise in advance if this question has already been asked but I hope that someone can point me in the right direction for a solution.

About My Data

I have downloaded 1000s of files from NASAs MODIS satellite. As my study area covers a large area, I've had to download the data over an area of 6 tiles. As a result, my list of downloaded files are actually 'grouped' together in bunches of 6 (despite appearing as just a list of files within explorer).

I have written some simple for loops within R to do some initial processing (in the example below this is to resample the tiles so that they can then be stitched together using raster::mosaic. This processing can happen on the files individually at this stage so I have found the below loop to be perfect for my needs:

resampled.raster<-list()

for (l in 1:24){
  cat(l,"\n")
  resampled.raster[[l]]<-resample(rst[[l]], s, method="ngb")
}

The Problem

The next stage of my processing requires me to mosaic the rasters in groups of 6, before moving onto the next batch of 6. For example, if I have a list of 24 files, the first 6 will need to be processed as a collection of files and then saved (in a list) before moving to the next 6. After 4 iterations of this (to make the 24 files total), I need the loop to stop.

The closest solution I have found from searching the internet is here but this isn't a solution to my problem.

Is this 'batch' processing possible with using a for loop within R? Many thanks in advance for your help.


Solution

  • First note that it is almost certainly wrong to use resample and mosaic. If these are tiles, you should be able to use merge to merge them.

    There surely is something unique about the names of the files that need to be grouped? That is what you need to use.

    Get a vector with filenames

    # ff <- list.files()
    

    Here a toy example

    ff <- c('fileA1', 'fileA2', 'fileB1', 'fileB2') 
    

    Get the unique groups

    code <- gsub('file', '', ff)
    code <- substr(code, 1, 1)
    uc <- unique(code)
    

    Loop over the groups

    for (u in uc) {
        files <- ff[u == code]
        r <- lapply(files, raster)
        r$filename <- paste0("merged/", u, ".tif")
        m <- do.call(merge, r)
    }