Search code examples
rterra

Controlling terra SpatRaster size in memory


Here I create a simple raster that only contains the integer "1".

r <- rast(matrix(rep(1, 100), nrow = 10))

This object does not have a datatype associated with it:

datatype(r.in.flt)
# ""

I write it to disk using datatype = "FLT4S" and then read it back in.

name.flt <- "test_FLT4S.tif"
writeRaster(r, name.flt, datatype = "FLT4S")
r.in.flt <- rast(name.flt)

I also write it to disk using datatype = "INT1U" and then read it back in.

name.int <- "test_INT1U.tif"
writeRaster(r, name.int, datatype = "INT1U")
r.in.int <- rast(name.int)

The incoming datatypes appear correctly:

datatype(r.in.flt)
# "FLT4S"
datatype(r.in.int)
# "INT1U"

And the corresponding file sizes show a correct reduction in size for the integer datatype:

file.size(name.flt)
# 665
file.size(name.int)
# 618

But in memory, the two objects appear to have the same size:

object.size(r.in.flt)
# 1304 bytes
object.size(r.in.int)
# 1304 bytes

QUESTIONS:

  1. Why does the original SpatRaster r not have a datatype associated with it?
  2. Is it possible to set to set the datatype of the SpatRaster r? How? I want to do this to make sure that the SpatRaster I'm working with is saved as an integer datatype so that it takes up the least amount of memory as I process a large dataset.
  3. Why does the object that was read in from the "INT1U" file not consume less memory in R?

Solution

  • The data type is only relevant to raster files. The object itself does not have one. Raster values that are in RAM are always numeric (double precision; 8 bytes). But note that methods like as.data.frame and extract return the type that is on disk.

    See terra::mem_info for RAM requirements.

    The main design principle of terra is that you do not have to read all values into memory. The data are automatically processed in smaller chunks if that is necessary. There may be exceptions, and I do not know your application so I cannot be sure, but you may be needlessly complicating things.