Search code examples
netcdfcdo-climate

Loop through multiple NetCDF files to calculate daily mean from hourly climate ERA5 datasets


I have hundreds of NETCDF files that I obtained from ERA5(Land) datasets. The temporal resolutions of the data are in hours but I need to compile them into daily mean. Single calculation is straightforward in CDO (see below). But when I tried to loop through the files, I'm getting an error message that suggests that I can only calculate them one at a time, which would be quite laborious. I was wondering if there is a workaround either in R or CDO. Here are my CDO syntaxes:

$ cdo daymean infile1980.nc outfile_day1980.nc  ##single operation works fine.

Trying to loop through

for i in C:/path/.*nc 
do 
    cdo daymean "${pattern}"* "${pattern}_day.nc" 
done

cdo (Abort): Too many input streams for operator 'daymean'!

The goal is to aggregate each year's hourly data into daily data


Solution

  • I found a workaround. Took a few steps but works. The first step is to merge all the .nc files

    ## "-b F64"  helps with precision
    ## "-f nc2" forces the files into nc2 to overcome size. 
    ## See here: https://code.mpimet.mpg.de/boards/1/topics/908
    
    $ cdo -b F64 -f nc2 mergetime *.nc all_data.nc  
    
    

    Next, compute the daily mean

    $ cdo daymean all_data.nc all_out.nc
    

    Then split the data by year

    $ cdo splityear all_out.nc  var_out_  ## it automatically appends "year" to each output name