I have an ERA5 nc file of hourly precipitation from March until October for a 30-year time span. The data is from midnight to noon local time (so 4 am to 4 pm in UTC time zone). I need to find the daily sum of precipitation up until noon for each pixel. I have previously used python
daily_precipitation = ERA5.tp.resample(time='24H').sum('time')*1000
but days between October and March are also summed up, and I do not need that.
When I use cdo daysum
cdo daysum precip_hourly.nc precip_sum.nc
I get an error:
Warning: Some data values (min=-32766 max=52989) are outside the valid range (-32768 - 32767) of the used output precision!
Use the CDO option -b F32 or -b F64 to increase the output precision.
cdf_put_vara_double: name=tp type=NC_SHORT minval=-32766.000000 maxval=52989.000000
How can I overcome this error, and what is actually going wrong?
The answer is in the error message (CDO tries to be helpful), you need to convert the files from 2 byte packed integers to floats
cdo -b f32 daysum in.nc out.nc
Many centers like ecmwf pack their data. This means that the fields are stored as integers (2 byte) which are then scaled and an offset is added to get the original field. If you try to look at the metadata with
ncdump -h file.nc
You will see these offset and scale factors in the meta data. The problem is when you combine files with different packing factors cdo doesn't know what to do. Instead of unpacking automatically, it forces you to specify it manually because you have to choose whether to go to single or double precision (4 or 8 bytes). Obviously the file size will be doubled or quadrupled as a result.