How to convert fixed size dimension to unlimited in a netcdf file

Question

I'm downloading daily 600MB netcdf-4 files that have this structure:

netcdf myfile {
dimensions:
        time_counter = 18 ;
        depth = 50 ;
        latitude = 361 ;
        longitude = 601 ;
variables:
        salinity
        temp, etc     

I'm looking for a better way to convert the time_counter dimension from a fixed size (18) to an unlimited dimension.

I found a way of doing it with the netcdf commands and sed. Like this:

ncdump myfile.nc | sed -e "s#^.time_counter = 18 ;#time_counter = UNLIMITED ; // (currently 18)#"  | ncgen -o myfileunlimited.nc

which worked for me for small files, but when dumping a 600 MB netcdf files, takes to much memory and time.

Somebody knows another method for accomplishing this?

Answer 1

Your answers are very insightful. I'm not really looking a way to improve this ncdump-sed-ncgen method, I know that dumping a netcdf file that is 600MB uses almost 5 times more space in a text file (CDL representation). To then modify some header text and generate the netcdf file again, doesn't feels very efficient.

I read the latest NCO commands documentation, and found a option specific to ncks "--mk_rec_dmn". Ncks mainly extracts and writes or appends data to a new netcdf file, then this seems the better approach, extract all the data of myfile.nc and write it with a new record dimension (unlimited dimension) which the "--mk_rec_dmn" does, then replace the old file.

ncks --mk_rec_dmn time_counter myfile.nc -o myfileunlimited.nc ; mv myfileunlimited.nc myfile.nc 

To do the opposite operation (record dimension to fixed-size) would be.

ncks --fix_rec_dmn time_counter myfile.nc -o myfilefixedsize.nc ; mv myfilefixedsize.nc myfile.nc