Search code examples
.netf#netcdf

Reading subset of netcdf data with sdslite

I am trying to read a subset of NETCDF data of a fairly massive variable of oceanographic data. The file itself is 40GB large. The variable having three dimensions; time, depth, and the corresponding element.

When attempting to read everything with the sdslite library as such:

let ds = DataSet.Open("massive_file.nc")

let u : single [,,] = ds.GetData<single[,,]> ("u" )

I get the exception

System.OverflowException: Arithmetic operation resulted in an overflow.
   at System.Buffer.ByteLength(Array array)
   at Microsoft.Research.Science.Data.NetCDF4.NetCdfVariable`1.ToMultidimArray[T](T[] ncArray, Int32[] shape)

Which tells me that I am trying to read too much data at once. However, I have been reading the docs and I cannot for the life of me see how I can read a subset of this data. So does anyone have any idea how to do this? I may just be blind and have missed it, but I sure do not get it.

Solution

  • Figured it out.

    let dims = ds.Dimensions.["nele"].Length
let u : single [,] = ds.GetData<single[,]> ("u", DataSet.ReduceDim(0), DataSet.Range(0, 32), DataSet.Range(0, 1000000) )

    Will get the 0th time step index, 32 depth indexes with all their corresponding elements.