Taking differences along time axis for the data with irregular overpass time

Question

I wonder how to take differences along the time axis of data with the irregular overpass time with Xarray.

Data:

• NASA SMAP L3 soil moisture data, regularly gridded
• Irregular overpass time. Two- or three-daily, depending on the overpass. Missing daily values are filled with NaNl.

What I want to do:

Calculate the difference of the soil moisture data along the time axis, ignoring NaN data. For example, in pandas dataframes, I can achieve it by dropping NaN data and take the diff(). Now I want to do it in Xarray; however, an element-wise drop is not supported in Xarray, so I am at a loss.

What I tried:

• Simply taking diff(). The resulting values end up in full of NaN because missing daily values are filled with NaN.
• Forward-filling the NaN data and taking diff(). This successfully calculates the differences that I want. However, I lose time information; the original time corresponding to the data becomes unclear, which I need for the next step.
• Looking into Sparse Array package ... but I didn't find anything helpful to calculate diff().

I am looking for any algorithms or workaround!

Answer 1

Thanks to @MichaelDelgado here is the answer.

da.bfill(dim="time").diff(dim="time").where(da.notnull().shift(time=+1))

Another note: I also dug into sparse array, but there was no good solution to it.