I have 200k data points and I'm trying to obtain derivative of fitted polynomial. I divided my data set into smaller ones every 0.5 K, the data is Voltage vs Temperature. My code roughly looks like this:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
testset=pd.read_csv('150615H0.csv',sep='\t')
x=np.linspace(1,220,219)
ub=min(testset['T(K)'])
lb=min(testset['T(K)'])-1
q={i:testset[(testset['T(K)'] < ub+i) & (testset['T(K)'] > lb+i)] for i in x}
f={j:np.polyfit(q[j]['T(K)'],q[j]['Vol(V)'],4) for j in q}
fs={k:np.poly1d(f[k]) for k in f}
fsd={l:np.polyder(fs[l],1) for l in fs}
for kk in q:
plt.plot(q[kk]['T(K)'],fsd[kk](q[kk]['T(K)']),color='blue',linewidth=2,label='fit')
Unsurprinsingly, the derivative is discontinuous and I don't like it. Is there any other way to fit polynomial locally and get continuous derivative at the same time ?
Have a look at the Savitzky-Gollay filter for an efficient local polynomial fitting.
It is implemented, for instance, in scipy.signal.savgol_filter. The derivative of the fitted polynomial can be obtained with the deriv=1 argument.