Forward-backward filtering with exponential moving average

Question

I'm trying to smooth a signal using exponential weighted moving average using both past and future values.

I have tried the built-in version of pandas ewm().mean(), which is however non-symmetric, i.e. it only uses the past and current value to produce a smoothed version of the current point. This naturally introduces a lag -- I'm looking for a version that is symmetric and thus lag-free. Symmetric smoothing should also introduce less boundary artifacts at the start of the time series.

Example code showcasing the problem:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

t = np.linspace(1, 4, 1000)
y = pd.Series(t**2 + 0.5 *np.sin(100*t), index=t)

plt.plot(t, y, label='original')
plt.plot(t, y.ewm(halflife=40).mean(), label='ewma')
plt.grid(True)
plt.legend(loc='upper left')
plt.show()

Output: Plot showing the problem with the smoothed signal lagging behind

Answer 1

To perform a symmetric smoothing using exponential weighted moving average, you can take the average of the forward and backward exponentially weighted moving average for each data point, like so:

Python:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

def symmetric_ewma(data, halflife):
    forward_ewma = data.ewm(halflife=halflife).mean()
    backward_ewma = data.iloc[::-1].ewm(halflife=halflife).mean().iloc[::-1]
    return (forward_ewma + backward_ewma) / 2

t = np.linspace(1, 4, 1000)
y = pd.Series(t**2 + 0.5 *np.sin(100*t), index=t)

plt.plot(t, y, label='original')
plt.plot(t, symmetric_ewma(y, halflife=40), label='symmetric_ewma')
plt.grid(True)
plt.legend(loc='upper left')
plt.show()

In this example, the "symmetric_ewma" function takes in the data series and the halflife value and returns the symmetric exponential weighted moving average.

The function calculates the forward exponentially weighted moving average and the backward exponentially weighted moving average by reversing the data series using the iloc[::-1] indexing method. It then takes the average of the two to obtain the symmetric exponential weighted moving average. That should be what you're looking for :)