What is the most straightforward way to perform a t-test in python and to include CIs of the difference? I've seen various posts but everything is different and when I tried to calculate the CIs myself it seemed slightly wrong... Here:
import numpy as np
from scipy import stats
g1 = np.array([48.7107107,
36.8587287,
67.7129929,
39.5538852,
35.8622661])
g2 = np.array([62.4993857,
49.7434833,
67.7516511,
54.3585559,
71.0933957])
m1, m2 = np.mean(g1), np.mean(g2)
dof = (len(g1)-1) + (len(g2)-1)
MSE = (np.var(g1) + np.var(g2)) / 2
stderr_diffs = np.sqrt((2 * MSE)/len(g1))
tcl = stats.t.ppf([.975], dof)
lower_limit = (m1-m2) - (tcl) * (stderr_diffs)
upper_limit = (m1-m2) + (tcl) * (stderr_diffs)
print(lower_limit, upper_limit)
returns:
[-30.12845447] [-0.57070077]
However, when I run the same test in SPSS, although I have the same t and p values, the CIs are -31.87286, 1.17371, and this is also the case in R. I can't seem to find the correct way to do this and would appreciate some help.
You're subtracting 1 when you compute dof, but when you compute the variance you're not using the sample variance:
MSE = (np.var(g1) + np.var(g2)) / 2
should be
MSE = (np.var(g1, ddof=1) + np.var(g2, ddof=1)) / 2
which gives me
[-31.87286426] [ 1.17370902]
That said, instead of doing the manual implementation, I'd probably use statsmodels' CompareMeans:
In [105]: import statsmodels.stats.api as sms
In [106]: r = sms.CompareMeans(sms.DescrStatsW(g1), sms.DescrStatsW(g2))
In [107]: r.tconfint_diff()
Out[107]: (-31.872864255548553, 1.1737090155485568)
(really we should be using a DataFrame here, not an ndarray, but I'm lazy).
Remember though that you're going to want to consider what assumption you want to make about the variance:
In [110]: r.tconfint_diff(usevar='pooled')
Out[110]: (-31.872864255548553, 1.1737090155485568)
In [111]: r.tconfint_diff(usevar='unequal')
Out[111]: (-32.28794665832114, 1.5887914183211436)
and if your g1 and g2 are representative, the assumption of equal variance might not be a good one.