Search code examples
pythonpandasplotconfidence-intervalerrorbar

Add 95% confidence intervals as error bars to pandas bar plot


I want to add 95% confidence interval error bars to a pandas bar plot, like here. This is what my data looks like:

ciRatings.head(20)

                            count   mean        std
condition   envCond         
c01         CSNoisyLvl1     40      4.875000    0.404304
            CSNoisyLvl2     40      4.850000    0.361620
            LabNoisyLvl1    52      4.826923    0.382005
            LabNoisyLvl2    52      4.826923    0.430283
            LabQuiet        92      4.826087    0.408930
c02         CSNoisyLvl1     40      2.825000    0.902631
            CSNoisyLvl2     40      3.000000    0.816497
            LabNoisyLvl1    52      3.250000    1.218726
            LabNoisyLvl2    52      3.096154    1.089335
            LabQuiet        92      2.956522    1.036828
c03         CSNoisyLvl1     40      3.750000    0.669864
            CSNoisyLvl2     40      3.775000    0.659740
            LabNoisyLvl1    52      4.307692    0.728643
            LabNoisyLvl2    52      4.288462    0.723188
            LabQuiet        92      3.967391    0.790758
c06         CSNoisyLvl1     40      4.450000    0.638508
            CSNoisyLvl2     40      4.250000    0.669864
            LabNoisyLvl1    52      4.692308    0.578655
            LabNoisyLvl2    52      4.384615    0.599145
            LabQuiet        92      4.717391    0.452735

I looked at the pandas documentation on how to use errorbars, and tried to copy their code example. I came up with the following:

# calculate range of CI around mean (as it is symmetric)
ci95_lower = []

for i in ciRatings.index:
    count, mean, std = ciRatings.loc[i]
    ci95_lower.append(mean - 1.96*std/math.sqrt(count))

ciRatings['CI95_lower'] = ci95_lower
ciRatings['CI95_range'] = ciRatings['mean'] - ciRatings['CI95_lower']

# extract CI range and means
ciRange = ciRatings[['CI95_range']]
ciRange = ciRange.unstack()
ciRatings = ciRatings[['mean']]

# bar plot with CI95 as error lines
ciBarPlot = ciRatings.unstack().plot(kind='bar', yerr=ciRange, capsize=4)

plt.show()

However, this results in the plot below, clearly without error bars. What was my mistake? I assume I have misunderstood what exactly I have to pass the plot function as the yerr argument.

bar plot

Edit: Using the answer by Quang Hoang, I changed my code as follows to achieve the desired confidence interval bars:

# calculate range of CI around mean (as it is symmetric)
ci95_lower = []

for i in ciRatings.index:
    count, mean, std = ciRatings.loc[i]
    ci95_lower.append(mean - 1.96*std/math.sqrt(count))

ciRatings['CI95_lower'] = ci95_lower
ciRatings['CI95_range'] = ciRatings['mean'] - ciRatings['CI95_lower']

# bar plot with CI95 lines
ciBarPlot = ciRatings['mean'].unstack(level=1).plot.bar(
            yerr=ciRatings['CI95_range'].unstack(level=1), capsize=4)

plt.show()

Solution

  • The given link suggests:

    fig, ax = plt.subplots(figsize=(12,8))
    (df['mean'].unstack(level=1)
               .plot.bar(yerr=df['std'].unstack(level=1) * 1.96,
                         ax=ax, capsize=4)
    )
    plt.show()
    

    Output:

    enter image description here