Normed histogram y-axis larger than 1

Question

Sometimes when I create a histogram, using say seaborn's displot function, with norm_hist = True, the y-axis is less than 1 as expected for a PDF. Other times it takes on values greater than one.

For example if I run

sns.set(); 
x = np.random.randn(10000)
ax = sns.distplot(x)

Then the y-axis on the histogram goes from 0.0 to 0.4 as expected, but if the data is not normal the y-axis can be as large as 30 even if norm_hist = True.

What am I missing about the normalization arguments for histogram functions, e.g. norm_hist for sns.distplot? Even if I normalize the data myself by creating a new variable thus:

new_var = data/sum(data)

so that the data sums to 1, the y-axis will still show values far larger than 1 (like 30 for example) whether the norm_hist argument is True or not.

What interpretation can I give when the y-axis has such a large range?

I think what is happening is my data is concentrated closely around zero so in order for the data to have an area equal to 1 (under the kde for example) the height of the histogram has to be larger than 1...but since probabilities can't be above 1 what does the result mean?

Also, how can I get these functions to show probability on the y-axis?

Answer 1

The rule isn't that all the bars should sum to one. The rule is that all the areas of all the bars should sum to one. When the bars are very narrow, their sum can be quite large although their areas sum to one. The height of a bar times its width is the probability that a value would all in that range. To have the height being equal to the probability, you need bars of width one.

Here is an example to illustrate what's going on.

import numpy as np
from matplotlib import pyplot as plt
import seaborn as sns

fig, axs = plt.subplots(ncols=2, figsize=(14, 3))

np.random.seed(2023)
a = np.random.normal(0, 0.01, 100000)
sns.histplot(a, bins=np.arange(-0.04, 0.04, 0.001), stat='density', ax=axs[0])
axs[0].set_title('Measuring in meters')
axs[0].containers[1][40].set_color('r')

a *= 1000
sns.histplot(a, bins=np.arange(-40, 40, 1), stat='density', ax=axs[1])
axs[1].set_title('Measuring in milimeters')
axs[1].containers[1][40].set_color('r')

plt.show()

The plot at the left uses bins of 0.001 meter wide. The highest bin (in red) is about 40 high. The probability that a value falls into that bin is 40*0.001 = 0.04.

The plot at the right uses exactly the same data, but measures in milimeter. Now the bins are 1 mm wide. The highest bin is about 0.04 high. The probability that a value falls into that bin is also 0.04, because of the bin width of 1.

As an example of a distribution for which the probability density function has zones larger than 1, see the Pareto distribution with α = 3.

By directly using plt.hist, which returns the bin edges and heights, the area can easily be calculated.

np.random.seed(2023)
a = np.random.normal(0, 0.01, 100000)
v = plt.hist(a, bins=np.arange(-0.04, 0.04, 0.001), density=True, ec='k')

left = v[1][:-1]
right = v[1][1:]
area = (v[0] * (right-left)).sum()

print(f'Area: {area}')

---

sns.distplot is deprecated

import numpy as np
from matplotlib import pyplot as plt
import seaborn as sns

fig, axs = plt.subplots(ncols=2, figsize=(14, 3))

a = np.random.normal(0, 0.01, 100000)
sns.distplot(a, bins=np.arange(-0.04, 0.04, 0.001), ax=axs[0])
axs[0].set_title('Measuring in meters')
axs[0].containers[0][40].set_color('r')

a *= 1000
sns.distplot(a, bins=np.arange(-40, 40, 1), ax=axs[1])
axs[1].set_title('Measuring in milimeters')
axs[1].containers[0][40].set_color('r')

plt.show()