Kernel Density Estimator ( with Gauss Kernel ) Sum f(x) = 1?

Question

I want to use KDE with the Gaussian Kernel. If I'm correct, the sum of all f(x) must be 1 ( ~ rounding ) ?

My Implementation looks like this:

    float K( float const& val)
     {
      const float p=1.0 / std::sqrt( 2.0 * M_PI);
      float result = 0.5 * (val*val);
      result = p * std::exp(- result);

      return result;
     };

std::vector< std::pair<float, float> kde( float *val, int len float h)
{
  std::vector< std::pair<float, float>> density( len );
  const float p = 1.0 / (h * len );

  for(int r=0;r<len;r++)
   {
    float sum = 0;

    for(int i=0;i<len;i++)
     sum += k( (val[r] - val[i]) / h );

    density[r] = std::make_pair( val[r], p*sum );
   }
  return density;
 }

And I choosed h > 0. Am i right that p*sum is the probability for the value val[r] ? The sum over all probability is > 1 ( but looks ok for me ).

Answer 1

You misinterpreted the assumptions on the probability density here. The density integrates to one, whereas its values at certain points are definitely not 1.

Let's discuss it using the following formula from the linked Wikipedia article which you seem to use:

This formula provides the density f_h(x) evaluated at point x.

From my review, your code correctly evaluates this quantity. Yet, you misinterpreted the quantity which should be one. As a density, the integral over the complete space should yield one, i.e.

This property is called normalization of the density.

Moreover, being a density itself, each summand of f_h(x) should yield 1/n when integrated over the whole space, when one also includes the normalization constant. Again, there's no guarantee on the values of the summands.

In one dimension, you can easily confirm the normalization by using the trapezoidal rule or another quadrature scheme (--if you provide a working example, I can try to do that.)