Comparing two distributions with relation among buckets

Question

I want to compare the following distributions with key-percentage.

dist1 = 200 - 0.1, 201-0.1, 500-0.8

dist2 = 200 - 0.15, 201 - 0.05, 500 - 0.8

dist3 = 200 - 0.1, 201-0.05, 500 - 0.85

dist1 is my original distribution. I want to compare it with dist2 , dist3. When I use something like KL divergence, I get KL(dist2,dist1)> KL(dist3,dist1) but in my current use case its the opposite, I want a metric which say dist2 is closer to dist1 than dist3 because there is only change between closer buckets i.e, 200,201 in dist2 compared to dist1 whereas in dist3 there is a movement from 201 bucket to 500 bucket.

Something like mean would work in this case but I want a more rigorous method of comparing these distributions which can capture all the variations.

Thanks

Answer 1

You may want to look into Earth mover’s distance. This measures the differences between two distributions by thinking of probability mass as a pile of dirt, then thinking of how much the dirt needs to move to transform one distribution into another. Moving dirt further takes more work than moving dirt less distance, whereas KL-divergence is insensitive to how far the probability mass travels.