Search code examples
information-retrievaldata-scienceprecision-recall

Precision-recall plot


I'd like to make sure that I plotted precision-recall curve. I have following data: recall = [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0] precision = [1, 1, 0.8, 0.7, 0.80, 0.65, 0.60, 0.72, 0.60, 0.73, 0.75] interpolated_precision = [1, 1, 0.80, 0.80, 0.80, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75] and prepared graph as shown below precision-recall curve

I'm not sure it is correct since I have seen figures with jiggles. An example is here: enter image description here I would be glad if anyone can confirm weather it is wrong or not.


Solution

  • The jagged lines / sawtooth pattern you usually see is more common with more data points (note at least 20 or so in the example figure, vs. exactly 10 for yours), that are coming from actual search results. You said nothing about where your data points are coming from.

    The reason the P-R figure often looks jagged is that every increase in recall is usually accompanied by a reduction in precision, at least temporarily, due to the likely addition of false positives. This is also the case in your figure, however, your "dips" seem smaller and your precision remains high throughout.

    However, there are two clear errors in your figure in the downward shifts for both precision and interpolated precision, since you are graphing the downward shifts as diagonal lines.

    For precision, any downward shift should always be a vertical line. You will not get this from a simple x-y plot of the points you described, e.g. in excel. These vertical lines contribute to the "jagged" look.

    For interpolated precision, the graph will always contain perpendicular straight lines, either horizontally or vertically. The definition of interpolated precision essentially requires that (see e.g. https://nlp.stanford.edu/IR-book/html/htmledition/evaluation-of-ranked-retrieval-results-1.html for the correct definition of interpolated precision at any point of recall).

    The key here is to realize that the data you are describing should not be graphed as independent observations, but rather as defining the P-R values for the rest of the graph in a particular way.