I'm using TREC_EVAL to evaluate a search engine. I'd like to know how it calculates the Mean Average Precision (MAP). I'm sure it doesn't calculate a simple average of the average precisions (AP). It seems a weighted arithmetic but I can't understand which weights are used.
MAP is indeed a simple arithmetic mean of the AP scores for individual topics. (But remember that AP for an individual topic is computed over all relevant documents. There is a frequently used incorrect definition of AP that computes 'AP' over only relevant retrieved documents, but that is a nonsensical measure as it rewards retrieving fewer relevant.)
The "Common Measures" section of the Appendix to each TREC proceedings has the definition of the most commonly used TREC eval measures. TREC proceedings are in the Publications section of the TREC web site, trec.nist.gov.
Ellen Voorhees TREC project manager NIST