pythonobject-detectionpycocotools
Evaluation on Coco-type data set returns error
I am using the Faster R-CNN model available from https://github.com/facebookresearch/maskrcnn-benchmark. I am trying to evaluate the results of a trained model on the KITTI data set, after converting it to Coco Format (2D object detection).
The results are 0 or -1 and sometimes it throws an error in the CocoApi toolkit at g["area"].
pycoco if g['ignore'] or (g['area']aRng[1]): "KeyError: 'area'"
From what I found while researching the problem, "area" is used for segmentation and I do not have that kind of annotation in my data set.
An small example of how my converted annotation file looks:
{
"images": [
{
"file_name": "007292.png",
"id": 1,
"width": 1392,
"height": 512
},
{
"file_name": "000603.png",
"id": 2,
"width": 1392,
"height": 512
},
{
"file_name": "004313.png",
"id": 3,
"width": 1392,
"height": 512
},
{
"file_name": "006401.png",
"id": 4,
"width": 1392,
"height": 512
},
{
"file_name": "005442.png",
"id": 5,
"width": 1392,
"height": 512
}
],
"annotations": [
{
"image_id": 1,
"id": 1,
"category_id": 1,
"bbox": [
589.08,
176.53,
26.719999999999914,
26.409999999999997
],
"iscrowd": 0
},
{
"image_id": 1,
"id": 2,
"category_id": 1,
"bbox": [
235.9,
190.63,
115.16,
57.78
],
"iscrowd": 0
},
{
"image_id": 1,
"id": 3,
"category_id": 1,
"bbox": [
426.57,
184.2,
42.0,
26.700000000000017
],
"iscrowd": 0
},
{
"image_id": 2,
"id": 4,
"category_id": 1,
"bbox": [
1211.2,
182.65,
11.799999999999955,
186.35
],
"iscrowd": 0
},
{
"image_id": 2,
"id": 5,
"category_id": 1,
"bbox": [
386.94,
180.98,
57.80000000000001,
30.55000000000001
],
"iscrowd": 0
},
{
"image_id": 2,
"id": 6,
"category_id": 1,
"bbox": [
736.21,
173.49,
113.90999999999997,
96.44999999999999
],
"iscrowd": 0
},
{
"image_id": 2,
"id": 7,
"category_id": 1,
"bbox": [
701.98,
174.7,
91.55999999999995,
66.01000000000002
],
"iscrowd": 0
},
{
"image_id": 2,
"id": 8,
"category_id": 1,
"bbox": [
682.42,
176.25,
58.200000000000045,
47.53
],
"iscrowd": 0
},
{
"image_id": 2,
"id": 9,
"category_id": 1,
"bbox": [
667.8,
175.85,
51.190000000000055,
39.24000000000001
],
"iscrowd": 0
},
{
"image_id": 2,
"id": 10,
"category_id": 1,
"bbox": [
654.6,
176.88,
31.110000000000014,
26.49000000000001
],
"iscrowd": 0
},
{
"image_id": 3,
"id": 11,
"category_id": 1,
"bbox": [
267.69,
179.7,
101.13,
33.120000000000005
],
"iscrowd": 0
},
{
"image_id": 3,
"id": 12,
"category_id": 1,
"bbox": [
461.31,
176.05,
72.38000000000005,
28.73999999999998
],
"iscrowd": 0
},
{
"image_id": 3,
"id": 13,
"category_id": 1,
"bbox": [
600.36,
177.08,
52.360000000000014,
23.299999999999983
],
"iscrowd": 0
},
{
"image_id": 4,
"id": 14,
"category_id": 1,
"bbox": [
1061.94,
96.68,
179.05999999999995,
277.32
],
"iscrowd": 0
},
{
"image_id": 4,
"id": 15,
"category_id": 1,
"bbox": [
280.52,
184.02,
148.01,
96.92999999999998
],
"iscrowd": 0
},
{
"image_id": 4,
"id": 16,
"category_id": 1,
"bbox": [
143.54,
179.75,
350.11,
194.25
],
"iscrowd": 0
},
{
"image_id": 4,
"id": 17,
"category_id": 1,
"bbox": [
861.45,
139.2,
178.20000000000005,
64.58000000000001
],
"iscrowd": 0
},
{
"image_id": 4,
"id": 18,
"category_id": 1,
"bbox": [
1018.27,
144.44,
88.04999999999995,
43.25
],
"iscrowd": 0
},
{
"image_id": 4,
"id": 19,
"category_id": 1,
"bbox": [
1061.23,
147.01,
100.31999999999994,
39.27000000000001
],
"iscrowd": 0
},
{
"image_id": 4,
"id": 20,
"category_id": 1,
"bbox": [
439.12,
184.57,
66.10000000000002,
43.43000000000001
],
"iscrowd": 0
},
{
"image_id": 4,
"id": 21,
"category_id": 1,
"bbox": [
381.68,
184.81,
98.5,
63.59
],
"iscrowd": 0
},
{
"image_id": 4,
"id": 22,
"category_id": 1,
"bbox": [
673.9,
172.28,
52.389999999999986,
36.53999999999999
],
"iscrowd": 0
},
{
"image_id": 4,
"id": 23,
"category_id": 1,
"bbox": [
473.3,
180.94,
49.079999999999984,
36.900000000000006
],
"iscrowd": 0
},
{
"image_id": 4,
"id": 24,
"category_id": 1,
"bbox": [
609.73,
179.26,
35.860000000000014,
27.670000000000016
],
"iscrowd": 0
},
{
"image_id": 4,
"id": 25,
"category_id": 1,
"bbox": [
668.0,
173.81,
88.37,
31.150000000000006
],
"iscrowd": 0
},
{
"image_id": 4,
"id": 26,
"category_id": 1,
"bbox": [
585.17,
172.42,
40.520000000000095,
15.630000000000024
],
"iscrowd": 0
},
{
"image_id": 5,
"id": 27,
"category_id": 1,
"bbox": [
192.88,
178.88,
74.23000000000002,
33.49000000000001
],
"iscrowd": 0
},
{
"image_id": 5,
"id": 28,
"category_id": 1,
"bbox": [
250.68,
179.92,
65.70999999999998,
26.27000000000001
],
"iscrowd": 0
},
{
"image_id": 5,
"id": 29,
"category_id": 1,
"bbox": [
306.54,
178.95,
55.48999999999995,
22.670000000000016
],
"iscrowd": 0
}
],
"categories": [
{
"name": "Car",
"id": 1
}
]
}
EDIT: I have added the area property to the labels, computed as bbox[2]*bbox[3], there is no error anymore, but the results are 0.
Any help would be gladly appreciated!
Solution
According to the 1. Detection Evaluation of the COCO official documents, AP by area are also evaluated.
Therefore, if there is no area in your own custom dataset, an error will occur in the following part of the code of site-packages/pycocotools/cocoeval.py.
if g['ignore'] or (g['area']<aRng[0] or g['area']>aRng[1]):
This code means that if the area exceeds the upper limit aRng[1] or is less than the lower limit aRng[0], it is ignored from the evaluation target.
If you are using your own custom dataset, you may not have an area in the ground truth. In that case, you may create an area from the width and height of the bounding box.
If you don't need to evaluate each area, you can comment out like below code.
# It does not evaluate for each area
# if g['ignore'] or (g['area']<aRng[0] or g['area']>aRng[1]):
if g['ignore']:
- Python method chaining in functional programming style
- flask-jwt-extended: Fake Authorization Header during testing (pytest)
- For loop through the list unless empty?
- Polars make all groups the same size
- Is there a way to specify a default base-template for all templates in django?
- How to tackle time limit exceeded error in leetcode
- Is pd.get_dummies() updated in newer versions of Pandas making it default to Booleans (True/False) instead of (0/1)?
- What's the function like sum() but for multiplication? product()?
- How to type hint a dynamically-created dataclass
- Issue with pulling the data with EIA API with Python
- 403 Forbidden Error when scraping a site, user-agents already used and updated. Any ideas?
- Fullstack web-hosting services
- How to handle an AnalysisException on Spark SQL?
- Python requests is slow and takes very long to complete HTTP or HTTPS request
- Is there a way to modify an element in a Numpy array based on the value of other elements?
- Tkinter grid manager height/width nonconsistent
- Sql Alchemy Insert Statement failing to insert, but no error
- How can I create a Polars struct while eval-ing a list?
- Excel using win32com and python
- django: on pypy, psyco, unladen swallow or cpython, which one is the fastest?
- How convert a list into multiple columns and a dataframe?
- Name not defined in type annotation
- Static type checkers and Language Servers not recognizing attributes of objects that are subclasses
- How do I get multiple OID values in PySNMP?
- how to log a file in Django
- Is there a simple and efficient way to evaluate Elementary Symmetric Polynomials in Python?
- Iterating over two lists one after another
- Python -i flag for production
- What is PyCompilerFlags in Python C API?
- How to make Python check whether a variable is a number or letter