mmdet.core.evaluation.mean_ap vs cocoeval

[MABatin]

Hi,
I was wondering how the function "eval_map" script from the mean_ap.py script differs from coco dataset evaluate? Because for the same model, "eval_map" gives mAP=0.9289 @ IoU=0.5 but coco evaluate gives mAP=0.9212 @ IoU=0.5.
What can be the reason for this discrepancy?

[abinav719]

The evaluation protocol might vary a bit. There is some slight difference among different datasets’ evaluation mechanisms. For example: In COCO, if you look at their source code, they rank all the detections based on the scores from high to low, and then cut off the results at the maximum number of detections allowed. For each detection, the algorithm iterates through all ground truth, and the previously unmatched ground truth with the highest IoU is matched with the detection.