fix(metrics): score mAR@K from each image's top K predictions only

[kevin9327]

Description

MeanAverageRecall can lower mAR@1 or mAR@10 when you add a prediction that ranks below the limit:

import numpy as np
import supervision as sv
from supervision.metrics import MeanAverageRecall

targets = sv.Detections(xyxy=np.array([[0, 0, 10, 10]], dtype=float), class_id=np.array([0]))
top = sv.Detections(xyxy=np.array([[0, 0, 10, 14]], dtype=float), class_id=np.array([0]), confidence=np.array([0.9]))
duplicate = sv.Detections(xyxy=np.array([[0, 0, 10, 10]], dtype=float), class_id=np.array([0]), confidence=np.array([0.1]))

MeanAverageRecall().update(top, targets).compute().mAR_at_1                                   # 0.5
MeanAverageRecall().update(sv.Detections.merge([top, duplicate]), targets).compute().mAR_at_1  # 0.0 on develop, 0.5 here

The class docstring says "mAR @ 1 considers only the highest confidence detection for each image", and #2136 introduced per-image top-K for that reason. But _compute matches all of an image's predictions to its targets first, and only then drops the ones ranked below K (prediction_indices < max_detections). The shared matcher (_match_detection_batch_with_target_indices, #2380) is greedy by highest IoU, not by confidence. So in the example, the 0.1 duplicate with IoU 1.0 takes the target, the 0.9 prediction with IoU 0.71 is left unmatched, and then the duplicate is cut off at K=1. Nothing counts.

pycocotools doesn't have this problem because it matches in score order, so a lower-ranked detection can never take a target from a higher-ranked one. With the IoU-first matcher, the limit has to be applied before matching.

This PR adds _match_top_predictions, which runs the same matcher once per detection limit on that image's top K predictions and stores the result along a new last axis of matches, (P, Th, K). _compute_average_recall_for_classes then reads the slice for each limit. Nothing else changes: not the matcher, not the per-image definition of K, and not the size buckets, which go through the same _compute.

Motivation and Context

Duplicates are the typical trigger: after NMS, two overlapping predictions of the same object can both survive when their mutual IoU is below the NMS threshold, and the less confident one may fit the target more tightly. Crowded images with more than 10 predictions hit the same effect at mAR@10. A random check makes the scale visible. I generated 300 three-image scenes with jittered duplicates and compared mAR_at_K against MeanAverageRecall run on each image's top-K predictions only, which is the documented definition:

	mAR@1 differs	mAR@10 differs	mAR@100 differs
develop	168 / 300	52 / 300	0 / 300
this branch	0 / 300	0 / 300	0 / 300

Changes Made

• src/supervision/metrics/mean_average_recall.py: new _match_top_predictions. _compute sorts by confidence first and stores per-limit matches, and _compute_average_recall_for_classes indexes them by limit. It also gains a docstring.
• tests/metrics/test_mean_average_recall.py: test_mar_at_k_ignores_predictions_ranked_below_k is the example above: mAR@1 0.5, since the top prediction's IoU of 100/140 passes 5 of the 10 thresholds, and mAR@10 and mAR@100 1.0.
• docs/changelog.md: entry under Unreleased.

Testing

New test on develop:

E       assert 0.0 == 0.5 ± 5.0e-07
FAILED tests/metrics/test_mean_average_recall.py::test_mar_at_k_ignores_predictions_ranked_below_k
1 failed

With this change:

$ pytest tests/metrics/test_mean_average_recall.py
18 passed
$ pytest tests/metrics          # without OpenCV
396 passed      (develop: 395 passed)
$ pytest            # src doctests + tests, OpenCV installed
4106 passed, 1 skipped      (develop: 4105 passed, 1 skipped)
$ ruff check && ruff format --check && docformatter --check   # changed files
All checks passed!
2 files already formatted

The existing mAR tests pass unchanged, including test_complex_integration_scenario and the per-image top-K tests from #2136. mypy on the changed files reports the same errors on develop and on this branch.

• Added/updated tests, and the full suite passes locally
• Updated docs (docstrings / mkdocs entry) for new or changed public API
• Added a changelog entry in docs/changelog.md under Unreleased (skip for lint/type/format-only or pure doc changes)

Additional Notes

Matching now runs three times per image, once for each limit, instead of once. Each run only sees up to K predictions, so the extra cost is small next to the IoU matrix, which is computed once as before.

I left the matcher itself alone. sv.match_detections, Precision, Recall and F1Score use it on purpose without ranking by confidence, and they don't truncate by rank, so this problem doesn't apply to them.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91%. Comparing base (eb58b7b) to head (c8f0453).
⚠️ Report is 3 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff           @@
##           develop   #2604   +/-   ##
=======================================
  Coverage       91%     91%           
=======================================
  Files           78      78           
  Lines        11388   11396    +8     
=======================================
+ Hits         10387   10395    +8     
  Misses        1001    1001           

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

🟡 Changes recommended

The regression test does not exercise the claimed mAR@10 cutoff behavior.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Fixes mAR@K by matching only each image’s top-K predictions before scoring.

Changes:

• Adds limit-specific matching.
• Adds a regression test and changelog entry.
• Assessment: Code 5/5, Testing 4/5, Docs 5/5.

File summaries

File	Description
src/supervision/metrics/mean_average_recall.py	Implements top-K matching per detection limit.
tests/metrics/test_mean_average_recall.py	Tests low-confidence duplicate handling.
docs/changelog.md	Documents the corrected behavior.

Review details

• Files reviewed: 3/3 changed files
• Comments generated: 1
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.