Evaluation metric

[fushh]

Thank you for the excellent work!

I noticed that the evaluation metric implemented in other_metric.py does not account for object categories.
See the relevant code here:

Rex-Omni/evaluation/metrics/other_metric.py

Lines 394 to 405 in 6508981

	# Flatten all boxes for matching
	all_gt_boxes = []
	all_pred_boxes = []
	for category in gt_boxes_dict:
	all_gt_boxes.extend(gt_boxes_dict[category])
	# Filter out None values from predictions
	pred_boxes_for_category = pred_boxes_dict.get(category, [])
	valid_pred_boxes = [
	box for box in pred_boxes_for_category if box is not None and box != "None"
	]
	all_pred_boxes.extend(valid_pred_boxes)

While this design is appropriate for single-class detection or grounding tasks (e.g., Dense200, HumanRef, RefCOCOg val/test), it may not be suitable for multi-class detection scenarios such as VisDrone, where classification accuracy is also important.

Am I missing something, or is this intentional?

[Mountchicken]

Hi @fushh
Evaluation is routed in calculate_metrics_for_sample by task_name, not one single metric for everything:

Detection tasks (common_object_detection, dense_object_detection, long_tailed_object_detection, etc.): GT and predictions stay as category-keyed dicts. Names are normalized (normalize_category_name) before matching, and preds are pulled per category from pred_boxes_dict. We also report InstructionFollowing, which measures whether predicted category names align with GT at the sample level.

pointing / pointing_referring: Category names are required — matching only happens when GT and pred share the same category.

referring_object_detection (default when task_name is missing): Category names are intentionally ignored; only box localization is evaluated. That fits single-target grounding (RefCOCOg, HumanRef, Dense200, etc.), where the category string is part of the referring expression, not a multi-class label to score separately.

So this isn’t an oversight — different tasks use different category-handling strategies on purpose. VisDrone-style multi-class detection goes through the detection branch, where category names are kept and normalized throughout evaluation.