Skip to content

feat: expose metric unit and target in the metrics API (CLIM-1103) - #603

Open
edvinstava wants to merge 6 commits into
masterfrom
eas/feat/clim-1103-friendly-metric-descriptions
Open

edvinstava wants to merge 6 commits into
masterfrom
eas/feat/clim-1103-friendly-metric-descriptions

Conversation

@edvinstava

@edvinstava edvinstava commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Adds optional unit and target metadata to metric specs and exposes them on the metric catalogue endpoint, so the frontend can drop its hardcoded MAPE unit and coverage/bias targets.

  • unit is a display suffix for the raw score (% for MAPE, periods for peak period lag). It does not rescale the score.
  • target is the ideal raw value for metrics where neither higher nor lower is better (0.8 for coverage 10-90, 0.5 for coverage 25-75 and ratio above truth, 0.0 for the peak metrics).
  • Convention: a metric with optimization_direction=None sets a target instead, and a score closer to it is better. Documented on the API field and in the contributor guide.
  • Score calculations and metric descriptions are unchanged for now.

Companion to frontend #255, which still needs to consume the new fields.

@edvinstava
edvinstava force-pushed the eas/feat/clim-1103-friendly-metric-descriptions branch from 07642b3 to 4344df0 Compare September 15, 2026 12:49
@edvinstava edvinstava changed the title feat: make metric descriptions user-friendly (CLIM-1103) feat: expose metric unit and target in the metrics API (CLIM-1103) Sep 15, 2026
@edvinstava
edvinstava marked this pull request as ready for review September 15, 2026 12:59
@knutdrand

Copy link
Copy Markdown
Contributor

Coverage metrics are imo (fighing with Claude about this) better the higher up til the target where its flat. That should be the message when showing it in FE. Only too low scores get a bad color. Should be reflected in API docs.

@knutdrand

Copy link
Copy Markdown
Contributor

I think the merge caused an error here with some preexisting work where Claude meant optimizing sensitivity and specificity was bad for HPO, but for info to user I think it should be ok, so reset these to the correct values @edvinstava

…hem from HPO

Sensitivity, specificity and outbreak accuracy lost optimization_direction
so HPO could not select a degenerate model on them, but the field also
feeds the metric catalogue the frontend colours scores by. Unsetting it
left clients with no rule at all for three metrics where higher is
plainly better to a reader.

Restore MAXIMIZE on all three and move the HPO decision to its own
valid_hpo_objective flag, which get_optimization_direction rejects on.
Presentation and objective selection stop sharing one field.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants