Skip to content

[autorevert] Add Autorevert Metrics page with visualizations and data retrieval #7625

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
izaitsevfb merged 5 commits into from
Jan 12, 2026
Merged

[autorevert] Add Autorevert Metrics page with visualizations and data retrieval #7625

izaitsevfb merged 5 commits into from
Jan 12, 2026

Conversation

@izaitsevfb
Copy link
Contributor

@izaitsevfb izaitsevfb commented Dec 29, 2025 •
edited
Loading

New HUD page at /metrics/autorevert to track autorevert system performance with signal-based analysis.

Features

  • Signal Recovery Detection: Identifies when job groups recover (2+ red → 2+ green commits)
  • Revert Attribution: Classifies reverts as autorevert vs human-initiated
  • Precision/Recall Metrics: TP (correct autoreverts), FP (unnecessary autoreverts), FN (missed autoreverts)
  • False Positive Analysis: GitHub API integration to detect autoreverts where PRs were relanded without changes
  • Weekly Trend Charts: Visualize autorevert activity over time

Key Metrics

Metric Definition
TP Autorevert that fixed a signal or PR had changes after
FP Autorevert where PR relanded unmodified
FN Human revert with signal recovery (autorevert should have caught)

@pytorch-bot pytorch-bot bot added the ci-no-td label Dec 29, 2025
@vercel
Copy link

vercel bot commented Dec 29, 2025 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
torchci Ready Ready Preview Dec 30, 2025 0:10am

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Dec 29, 2025
@izaitsevfb izaitsevfb marked this pull request as draft December 29, 2025 20:36
@izaitsevfb izaitsevfb marked this pull request as ready for review January 6, 2026 18:36
@izaitsevfb izaitsevfb requested a review from jeanschmidt January 6, 2026 18:36
@izaitsevfb izaitsevfb merged commit 6f38eaa into main Jan 12, 2026
5 checks passed
@izaitsevfb izaitsevfb deleted the hud-autorevert-metrics branch January 12, 2026 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jeanschmidt jeanschmidt jeanschmidt approved these changes

Assignees

No one assigned

Labels

ci-no-td CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@izaitsevfb @jeanschmidt