Repository navigation
QA: Make make run-mic easier to read and to target - #50652
Open
tarekziade wants to merge 11 commits into
Open
tarekziade wants to merge 11 commits into
tarekziade wants to merge 11 commits into
Conversation
After the summary, list failing contracts whose package versions differ from the ones their baseline was recorded with. They still count as failures; the warning flags numeric deltas that may be version drift rather than regressions.
Show a dict mismatch from the runner as one line per differing key instead of two full dicts, list each mismatched package on its own line, and keep only the target names in the end-of-run environment warning.
A heartbeat now redraws in place like the progress bar, so it no longer stays above the first result, and blank lines separate the results from the setup lines and the summary.
MODEL may now name a model the way it is spelled elsewhere (Wav2Vec2, audio-flamingo-3): case and punctuation are ignored against directory names, model types and config class names. A typo is not guessed; close directory names are suggested instead. Before running, the equivalent `tmic check` line is printed so it can be copied into a shell.
MODEL='qwen*' selects the contracts of every matching models/ directory. The unknown-model error now explains both names and patterns.
A target with no reviewed baseline for this platform passes after it loads and runs, without any output comparison. Name those targets after the summary so a pass is not read as a checked result.
An unquoted * is expanded (or rejected) by the shell; + is not. MODEL=qwen+ now selects every qwen* model directory; * still works when quoted. Document model names and patterns in the testing guide.
The MIC heartbeat printed "... 16s in" lines into the output while a contract loaded. The checker window's spinner title now carries a running timer (for every checker), so MIC prints nothing while it waits.
checkers.py sets CHECKERS_COLOR=1 for its children when it shows them in a terminal (and NO_COLOR is unset); check_mic then colors failing result lines and the failed/errored summary counts red. CI and piped output stay plain. The checker window resets the color after truncating a line.
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. Pages changed in this PR: |
ydshieh
approved these changes
Nov 12, 2026
ydshieh
left a comment
ydshieh
left a comment
Collaborator
There was a problem hiding this comment.
You are a better checkers than me, so I don't what else I could check but simply ✅
Thanks!
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
make run-micoutput was hard to act on locally, improved it.Also improved how to select models, not an exact directory and you can use
+to select severals.example:
qwen+last, pins to the newest baselines, that were recorded with torch 2.15