Summary: Add MCU model verification to CI

[psiddh]

Test Plan: OSS CI

Reviewers:

Subscribers:

Tasks:

Tags:

Summary

[PLEASE REMOVE] See CONTRIBUTING.md's Pull Requests for ExecuTorch PR guidelines.

[PLEASE REMOVE] If this PR closes an issue, please add a Fixes #<issue-id> line.

[PLEASE REMOVE] If this PR introduces a fix or feature that should be the upcoming release notes, please add a "Release notes: " label. For a list of available release notes labels, check out CONTRIBUTING.md's Pull Requests.

Test plan

[PLEASE REMOVE] How did you test this PR? Please write down any manual commands you used and note down tests that you have written if applicable.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13572

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Multiple CI trunk failures after landing https://github.com/pytorch/pytorch/pull/161002

❌ 10 New Failures, 3 Unrelated Failures

As of commit 81ce0da with merge base e5f94da ():

NEW FAILURES - The following jobs have failed:

• Apple / build-benchmark-app / macos-job (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• Apple / build-demo-ios / macos-job (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• Apple / build-frameworks-ios / macos-job (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• Build documentation / build (buck2) / Build doc (gh)
  At least one of the pre-conditions you specified did not hold
• Build Presets / windows (pybind) / build (gh)
  Process completed with exit code 1.
• Build Presets / windows (windows) / build (gh)
  Process completed with exit code 1.
• pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh)
  RuntimeError: Command docker exec -t 18c17e0f6e0bbc3dc20196c96e128230c0a790383f7e7c68984f03f5bd1f238a /exec failed with exit code 1
• trunk / test-arm-backend (test_pytest_ops_ethosu_fvp) / linux-job (gh)
  RuntimeError: Command docker exec -t bf5c1dc2a3c23ee1d9214cc1978eedc0ea43ee89e8716c64681d0c8ac5d3b72e /exec failed with exit code 1
• trunk / test-coreml-delegate / macos-job (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• trunk / test-huggingface-transformers-coreml (distilbert|coreml_fp32_gpu|--quantize) / macos-job (gh)
  The process '/opt/homebrew/bin/git' failed with exit code 1

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• trunk / unittest-release / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[digantdesai]

Looks good. Kicked them off on the CI. Let's see how long the job takes else we can reduce the number of models running serially on the job or somehow run them in parallel.

[psiddh]

Looks good. Kicked them off on the CI. Let's see how long the job takes else we can reduce the number of models running serially on the job or somehow run them in parallel.

Looks like job passed : https://github.com/pytorch/executorch/actions/runs/17146863588/job/48644753234?pr=13572

════════════════════════════════════════════════════════════════
🏁 MCU MODEL VALIDATION SUMMARY - TARGET: ethos-u85-128
════════════════════════════════════════════════════════════════

lstm : ✅ Passed
mv2 : ✅ Passed
mv3 : ✅ Passed
resnet18 : ✅ Passed

[digantdesai]

please resolve comments before merging.

[digantdesai]

why do we need to have ethos targets for this script? just take cortex-m55, cortex-m85?

[psiddh]

Yes this involves changing build_executor_runner.sh script (which the newly added 'run_mcu_models_fvp.sh' relies on). In the build_executor_runner.sh, it has something like this

if [[ ${target} == "ethos-u55" ]]; then
target_cpu=cortex-m55
else
target_cpu=cortex-m85
fi

[digantdesai]

let's do

if [[ ${target} == "cortex-m55" ]]; then
   target_for_build_et_runner = "ethos-u55-128"
else ...

[digantdesai]

ah perfect!! ❤️

[psiddh]

Latest run passed : https://github.com/pytorch/executorch/actions/runs/17278005534/job/49039591722?pr=13572

As a follow up ,check why removing "--use-pt-pinned-commit" is causing assertion failures while building portable kernels