Skip to content

Fix UV benchmark pipeline: Linux PATH, venv dot-source params, and install pattern #1148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Copilot wants to merge 2 commits into
base: users/pahallis/run-bench
Choose a base branch
from
+11 −14
Draft

Fix UV benchmark pipeline: Linux PATH, venv dot-source params, and install pattern #1148

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
38566ab
Initial plan
Copilot Mar 5, 2026
994ea36
Fix UV pipeline: prepend PATH for Linux, remove broken venv dot-sourc…
Copilot Mar 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 9 additions & 12 deletions pipelines/scripts/Invoke-CopilotBenchmarks.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@
This script runs in Azure DevOps under an AzureCLI@2 task with federated authentication.
Feed authentication is handled by a preceding PipAuthenticate@1 task that sets
PIP_EXTRA_INDEX_URL for the azure-sdk/internal/MicrosoftSweBench feed.
The script retrieves a GitHub PAT from KeyVault, installs MSBench CLI, and invokes:
The script retrieves a GitHub PAT from KeyVault, installs MSBench CLI via
uv tool install, and invokes:
msbench-cli run --agent github-copilot-cli --benchmark <benchmark> --model <model>

MSBench CLI reference:
Expand Down Expand Up @@ -41,7 +42,6 @@ if (!$Model) {
throw "Model parameter is required."
}

$repoRoot = Join-Path $PSScriptRoot ".." ".." -Resolve
$vaultName = "kv-msbench-eval-azuremcp"
$secretName = "azure-eval-gh-pat"

Expand All @@ -51,9 +51,6 @@ Write-Host "NoWait: $NoWait"

$pipelineRun = $env:TF_BUILD -eq "True"

. "$PSScriptRoot/Create-Venv.ps1" -VenvName "venv" -RepoRoot $repoRoot
. "$PSScriptRoot/Activate-Venv.ps1" -VenvName "venv" -RepoRoot $repoRoot

# --- Retrieve GitHub PAT from KeyVault ---
try {
Write-Host "Retrieving GitHub PAT from KeyVault $vaultName secret $secretName"
Expand Down Expand Up @@ -96,16 +93,16 @@ if ($env:PIP_EXTRA_INDEX_URL) {
}


Write-Host "`n> uv pip install msbench-cli"
& uv pip install msbench-cli
Write-Host "`n> uv tool install msbench-cli"
uv tool install msbench-cli
if ($LASTEXITCODE -ne 0) {
throw "uv pip install msbench-cli failed with exit code $LASTEXITCODE"
throw "uv tool install msbench-cli failed with exit code $LASTEXITCODE"
}

Write-Host "`n> uv run 'msbench-cli' version"
uv run 'msbench-cli' version
Write-Host "`n> msbench-cli version"
msbench-cli version
if ($LASTEXITCODE -ne 0) {
throw "uv run msbench-cli failed with exit code $LASTEXITCODE"
throw "msbench-cli version failed with exit code $LASTEXITCODE"
}

$runArgs = @(
Expand All @@ -121,7 +118,7 @@ if ($NoWait) {
}

Write-Host "`n> msbench-cli $($runArgs -join ' ')"
uv run 'msbench-cli' @runArgs
msbench-cli @runArgs
if ($LASTEXITCODE -ne 0) {
throw "msbench-cli run failed with exit code $LASTEXITCODE"
}
4 changes: 2 additions & 2 deletions pipelines/templates/steps/install-uv.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ steps:
targetType: inline
script: |
echo "##vso[task.prependpath]$HOME/.local/bin"
displayName: 'Prepend path for MacOS'
condition: eq(variables['Agent.OS'], 'Darwin')
displayName: 'Prepend $HOME/.local/bin to PATH (Linux/macOS)'
condition: or(eq(variables['Agent.OS'], 'Linux'), eq(variables['Agent.OS'], 'Darwin'))

- task: PowerShell@2
displayName: 'Install uv (Windows)'
Expand Down