The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
964280	dotnet/sdk	Run Tests	Log	dotnet/sdk#46425
964171	dotnet/sdk	Build	Log	dotnet/sdk#47110
964086	dotnet/sdk	Run Tests	Log	dotnet/sdk#47064
964211	dotnet/roslyn	Run Integration Tests	Log
2651549	dotnet-performance	Send job to Helix (Unix)	Log
2651524	dotnet-aspire	Build	Log
2651494	dotnet-runtime	Send job to Helix (Unix)	Log
963864	dotnet/sdk	Run Tests	Log	dotnet/sdk#47140
963907	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112909
962591	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112878
2651459	dotnet-dotnet	Checkout dotnet-dotnet@rid-specific-publish to s	Log
963766	dotnet/sdk	Run Tests	Log	dotnet/sdk#47128
2651407	dotnet-dotnet	Build
964039	dotnet/runtime	Send tests to Helix (Unix)	Log
964063	dotnet/runtime	Send tests to Helix (Unix)	Log
964066	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112930
964029	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112921
2651360	dotnet-runtime	Build product (browser_wasm)	Log
2651357	dotnet-runtime	Build product	Log
964048	dotnet/aspnetcore	Run E2E tests	Log	dotnet/aspnetcore#60634
963724	dotnet/sdk	Run Tests	Log	dotnet/sdk#47100
2651342	dotnet-runtime	Secure Supply Chain Analysis (auto-injected by policy)	Log
963997	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112965
963983	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112964
963988	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112938
2651212	dotnet-aspnetcore	Generate SBOM manifest	Log
963932	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#111072
963958	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112787
963858	dotnet/sdk	Build	Log	dotnet/sdk#47141
963886	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112667
2651303	dotnet-performance	Send job to Helix (Unix)	Log
963891	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112957
963850	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#111759
2651248	dotnet-runtime	Send job to Helix (Unix)	Log
963788	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112907
963830	dotnet/runtime	Send to Helix	Log	dotnet/runtime#110818
2651234	dotnet-aspnetcore	Checkout dotnet-aspnetcore@release/10.0-preview2 to s	Log
963793	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112951
2651191	dotnet-runtime	Send job to Helix (Windows)	Log
963175	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112849
963759	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#111543
963706	dotnet/sdk	Build	Log	dotnet/sdk#47131
963751	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112171
963755	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112730
963680	dotnet/sdk	Build	Log	dotnet/sdk#47139
962777	dotnet/xharness	Run tests in Helix	Log	dotnet/xharness#1364
2651111	dotnet-runtime	Send job to Helix (Windows)	Log
963645	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112949
2651089	dotnet-runtime	Send job to Helix (Windows)	Log
963683	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112921
963606	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112945
2651050	dotnet-runtime	Send job to Helix (Unix)	Log
2651071	dotnet-performance	Send job to Helix (Unix)	Log
2650973	dotnet-aspnetcore	🛡 Generate SBoM Manifest	Log
2651034	dotnet-runtime	Send job to Helix (Unix)	Log
963611	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112851
963589	dotnet/runtime	Send to Helix	Log
2651009	dotnet-sdk	🛡 Guardian: TSAUpload	Log
963585	dotnet/runtime	Send to Helix	Log
963588	dotnet/runtime	Send to Helix	Log
2650993	dotnet-sdk	🛡 Generate SBoM Manifest	Log
2650984	dotnet-sdk	🛡 Generate SBoM Manifest	Log
963535	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112863
963522	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112944
2650865	dotnet-efcore	🛡 Generate SBoM Manifest	Log
2650972	dotnet-arcade-services	🛡 Generate SBoM Manifest	Log
2650975	dotnet-runtime	Send job to Helix (Windows)	Log
2650961	dotnet-runtime	🛡 Generate SBoM Manifest	Log
963494	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112815
963460	dotnet/runtime	Send tests to Helix (Unix)	Log
2650925	dotnet-sdk	🛡 Generate SBoM Manifest	Log
963423	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112782
963396	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112728
963341	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112938
963315	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112937
963225	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112934
963308	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112936
2650845	dotnet-runtime	Send job to Helix (Unix)	Log
963307	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112935
963270	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112833
963187	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112736
963180	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112815
963121	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112933
2650756	dotnet-runtime	Send job to Helix (Windows)	Log
963093	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112667
962370	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112050
963070	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112921
961835	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112893
962979	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112460
962946	dotnet/runtime	Send tests to Helix (Unix)	Log
2650675	dotnet-performance	Send job to Helix (Unix)	Log
962939	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112928
962933	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112168
2650614	dotnet-runtime	Send job to Helix (Unix)	Log
962972	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112930
2650582	dotnet-aspnetcore	Disk utilization	Log
962404	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112826
962895	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112925
2650580	dotnet-dotnet-docker	Secure Supply Chain Analysis (auto-injected by policy)	Log
962825	dotnet/runtime	Send to Helix	Log
Displaying 100 of 1883 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
77	432	1883

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)