The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
982138	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#77619
982086	dotnet/runtime	Send to Helix	Log	dotnet/runtime#113564
982070	dotnet/runtime	Send to Helix	Log
982065	dotnet/runtime	Send to Helix	Log
982060	dotnet/runtime	Send to Helix	Log
982067	dotnet/runtime	Send to Helix	Log
982034	dotnet/runtime	AOT compile CoreCLR tests	Log
2664522	dotnet-sdk	Run AoT Tests in Helix	Log	#48510
2664521	dotnet-sdk	Run AoT Tests in Helix	Log	#48508
2664523	dotnet-sdk	Run AoT Tests in Helix	Log	#48509
981872	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#113576
981835	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#111004
2664376	dotnet-runtime	Send job to Helix (Unix)	Log
981795	dotnet/runtime	Send tests to Helix (Windows)	Log
981800	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#77630
981813	dotnet/roslyn	Test	Log	dotnet/roslyn#77628
2664345	dotnet-sdk	Run AoT Tests in Helix	Log	#48502
2664344	dotnet-sdk	Run AoT Tests in Helix	Log	#48501
2664217	dotnet-runtime	Send job to Helix (Unix)	Log
981548	dotnet/runtime	Send to Helix	Log
981611	dotnet/runtime	Send to Helix	Log	dotnet/runtime#110818
981606	dotnet/runtime	Send to Helix	Log	dotnet/runtime#113524
2664105	dotnet-runtime	Send job to Helix (Unix)	Log
981536	dotnet/runtime	AOT compile CoreCLR tests	Log
2664078	dotnet-sdk	Run Tests in Helix	Log	#48477
981357	dotnet/roslyn	Test	Log
981320	dotnet/runtime	Send to Helix	Log
980977	dotnet/runtime	Send tests to Helix (Unix)	Log
2663869	dotnet-dotnet	Build	Log
2663880	dotnet-runtime	Send job to Helix (Unix)	Log
2663886	dotnet-roslyn	Test	Log
2663861	dotnet-roslyn	Test	Log
2663859	dotnet-roslyn	Test	Log
981245	dotnet/roslyn	Run Integration Tests	Log
981041	dotnet/runtime	Send to Helix	Log
2663664	dotnet-sdk	Run TemplateEngine Tests	Log	#48482
2663663	dotnet-sdk	Run FullFramework Tests	Log	#48483
2663766	dotnet-sdk	Run AoT Tests in Helix	Log	#48479
2663764	dotnet-sdk	Run AoT Tests in Helix	Log	#48478
2663765	dotnet-sdk	Run AoT Tests in Helix	Log	#48477
2663763	dotnet-efcore	Send job to helix	Log	#48489
981187	dotnet/roslyn	Test	Log	dotnet/roslyn#77580
2663752	dotnet-arcade-validation	🛡 Guardian: Check if the path to run Secret Scanning is a directory (1ES PT)	Log
2663748	dotnet-roslyn	Test	Log
2663731	dotnet-roslyn	Test	Log
2663729	dotnet-roslyn	Test	Log
2663676	dotnet-runtime	Send tests to Helix (Unix)	Log	#48447
979726	dotnet/sdk	Run Tests	Log	dotnet/sdk#47338
2663675	dotnet-roslyn	Test	Log
2663606	dotnet-sdk	🛡 Guardian: AntiMalware Scanner (Binary)	Log	#48453
2663627	dotnet-roslyn	Test	Log
2663605	dotnet-runtime	Send job to Helix (Unix)	Log
2663607	dotnet-sdk	Run TestBuild Tests	Log	#48454
2663604	dotnet-arcade-validation	PowerShell@2	Log
2663597	dotnet-arcade-validation	Install MicroBuild plugin	Log
980872	dotnet/runtime	Send to Helix	Log
2663504	dotnet-dotnet	Build	Log
980755	dotnet/runtime	Send tests to Helix (Unix)	Log
980869	dotnet/runtime	Send job to Helix (Unix)	Log
980728	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#113483
2663552	dotnet-sdk	Run Tests in Helix	Log	#48479
2663557	dotnet-runtime	Initialize containers	Log
2663514	dotnet-runtime	Download Build Assets	Log	#48474
980763	dotnet/runtime	Send to Helix	Log
2663553	dotnet-performance	Run performance job script	Log
2663529	dotnet-performance	Send job to Helix (Windows)	Log
980786	dotnet/sdk	Initialize containers	Log	dotnet/sdk#47167
980693	dotnet/runtime	Send to Helix	Log	dotnet/runtime#113541
2663496	dotnet-performance	Send job to Helix (Windows)	Log
2663464	dotnet-runtime	Send job to Helix (Unix)	Log
980702	dotnet/sdk	🟣 Build	Log	dotnet/sdk#47480
980701	dotnet/sdk	Initialize containers	Log	dotnet/sdk#47480
2663352	dotnet-runtime	Send job to Helix (Unix)	Log
980252	dotnet/runtime	Send to Helix	Log
2663215	dotnet-extensions	Build Azure DevOps plugin	Log	#48455
2663144	dotnet-runtime	Send job to Helix (Unix)	Log
980124	dotnet/sdk	Build	Log	dotnet/sdk#47167
980084	dotnet/sdk	Build	Log	dotnet/sdk#47480
980073	dotnet/runtime	Send to Helix	Log
2662942	dotnet-runtime	Send tests to Helix (Unix)	Log	#48447
2662947	dotnet-wcf	OneLocBuild	Log
2662910	dotnet-runtime	Send job to Helix (Windows)	Log
2662865	dotnet-performance	Send job to Helix (Windows)	Log
2662841	dotnet-dotnet	Checkout dotnet-dotnet@dev/ellahathaway/signing-validation to s	Log
2662834	dotnet-dotnet	🛡 Component Governance	Log
2662830	dotnet-dotnet	Secure Supply Chain Analysis (auto-injected by policy)	Log
2662819	dotnet-dotnet	Download secrets: EngKeyVault	Log
2662782	dotnet-roslyn	Test	Log
2662777	dotnet-roslyn	Test	Log
2662645	dotnet-sdk	Run Tests in Helix	Log	#48427
2662650	dotnet-sdk	Run AoT Tests	Log	#48424
2662737	dotnet-roslyn	Test	Log
2662730	dotnet-roslyn	Test	Log
979543	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#110818
979534	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#113493
979525	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#113491
979522	dotnet/runtime	Send tests to Helix (Windows)	Log
979507	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#113490
979496	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#113184
2662664	dotnet-roslyn	Test	Log
Displaying 100 of 1074 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
12	317	1050

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)