Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The active test run was aborted. Reason: Test host process crashed #451

Open
3 tasks
fanyang-mono opened this issue Jul 25, 2023 · 8 comments
Open
3 tasks

Comments

@fanyang-mono
Copy link
Member

fanyang-mono commented Jul 25, 2023

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=351450

Build leg reported

chrome-DebuggerTests.EvaluateOnCallFrameTests.WorkItemExecution

Pull Request

dotnet/runtime#89217

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The active test run was aborted. Reason: Test host process crashed",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

  • Feature changes/additions
  • Bug fixes
  • Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Report

Build Definition Step Name Console log Pull Request
2631578 dotnet-interactive Test / Blame Log
931325 dotnet/roslyn Run Integration Tests Log
930761 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76932
928955 dotnet/roslyn Run Integration Tests Log
928927 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76918
928738 dotnet/roslyn Run Integration Tests Log
927094 dotnet/roslyn Run Integration Tests Log
2625793 dotnet-interactive Test / Blame Log
924837 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76852
2625489 dotnet-interactive Test / Blame Log
2625493 dotnet-interactive Test / Blame Log
923681 dotnet/roslyn Run Integration Tests Log
923659 dotnet/roslyn Run Integration Tests Log
921574 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#75581
2620937 dotnet-interactive Test / Blame Log
2620694 dotnet-interactive Test / Blame Log
917698 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76752
2619106 dotnet-interactive Test / Blame Log
914993 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76725
914308 dotnet/roslyn Run Integration Tests Log
914177 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76692
913987 dotnet/roslyn Run Integration Tests Log
912714 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76695
2616802 dotnet-interactive Test / Blame Log
2616717 dotnet-interactive Test / Blame Log
2615694 dotnet-fsharp CmdLine@2 Log
2615469 dotnet-interactive Test / Blame Log
2614768 dotnet-interactive Test / Blame Log
909122 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#75970
2614588 dotnet-fsharp CmdLine@2 Log
908618 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76649
908555 dotnet/roslyn Run Integration Tests Log
908425 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76646
2614116 dotnet-interactive Test / Blame Log
2614101 dotnet-interactive Test / Blame Log
2613990 dotnet-interactive Test / Blame Log
2613544 dotnet-interactive Test / Blame Log
2612493 dotnet-interactive Test / Blame Log
906308 dotnet/roslyn Run Integration Tests Log
2612433 dotnet-interactive Test / Blame Log
2612333 dotnet-interactive Test / Blame Log
905800 dotnet/roslyn Run Integration Tests Log
2611980 dotnet-interactive Test / Blame Log
2611872 dotnet-interactive Test / Blame Log
2611807 dotnet-interactive Test / Blame Log
905108 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#70455
905102 dotnet/roslyn Run Integration Tests Log
Build Definition Test Pull Request
935914 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46392
935830 dotnet/aspire Aspire.Hosting.Azure.Tests.WorkItemExecution
935795 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46195
935782 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46439
935722 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46370
935714 dotnet/aspire Aspire.Hosting.Azure.Tests.WorkItemExecution dotnet/aspire#7331
935565 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46433
934380 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46409
935016 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46371
935378 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46430
935357 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46411
935366 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46429
935124 dotnet/sdk Microsoft.WebTools.AspireService.Tests.dll.WorkItemExecution dotnet/sdk#46387
935226 dotnet/aspire Aspire.Hosting.SqlServer.Tests.WorkItemExecution dotnet/aspire#7328
935207 dotnet/aspire Aspire.Workload.Tests.BuildAndRunTemplateTests.WorkItemExecution dotnet/aspire#7325
935118 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46313
934910 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46281
934463 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46266
933991 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46366
934424 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46356
935035 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46370
934892 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46420
934766 dotnet/sdk Microsoft.WebTools.AspireService.Tests.dll.WorkItemExecution dotnet/sdk#46327
934780 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46361
934667 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46419
934669 dotnet/aspire Aspire.EndToEnd.Tests.IntegrationServicesTests.Aspire.EndToEnd.Tests.IntegrationServicesTests.VerifyComponentWorks(resourceName: postgres) dotnet/aspire#7329
934531 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46290
934522 dotnet/aspire Aspire.Workload.Tests.BuildAndRunTemplateTests.WorkItemExecution dotnet/aspire#7325
934464 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46358
934469 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46319
934379 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46315
934417 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46410
934374 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46336
934345 dotnet/aspire Aspire.Workload.Tests.BuildAndRunTemplateTests.WorkItemExecution dotnet/aspire#7325
934233 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46290
934328 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46407
934319 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46370
932961 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46358
934235 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46365
934166 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46336
934165 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46350
934155 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46404
934148 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46357
932901 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46356
934081 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46375
934051 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46195
934060 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46348
933804 dotnet/sdk dotnet-watch.Tests.dll.2.WorkItemExecution dotnet/sdk#46346
933593 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46392
933709 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46281
933605 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46331
933581 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46393
933574 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46346
933506 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46390
933362 dotnet/aspnetcore Templates.Test.BaselineTest.Template_Produces_The_Right_Set_Of_FilesAsync
2630707 dotnet-aspnetcore Microsoft.AspNetCore.Mvc.FunctionalTests--net10.0.WorkItemExecution
933367 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46389
933154 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46371
932472 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46375
933036 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46357
932869 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46315
932952 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46318
932941 dotnet/sdk Microsoft.WebTools.AspireService.Tests.dll.WorkItemExecution dotnet/sdk#46386
932880 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46384
932957 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46266
932814 dotnet/aspire Aspire.Hosting.Milvus.Tests.WorkItemExecution dotnet/aspire#7309
932786 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46281
932782 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46352
932772 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46365
932650 dotnet/aspnetcore Microsoft.AspNetCore.Mvc.FunctionalTests--net10.0.WorkItemExecution dotnet/aspnetcore#60080
932669 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46310
932530 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46377
932416 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46374
932408 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46255
932326 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46261
932363 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46372
932263 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46371
932238 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46368
932179 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46365
932122 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46300
932035 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46341
931889 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46327
931529 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46300
931514 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46354
931544 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46358
931583 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45384
931545 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46318
931540 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46357
931486 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#46315
931518 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46355
931440 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46350
931437 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.12.WorkItemExecution dotnet/sdk#45637
931434 dotnet/sdk Microsoft.NET.Build.Tests.dll.11.WorkItemExecution dotnet/sdk#46349
931443 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46336
931321 dotnet/sdk Microsoft.NET.Build.Tests.dll.7.WorkItemExecution dotnet/sdk#46341
931245 dotnet/aspnetcore Microsoft.AspNetCore.Mvc.FunctionalTests--net10.0.WorkItemExecution
931199 dotnet/aspire Aspire.Hosting.Elasticsearch.Tests.WorkItemExecution dotnet/aspire#7232
931173 dotnet/aspire Aspire.Hosting.Elasticsearch.Tests.WorkItemExecution dotnet/aspire#7219
931057 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45520
931151 dotnet/sdk dotnet-watch.Tests.dll.5.WorkItemExecution dotnet/sdk#46183
Displaying 100 of 578 results

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
27 160 625

Known issue validation

Build: 🔎
Result validation: ⚠️ Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: 🔎" line.
Validation performed at: 2/8/2024 9:24:30 PM UTC

@andriipatsula
Copy link
Member

Hello @fanyang-mono, could you please update the "ErrorMessage" : "" by following the step by step documentation on how to create a known issue

@fanyang-mono
Copy link
Member Author

Updated.

@pavelsavara
Copy link
Member

pavelsavara commented Nov 7, 2023

This is duplicate of dotnet/runtime#89409, except here it's not only wasm debugger

@dougbu
Copy link
Member

dougbu commented Nov 23, 2023

@pavelsavara and @fanyang-mono, should this issue be closed in favour of dotnet/runtime#94560❔ if not, the error message isn't sufficiently precise to understand what (if anything) we (DNCEng) could do about it.

@pavelsavara
Copy link
Member

@pavelsavara and @fanyang-mono, should this issue be closed in favour of

You mean dotnet/runtime#89409

I think we should keep this one, because this one has some hits which are not just wasm debugger related.
Here we see other repo failures, on roslyn and aspnetcore CI.

if not, the error message isn't sufficiently precise to understand what (if anything) we (DNCEng) could do about it.

I have not investigated it myself yet. But got questions which could help to narrow it down

  • are there crash dump files on the machine ?
  • are there some hints on what's going on in the log files before crash ?
  • is this OOM or stack overflow in disguise ?
  • did we run out of disk space or number of inodes ?
  • hit some limit in docker container ?
  • is this always the same OS, version, image ?
  • could this be hardware issue ?

@fanyang-mono
Copy link
Member Author

This issue was originally created to track issues happened to wasm debugger tests (belonging to dotnet/runtime repo). However, the error message was generic enough to include issues from other repos, such as dotnet/roslyn and dotnet/aspnetcore. We need insights from owners from dotnet/roslyn and dotnet/aspnetcore to confirm that they are aware of these issue. And if they are okay to close this issue as well. We could close then.

@mitchdenny
Copy link
Member

We just got an instance of this in the dotnet/aspire repo. It wasn't a WASM related failure for us. Just looks like the test process crashed and took a few tests with it.

@nagilson
Copy link
Member

nagilson commented Dec 3, 2024

@marcpopMSFT was working on getting the test infrastructure to generate crash dumps when it times out to help with investigation. I think it was generating the dumps but not yet correctly uploading them to the artifacts.

We are waiting to hear from the infra team on an issue with this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants