Flaky CI tests issues that relating to dotMemory/dotTrace

[filzrev]

DotMemorySmokeTest and DotTraceSmokeTest are frequently failed on CI.

As far as I've confirmed it failed with following errors.

1. VSTest process crashed by unknown reason. (No exception is recorded)
2. Failed to download dependencies.
3. Failed with attach/detach timeout error. (Default: 30 seconds that can be customized with UseCustomResponseTimeout)
   4. Failed with following exception

   System.InvalidOperationException: The profiling session isn't active: forgot to call Attach()?
   at JetBrains.Profiler.SelfApi.DotMemory.GetSnapshot(String name)
   at BenchmarkDotNet.Diagnosers.SnapshotProfilerBase.Stop(ILogger logger) in /_/src/BenchmarkDotNet/Diagnosers/SnapshotProfilerBase.cs:line 144
   

   Above exception occurred when failed with 2. or 3.

[filzrev]

1. VSTest process crashed by unknown reason. (No exception is recorded)

I've tested following steps.
But no useful information found.

1. Gets VSTest diag log with --diag:/home/runner/work/BenchmarkDotNet/BenchmarkDotNet/artifacts/log.txt
2. Gets crash dump file with --blame-crash options.
3. Gets dotMemory diag logs with following code.

   .UseLogLevelVerbose()
   .UseLogFile(Environment.GetEnvironmentVariable("GITHUB_WORKSPACE") + $"/artifacts/dotMemory_{DateTime.Now.ToString("HHmmss_fff")}.log")
   

As far as I've confirmed log files/crash dump file.
VSTest process crash reason is not recorded on logs. and when when crashed occurred.
BenchmarkDotNet is executing code at random location and it seems not causing process crash.

Then, I've separate DotMemoryTests to separate projects and migrated to xUnit v3 with MTPv2.
And confirmed process crash issue is continued.
So it's expected not VSTest specific issue.

Currently it seems happens when using InProcessEmitToolchain or InProcessNoEmitToolchain
and not happens when using CsProjCoreToolchain.

[filzrev]

This error seems be happens when ExecuteOnSeparateThread is enabled on InProcess toolchain.
When disable ExecuteOnSeparateThread, process crash issue is not observed.

So it's expected to be caused by some threading related issue.

From dump file. It's expected crash occurred on waiting Attach with JetBrains.Profiler.Api.
https://github.com/JetBrains/profiler-self-api/blob/02f8410d26c184cb50ddaead6fcce89d7f34517c/JetBrains.Profiler.SelfApi/src/Impl/ConsoleProfiler.cs#L214-L223

Dump StackTrace

Not Flagged		6170	15	Worker Thread	Background	JetBrains.Profiler.SelfApi.dll!JetBrains.Profiler.SelfApi.Impl.ConsoleProfiler.AwaitConnected

[Managed to Native Transition]
System.Private.CoreLib.dll!System.Threading.Thread.Sleep(int millisecondsTimeout) Line 634
JetBrains.Profiler.SelfApi.dll!JetBrains.Profiler.SelfApi.Impl.ConsoleProfiler.AwaitConnected(int milliseconds)
JetBrains.Profiler.SelfApi.dll!JetBrains.Profiler.SelfApi.DotMemory.Attach(JetBrains.Profiler.SelfApi.DotMemory.Config config)
BenchmarkDotNet.dll!BenchmarkDotNet.Diagnosers.SnapshotProfilerBase.Start(BenchmarkDotNet.Loggers.ILogger logger, BenchmarkDotNet.Diagnosers.DiagnoserActionParameters parameters)
BenchmarkDotNet.dll!BenchmarkDotNet.Diagnosers.SnapshotProfilerBase.Handle(BenchmarkDotNet.Engines.HostSignal signal, BenchmarkDotNet.Diagnosers.DiagnoserActionParameters parameters)
BenchmarkDotNet.dll!BenchmarkDotNet.Diagnosers.CompositeDiagnoser.Handle(BenchmarkDotNet.Engines.HostSignal signal, BenchmarkDotNet.Diagnosers.DiagnoserActionParameters parameters)
BenchmarkDotNet.dll!BenchmarkDotNet.Engines.Engine.RunCore()
BenchmarkDotNet.dll!BenchmarkDotNet.Engines.Engine.Run()
BenchmarkDotNet.dll!BenchmarkDotNet.Toolchains.InProcess.NoEmit.InProcessNoEmitRunner.Runnable.RunCore(BenchmarkDotNet.Engines.IHost host, BenchmarkDotNet.Toolchains.Parameters.ExecuteParameters parameters)
System.Private.CoreLib.dll!System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, object[] parameters, System.Globalization.CultureInfo culture) Line 143
BenchmarkDotNet.dll!BenchmarkDotNet.Toolchains.InProcess.NoEmit.InProcessNoEmitRunner.Run(BenchmarkDotNet.Engines.IHost host, BenchmarkDotNet.Toolchains.Parameters.ExecuteParameters parameters)
BenchmarkDotNet.dll!BenchmarkDotNet.Toolchains.InProcess.NoEmit.InProcessNoEmitExecutor.ExecuteCore(BenchmarkDotNet.Engines.IHost host, BenchmarkDotNet.Toolchains.Parameters.ExecuteParameters parameters)
BenchmarkDotNet.dll!BenchmarkDotNet.Toolchains.InProcess.NoEmit.InProcessNoEmitExecutor.Execute.AnonymousMethod__0()
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 138

Currently it seems happens when using InProcessEmitToolchain or InProcessNoEmitToolchain
and not happens when using CsProjCoreToolchain.

When using CsProjCoreToolchain.
It's don't use JetBrains.Profiler.Api and use dotMemory command line API instead
So crash issue seems not happens on CsProjCoreToolchain.

[filzrev]

I've recently encountered DotTraceSmokeTest failure with following errors. (On windows-latest)

2026-01-29T20:01:08.0938861Z [xUnit.net 00:21:23.90]     DotTraceSmokeTest [FAIL]
2026-01-29T20:01:08.0941813Z [xUnit.net 00:21:23.90]       Assert.Single() Failure: The collection was empty 
...
2026-01-29T20:01:08.1016172Z [xUnit.net 00:21:23.90]         Attaching dotTrace to the process...
2026-01-29T20:01:08.1016517Z [xUnit.net 00:21:23.90]         dotTrace is successfully attached
2026-01-29T20:01:08.1016888Z [xUnit.net 00:21:23.90]         No Workload Results were obtained from the run.
2026-01-29T20:01:08.1017318Z [xUnit.net 00:21:23.90]         // Benchmark Process 7716 has exited with code -1073741819.

It seems -1073741819 is relating to Access Denied on file system.

[filzrev]

I've recently encountered -1073741819 Access Denied exit code multiple times on CI.

dotMemory/dotTrace tests might be better move dotMemory/dotTrace tests to ManualRunning project.

ErrorLog

2026-02-27T09:06:38.8539409Z [xUnit.net 00:21:53.74]         Attaching dotTrace to the process...
2026-02-27T09:06:38.8539754Z [xUnit.net 00:21:53.74]         dotTrace is successfully attached
2026-02-27T09:06:38.8540119Z [xUnit.net 00:21:53.74]         ProcessData operation is interrupted by EndOfStream.
2026-02-27T09:06:38.8540647Z [xUnit.net 00:21:53.74]         No Workload Results were obtained from the run.
2026-02-27T09:06:38.8541053Z [xUnit.net 00:21:53.74]         // Benchmark Process 3572 has exited with code -1073741819.