[BUG] coverlet report 0% coverage for ASP.NET Core xUnit integration tests using WebApplicationFactory

[Bart-Med]

Describe the bug
Both coverlet.msbuild and coverlet.collector report 0% code coverage when running xUnit integration tests against an ASP.NET Core API using WebApplicationFactory<Program>.
The tests issue real HTTP requests via HttpClient created from the factory. Despite multiple endpoints being exercised, coverage is always reported as zero. If I disable test concurrency/parallelization in xUnit, the code coverage is calculated correctly, so the issue only occurs when the tests are executed concurrently.

To Reproduce
Repro repository: https://github.com/Bart-Med/coverlet-coverage-repro
From the repo root:
msbuild

• dotnet test Demo.Api.IntegrationTests/Demo.Api.IntegrationTests.csproj --logger trx --configuration Debug --no-restore /p:UsePackageReference=true /p:UseExternalPackageReference=true --settings coverlet.runsettings /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura

---

collector

• dotnet test Demo.Api.IntegrationTests/Demo.Api.IntegrationTests.csproj --configuration Debug --no-restore /p:UsePackageReference=true /p:UseExternalPackageReference=true --settings Sources/coverlet.runsettings --collect:"XPlat Code Coverage"

Expected behavior
Non-zero coverage.

Actual behavior
Coverage is reported as 0% in both drivers.

Configuration (please complete the following information):
coverlet.msbuild: 6.0.4
coverlet.collector: 6.0.4
Microsoft.NET.Test.Sdk: 18.0.0
xUnit: xunit.v3 3.2.0, xunit.runner.visualstudio 3.1.5
Target Framework: net8.0
OS / Distro: windows/linux
Architecture: x64

[gbschenkel]

I am getting this issue too, in my case I am targeting a library to net48, netstandard2.0, net8.0 and net10.0, with a 1k tests.

[Bertk]

Thank you for the information.

I updated Demo.Api.IntegrationTests.csproj and started test from CLI without runstettings file.

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <!-- Domyślnie Xunit.DependencyInjection szuka klasy Startup, ale unikamy powtórzeń między projektami w CTRL+T. -->
    <XunitStartupFullName>Demo.Api.IntegrationTests.DemoApiIntegrationTestsStartup</XunitStartupFullName>
    <IsPackable>false</IsPackable>
    <IsTestProject>true</IsTestProject>
  </PropertyGroup>

  <ItemGroup>
    <ProjectReference Include="..\Demo.Api\Demo.Api.csproj" />
  </ItemGroup>

  <ItemGroup>
    <!--<PackageReference Include="coverlet.msbuild" Version="6.0.4"/>--> 
    <!--<PackageReference Include="coverlet.msbuild" Version="8.0.0-preview.16.g266a82ffcb"/>-->
    <PackageReference Include="coverlet.collector" Version="8.0.0-preview.16.g266a82ffcb" />

    <PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" Version="8.0.12" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
    <PackageReference Include="xunit.v3" Version="3.0.0" />
    <PackageReference Include="xunit.runner.visualstudio" Version="3.1.5">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="Xunit.DependencyInjection" Version="11.1.0" />
    <PackageReference Include="Xunit.DependencyInjection.Logging" Version="11.0.0" />
  </ItemGroup>

  <ItemGroup>
    <Using Include="Xunit" />
  </ItemGroup>

</Project>

Command line:

dotnet build -c Debug  
dotnet test Demo.Api.IntegrationTests/Demo.Api.IntegrationTests.csproj --configuration Debug --no-build /p:UsePackageReference=true /p:UseExternalPackageReference=true --collect:"XPlat Code Coverage" -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura,opencover

with vstest setting file

dotnet build -c Debug  
dotnet test Demo.Api.IntegrationTests/Demo.Api.IntegrationTests.csproj --configuration Debug --no-build /p:UsePackageReference=true /p:UseExternalPackageReference=true --collect:"XPlat Code Coverage"  --settings .\coverlet.runsettings -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura,opencover

C:\GitHub\coverlet-issue-1782>dotnet test Demo.Api.IntegrationTests/Demo.Api.IntegrationTests.csproj --configuration Debug --no-build /p:UsePackageReference=true /p:UseExternalPackageReference=true --collect:"XPlat Code Coverage" -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura,opencover
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 8.0.22)
[xUnit.net 00:00:00.18]   Discovering: Demo.Api.IntegrationTests
[xUnit.net 00:00:00.44]   Discovered:  Demo.Api.IntegrationTests
[xUnit.net 00:00:00.69]   Starting:    Demo.Api.IntegrationTests
[xUnit.net 00:00:01.41]   Finished:    Demo.Api.IntegrationTests (ID = 'acd5d3ab4de2c68a6228384e4d6a663aa9fd961733b1d32e6943bff5385a9e72')
  Demo.Api.IntegrationTests test net8.0 succeeded (18,0s)

Test summary: total: 62; failed: 0; succeeded: 62; skipped: 0; duration: 18,0s
Build succeeded in 18,4s

Attachments:
  C:\GitHub\coverlet-issue-1782\Demo.Api.IntegrationTests\TestResults\bb727863-f2f1-4d21-8c17-5e8870258915\coverage.cobertura.xml
  C:\GitHub\coverlet-issue-1782\Demo.Api.IntegrationTests\TestResults\bb727863-f2f1-4d21-8c17-5e8870258915\coverage.opencover.xml

coverage.cobertura.zip

I always build the test before executing running it.

dotnet --list-sdks
8.0.416 [C:\Program Files\dotnet\sdk]
9.0.308 [C:\Program Files\dotnet\sdk]
10.0.100 [C:\Program Files\dotnet\sdk]