Skip to content
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .config/dotnet-tools.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@
"isRoot": true,
"tools": {
"dotnet-reportgenerator-globaltool": {
"version": "5.4.3",
"version": "5.4.7",
"commands": [
"reportgenerator"
],
"rollForward": false
}
}
}
}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -319,4 +319,5 @@ FolderProfile.pubxml
nuget.config
*.dmp
Playground*/
# extended playground
coverlet.MTP/
6 changes: 3 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ Contributions are highly welcome, however, except for very small changes, kindly

## Requirements

[.NET SDK 6.0](https://dotnet.microsoft.com/en-us/download/dotnet/6.0)
[.NET SDK 7.0](https://dotnet.microsoft.com/en-us/download/dotnet/7.0)
[.NET SDK 8.0](https://dotnet.microsoft.com/en-us/download/dotnet/8.0)
STS version [.NET SDK 9.0](https://dotnet.microsoft.com/en-us/download/dotnet/9.0) for development environment

LTS version [.NET SDK 8.0](https://dotnet.microsoft.com/en-us/download/dotnet/8.0) for runtime environment

## Building the Project

Expand Down
31 changes: 0 additions & 31 deletions Directory.Build.props
Original file line number Diff line number Diff line change
Expand Up @@ -48,35 +48,4 @@
<VSTestLogger>@(VSTestLogger)</VSTestLogger>
</PropertyGroup>

<PropertyGroup>
<!--
Do not change versions since we need to support VSTest DataCollectors. We need to load assembly version 1.6.0 to properly work
We can check minimum supported package version here https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Microsoft.TestPlatform.ObjectModel.csproj#L34
-->
<!-- VSTest dependency versions -->
<!-- https://github.com/microsoft/vstest/blob/fde8bf79d3f0f80e3548f873a56ffb4100c0ae49/eng/Versions.props#L22C1-L54C2 -->
<MicrosoftBuildUtilitiesCorePackageVersion>15.9.20</MicrosoftBuildUtilitiesCorePackageVersion>
<SystemReflectionMetadataVersion>1.6.0</SystemReflectionMetadataVersion>
<SystemCollectionsImmutableVersion>1.5.0</SystemCollectionsImmutableVersion>

<!-- some additional information about dependency management limitations for MSBuild tasks (coverlet.msbuild.tasks)
- msBuild task with dependencies https://natemcmaster.com/blog/2017/11/11/msbuild-task-with-dependencies/
- Aligning versions with MSBuild
- If you need a dependency that is also used in MSBuild itself, you have to align with the version MSBuild uses.
- This includes System.Reflection.Metadata, System.Collections.Immutable, NuGet, Newtonsoft.Json, and others.
- See example: dependencies.props (https://github.com/aspnet/BuildTools/blob/1f3f14382764e06b7e691e5ee89d12a280249284/build/dependencies.props#L19-L29)"

.NET 8.0.3xx (MSBuild) core-sdk-tasks version details https://github.com/dotnet/installer/blob/release/8.0.3xx/src/core-sdk-tasks/core-sdk-tasks.csproj

ALIGN PACKAGE VERSIONS of coverlet.core with coverlet.collector (vstest) and coverlet.msbuild.tasks (.NET SDK MSBuild)

e.g. candidates for version conflicts:
<MicrosoftBuildTaskVersion>15.7.179</MicrosoftBuildTaskVersion>
<MicrosoftBuildTaskUtilitiesCoreVersion>15.7.179</MicrosoftBuildTaskUtilitiesCoreVersion>
<NuGetBuildTasksPackageVersion>6.9.0-rc.86</NuGetBuildTasksPackageVersion>
<MicrosoftBuildTaskSystemReflectionMetaData>1.4.2</MicrosoftBuildTaskSystemReflectionMetaData>
<MicrosoftBuildTaskSystemCollectionImmutable>1.5.0</MicrosoftBuildTaskSystemCollectionImmutable> >= 1.3.1
-->
</PropertyGroup>

</Project>
14 changes: 8 additions & 6 deletions Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,16 @@
<CentralPackageTransitivePinningEnabled>true</CentralPackageTransitivePinningEnabled>
</PropertyGroup>
<ItemGroup>
<GlobalPackageReference Include="Microsoft.SourceLink.GitHub" Version="8.0.0" />
<GlobalPackageReference Include="Nerdbank.GitVersioning" Version="3.7.115" />
</ItemGroup>
<PropertyGroup>
<MicrosoftBuildVersion>17.11.31</MicrosoftBuildVersion> <!-- .NET 8.0 support -->
<MicrosoftCodeAnalysisVersion>4.12.0</MicrosoftCodeAnalysisVersion>
<NugetPackageVersion>6.14.0</NugetPackageVersion>
<!-- Test Platform, .NET Test SDK and Object Model -->
<MicrosoftNETTestSdkVersion>17.14.1</MicrosoftNETTestSdkVersion>
<NugetPackageVersion>6.14.0</NugetPackageVersion>
<XunitV3Version>2.0.0</XunitV3Version>
<XunitRunnerVisualstudioVersion>3.0.2</XunitRunnerVisualstudioVersion>
<XunitV3Version>3.0.0</XunitV3Version>
<XunitRunnerVisualstudioVersion>3.1.3</XunitRunnerVisualstudioVersion>
</PropertyGroup>
<ItemGroup>
<PackageVersion Include="DotNetConfig" Version="1.2.0" />
Expand All @@ -39,7 +38,6 @@
vstest 17.8 version
NuGetFrameworksVersion is defined here https://github.com/microsoft/vstest/blob/9a0c41811637edf4afe0e265e08fdd1cb18109ed/eng/Versions.props#L94C1-L94C1
-->
<PackageVersion Include="Newtonsoft.Json" Version="13.0.3" />
<PackageVersion Include="NuGet.Frameworks" Version="$(NugetPackageVersion)" />
<PackageVersion Include="NuGet.Packaging" Version="$(NugetPackageVersion)" />
<PackageVersion Include="NuGet.Versioning" Version="$(NugetPackageVersion)" />
Expand All @@ -57,14 +55,18 @@
<PackageVersion Include="System.Buffers" Version="4.6.1" />
<PackageVersion Include="System.Collections.Immutable" Version="8.0.0" />
<PackageVersion Include="System.Configuration.ConfigurationManager" Version="8.0.1" />
<PackageVersion Include="System.Diagnostics.EventLog" Version="8.0.2" />
<PackageVersion Include="System.Formats.Asn1" Version="8.0.2" />
<PackageVersion Include="System.IO.Pipelines" Version="8.0.0" />
<PackageVersion Include="System.Linq.Async" Version="6.0.3" />
<PackageVersion Include="System.Memory" Version="4.6.3" />
<PackageVersion Include="System.Net.Http" Version="4.3.4" />
<PackageVersion Include="System.Reflection.Metadata" Version="8.0.1" />
<PackageVersion Include="System.Runtime.CompilerServices.Unsafe" Version="6.1.2" />
<PackageVersion Include="System.Security.AccessControl" Version="6.0.1" />
<PackageVersion Include="System.Security.Cryptography.Pkcs" Version="8.0.1" />
<PackageVersion Include="System.Text.Encoding.CodePages" Version="8.0.0" />
<PackageVersion Include="System.Text.Json" Version="8.0.6" /> <!-- used by tests -->
<PackageVersion Include="System.Text.Json" Version="8.0.6" />
<PackageVersion Include="System.Text.RegularExpressions" Version="4.3.1" />
<PackageVersion Include="System.Threading.Tasks.Extensions" Version="4.6.3" />
</ItemGroup>
Expand Down
5 changes: 4 additions & 1 deletion Documentation/Changelog.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
Expand All @@ -7,12 +8,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## Unreleased

### Fixed

- Fix branchpoint exclusion for sdk 8.0.407 [#1741](https://github.com/coverlet-coverage/coverlet/issues/1741)
- Use `<TargetFramework>netstandard2.0</TargetFramework>` for _coverlet.collector_ and _coverlet.msbuild.tasks_ Packages´

### Improvements
- Use [xunit.v3](https://xunit.net/docs/getting-started/v3/whats-new) for tests and example code

- Upgraded minimum required .NET SDK and runtime to .NET 8.0 LTS (Long Term Support) (**Breaking Change**)
- Use [xunit.v3](https://xunit.net/docs/getting-started/v3/whats-new) for tests and example code

## Release date 2024-01-20
### Packages
Expand Down
100 changes: 26 additions & 74 deletions Documentation/Examples/MSBuild/DeterministicBuild/HowTo.md
Original file line number Diff line number Diff line change
@@ -1,57 +1,34 @@
To run test we need to generates packages to reference in on test project.
Run from repo root
# Using Deterministic Builds with Coverlet MSBuild Integration

## Prerequisites

Before running tests with deterministic builds, you need to generate the local NuGet packages:

```shell
C:\git\coverlet
λ dotnet pack
Microsoft (R) Build Engine version 17.7.4+3ebbd7c49 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

Restore completed in 73,36 ms for C:\git\coverlet\src\coverlet.core\coverlet.core.csproj.
Restore completed in 73,41 ms for C:\git\coverlet\test\coverlet.testsubject\coverlet.testsubject.csproj.
Restore completed in 73,33 ms for C:\git\coverlet\test\coverlet.tests.projectsample.excludedbyattribute\coverlet.tests.projectsample.excludedbyattribute.csproj.
Restore completed in 73,34 ms for C:\git\coverlet\src\coverlet.collector\coverlet.collector.csproj.
Restore completed in 73,35 ms for C:\git\coverlet\test\coverlet.tests.xunit.extensions\coverlet.tests.xunit.extensions.csproj.
Restore completed in 75,92 ms for C:\git\coverlet\test\coverlet.integration.tests\coverlet.integration.tests.csproj.
Restore completed in 73,41 ms for C:\git\coverlet\src\coverlet.console\coverlet.console.csproj.
Restore completed in 73,36 ms for C:\git\coverlet\test\coverlet.tests.projectsample.empty\coverlet.tests.projectsample.empty.csproj.
Restore completed in 73,47 ms for C:\git\coverlet\src\coverlet.msbuild.tasks\coverlet.msbuild.tasks.csproj.
Restore completed in 73,37 ms for C:\git\coverlet\test\coverlet.core.tests.samples.netstandard\coverlet.core.tests.samples.netstandard.csproj.
Restore completed in 76,37 ms for C:\git\coverlet\test\coverlet.collector.tests\coverlet.collector.tests.csproj.
Restore completed in 77,05 ms for C:\git\coverlet\test\coverlet.integration.template\coverlet.integration.template.csproj.
Restore completed in 77,2 ms for C:\git\coverlet\test\coverlet.core.performancetest\coverlet.core.performancetest.csproj.
Restore completed in 87,7 ms for C:\git\coverlet\test\coverlet.core.tests\coverlet.core.tests.csproj.
coverlet.core -> C:\git\coverlet\src\coverlet.core\bin\Debug\netstandard2.0\coverlet.core.dll
coverlet.collector -> C:\git\coverlet\src\coverlet.collector\bin\Debug\netcoreapp2.0\coverlet.collector.dll
coverlet.msbuild.tasks -> C:\git\coverlet\src\coverlet.msbuild.tasks\bin\Debug\netstandard2.0\coverlet.msbuild.tasks.dll
coverlet.console -> C:\git\coverlet\src\coverlet.console\bin\Debug\net6.0\coverlet.console.dll
Successfully created package 'C:\git\coverlet\bin\Debug\Packages\coverlet.collector.6.0.1-preview.6.g918cd179e0.nupkg'.
Successfully created package 'C:\git\coverlet\bin\Debug\Packages\coverlet.collector.6.0.1-preview.6.g918cd179e0.snupkg'.
Successfully created package 'C:\git\coverlet\bin\Debug\Packages\coverlet.msbuild.6.0.1-preview.6.g918cd179e0.nupkg'.
Successfully created package 'C:\git\coverlet\bin\Debug\Packages\coverlet.msbuild.6.0.1-preview.6.g918cd179e0.snupkg'.
Successfully created package 'C:\git\coverlet\bin\Debug\Packages\coverlet.console.6.0.1-preview.6.g918cd179e0.nupkg'.
Successfully created package 'C:\git\coverlet\bin\Debug\Packages\coverlet.console.6.0.1-preview.6.g918cd179e0.snupkg'.
# Run from repository root
dotnet pack
```

Add msbuild package version generated to `"..\Documentation\Examples\MSBuild\DeterministicBuild\XUnitTestProject1\XUnitTestProject1.csproj"`
## Project Setup

Update your test project file `XUnitTestProject1.csproj`:

```xml
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<TargetFramework>net8.0</TargetFramework>
<IsPackable>false</IsPackable>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.13.0" />
<PackageReference Include="xunit.v3" Version="1.1.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.0.2">
<PackageReference Include="xunit.v3" Version="2.0.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.0">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
</PackageReference>
<!-- version comes from coverlet.msbuild.6.0.1-preview.8.gcb9b802a5f.nupkg -->
<PackageReference Include="coverlet.msbuild" Version="coverlet.msbuild.6.0.1-preview.8.gcb9b802a5f.nupkg" >
<!-- Use the version from your locally built package -->
<PackageReference Include="coverlet.msbuild" Version="8.0.1-preview.8.gcb9b802a5f">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
Expand All @@ -60,50 +37,25 @@ Add msbuild package version generated to `"..\Documentation\Examples\MSBuild\Det
<ItemGroup>
<ProjectReference Include="..\ClassLibrary1\ClassLibrary1.csproj" />
</ItemGroup>

</Project>
```

Go to test project folder and run
## Running Tests

Navigate to your test project directory and run:

```shell
C:\git\coverlet\Documentation\Examples\MSBuild\DeterministicBuild (detbuilddocs -> origin)
λ dotnet test /p:CollectCoverage=true /p:DeterministicSourcePaths=true
Test run for C:\git\coverlet\Documentation\Examples\MSBuild\DeterministicBuild\XUnitTestProject1\bin\Debug\net6.0\XUnitTestProject1.dll(.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.5.0
Copyright (c) Microsoft Corporation. All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern.

Test Run Successful.
Total tests: 1
Passed: 1
Total time: 1,1471 Seconds

Calculating coverage result...
Generating report 'C:\git\coverlet\Documentation\Examples\MSBuild\DeterministicBuild\XUnitTestProject1\coverage.json'

+---------------+------+--------+--------+
| Module | Line | Branch | Method |
+---------------+------+--------+--------+
| ClassLibrary1 | 100% | 100% | 100% |
+---------------+------+--------+--------+

+---------+------+--------+--------+
| | Line | Branch | Method |
+---------+------+--------+--------+
| Total | 100% | 100% | 100% |
+---------+------+--------+--------+
| Average | 100% | 100% | 100% |
+---------+------+--------+--------+
dotnet test /p:CollectCoverage=true /p:DeterministicSourcePaths=true
```

You should see on output folder the coverlet source root mapping file generated. The filename starts with 'CoverletSourceRootsMapping_'. Do not use `--no-build` option
This is the confirmation that you're running coverage on deterministic build e.g. `Documentation\Examples\MSBuild\DeterministicBuild\XUnitTestProject1\bin\Debug\net6.0\CoverletSourceRootsMapping_XUnitTestProject1`
> **Important**: Do not use the `--no-build` option as it will prevent the generation of deterministic build artifacts.

## Verification

After running the tests, verify the deterministic build by checking for the source root mapping file:

```text
Documentation\Examples\MSBuild\DeterministicBuild\XUnitTestProject1\bin\Debug\net6.0\CoverletSourceRootsMapping_XUnitTestProject1
bin\Debug\net8.0\CoverletSourceRootsMapping_XUnitTestProject1
```

The presence of this file confirms that your coverage report was generated with deterministic build settings.
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<TargetFramework>net8.0</TargetFramework>
<IsPackable>false</IsPackable>
<ManagePackageVersionsCentrally>false</ManagePackageVersionsCentrally>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.13.0" />
<PackageReference Include="xunit.v3" Version="1.1.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.0.2">
<PackageReference Include="xunit.v3" Version="2.0.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.0">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
</PackageReference>
Expand Down
61 changes: 47 additions & 14 deletions Documentation/Examples/MSBuild/MergeWith/HowTo.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,57 @@
**Run from solution root sln**
# Merging Coverage Reports

To merge report together you need to run separate test and merge in one `json` format file.
Last command will join and create final needed format file.
## Running Tests Separately

```shell
dotnet test XUnitTestProject1\XUnitTestProject1.csproj /p:CollectCoverage=true /p:CoverletOutput=../CoverageResults/
dotnet test XUnitTestProject2\XUnitTestProject2.csproj /p:CollectCoverage=true /p:CoverletOutput=../CoverageResults/ /p:MergeWith="../CoverageResults/coverage.json"
dotnet test XUnitTestProject3\XUnitTestProject3.csproj /p:CollectCoverage=true /p:CoverletOutput=../CoverageResults/ /p:MergeWith="../CoverageResults/coverage.json" /p:CoverletOutputFormat="opencover"
To merge coverage reports, run tests for each project and combine them into a single file:

```bash
# Generate coverage for first project
dotnet test XUnitTestProject1/XUnitTestProject1.csproj \
/p:CollectCoverage=true \
/p:CoverletOutput=../CoverageResults/

# Merge coverage from second project
dotnet test XUnitTestProject2/XUnitTestProject2.csproj \
/p:CollectCoverage=true \
/p:CoverletOutput=../CoverageResults/ \
/p:MergeWith="../CoverageResults/coverage.json"

# Merge coverage from third project and generate final OpenCover report
dotnet test XUnitTestProject3/XUnitTestProject3.csproj \
/p:CollectCoverage=true \
/p:CoverletOutput=../CoverageResults/ \
/p:MergeWith="../CoverageResults/coverage.json" \
/p:CoverletOutputFormat="opencover"
```

You can merge also running `dotnet test` and merge with single command from a solution file, but you need to ensure that tests will run sequentially(`-m:1`). This slow down testing but avoid invalid coverage result.
## Running Tests from Solution

```shell
dotnet test /p:CollectCoverage=true /p:CoverletOutput=../CoverageResults/ /p:MergeWith="../CoverageResults/coverage.json" /p:CoverletOutputFormat=\"opencover,json\" -m:1
To merge coverage using a single command (requires sequential execution):

```bash
dotnet test \
/p:CollectCoverage=true \
/p:CoverletOutput=../CoverageResults/ \
/p:MergeWith="../CoverageResults/coverage.json" \
/p:CoverletOutputFormat="opencover,json" \
-m:1
```

N.B. You need to specify `json` format plus another format(the final one), because Coverlet can only merge proprietary format. At the end you can delete temporary `coverage.json` file.
> **Note**: Sequential execution (`-m:1`) ensures accurate coverage but increases test duration.

## Important Considerations

- Include `json` format alongside your desired output format
- Coverlet only merges its proprietary JSON format
- The temporary `coverage.json` file can be deleted after merging

You can also merge the coverage result and generate another valid format to export the content than opencover, like cobertura.
## Example with Cobertura Output

```shell
dotnet test /p:CollectCoverage=true /p:CoverletOutput=../CoverageResults/ /p:MergeWith="../CoverageResults/coverage.json" /p:CoverletOutputFormat=\"cobertura,json\" -m:1
```bash
dotnet test \
/p:CollectCoverage=true \
/p:CoverletOutput=../CoverageResults/ \
/p:MergeWith="../CoverageResults/coverage.json" \
/p:CoverletOutputFormat="cobertura,json" \
-m:1
```
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<TargetFramework>net8.0</TargetFramework>
<IsPackable>false</IsPackable>
<ManagePackageVersionsCentrally>false</ManagePackageVersionsCentrally>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="coverlet.msbuild" Version="6.0.0">
<PackageReference Include="coverlet.msbuild" Version="8.0.0">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
Expand Down
Loading