Multiple nuget warnings/errors after moving to 0.13.6 breaking the build

[olstakh]

When moving to the latest 0.13.6 benchmark version - we observe new nuget warnings that break our builds

Minimal repro: https://github.com/olstakh/BenchmarkBuildFailure/tree/main

Packages.Prop file has BenchmarkVersion variable defined, set to 0.13.5

When running

dotnet run -c Release --filter *

from the root - no warnings/errors, benchmark is executed, report is produced

After changing BenchmarkVersion in Packages.Prop file to 0.13.6 and running the same command:

C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency BenchmarkDotNet does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency BenchmarkDotNet.Diagnostics.Windows does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency coverlet.collector does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency FluentAssertions does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency Microsoft.AspNetCore.Mvc.Testing does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency Microsoft.Internal.Analyzers does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency Microsoft.Internal.Analyzers.Rulesets does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency Microsoft.NET.Test.Sdk does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency Microsoft.VisualStudio.Threading.Analyzers does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency Moq does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency Polly does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency StyleCop.Analyzers does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency xunit does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency xunit.categories does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency xunit.runner.visualstudio does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1701: Package 'BenchmarkDotNet 0.5.0' was restored using '.NETFramework,Version=v4.6.1, .NETFramework,Version=v4.6.2, .NETFramework,Version=v4.7, .NETFramework,Version=v4.7.1, .NETFramework,Version=v4.7.2, .NETFramework,Version=v4.8, .NETFramework,Version=v4.8.1' instead of the project target framework 'net6.0'. This package may not be fully compatible with your project.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1701: Package 'BenchmarkDotNet.Diagnostics.Windows 0.9.6' was restored using '.NETFramework,Version=v4.6.1, .NETFramework,Version=v4.6.2, .NETFramework,Version=v4.7, .NETFramework,Version=v4.7.1, .NETFramework,Version=v4.7.2, .NETFramework,Version=v4.8, .NETFramework,Version=v4.8.1' instead of the project target framework 'net6.0'. This package may not be fully compatible with your project.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1701: Package 'Moq 1.0.0' was restored using '.NETFramework,Version=v4.6.1, .NETFramework,Version=v4.6.2, .NETFramework,Version=v4.7, .NETFramework,Version=v4.7.1, .NETFramework,Version=v4.7.2, .NETFramework,Version=v4.8, .NETFramework,Version=v4.8.1' instead of the project target framework 'net6.0'. This package may not be fully compatible with your project.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1701: Package 'Polly 1.0.0' was restored using '.NETFramework,Version=v4.6.1, .NETFramework,Version=v4.6.2, .NETFramework,Version=v4.7, .NETFramework,Version=v4.7.1, .NETFramework,Version=v4.7.2, .NETFramework,Version=v4.8, .NETFramework,Version=v4.8.1' instead of the project target framework 'net6.0'. This package may not be fully compatible with your project.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1701: Package 'xunit 1.7.0.1540' was restored using '.NETFramework,Version=v4.6.1, .NETFramework,Version=v4.6.2, .NETFramework,Version=v4.7, .NETFramework,Version=v4.7.1, .NETFramework,Version=v4.7.2, .NETFramework,Version=v4.8, .NETFramework,Version=v4.8.1' instead of the project target framework 'net6.0'. This package may not be fully compatible with your project.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1701: Package 'xunit.categories 1.0.0' was restored using '.NETFramework,Version=v4.6.1, .NETFramework,Version=v4.6.2, .NETFramework,Version=v4.7, .NETFramework,Version=v4.7.1, .NETFramework,Version=v4.7.2, .NETFramework,Version=v4.8, .NETFramework,Version=v4.8.1' instead of the project target framework 'net6.0'. This package may not be fully compatible with your project.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : warning NU1701: Package 'xunit.runner.visualstudio 0.99.2' was restored using '.NETFramework,Version=v4.6.1, .NETFramework,Version=v4.6.2, .NETFramework,Version=v4.7, .NETFramework,Version=v4.7.1, .NETFramework,Version=v4.7.2, .NETFramework,Version=v4.8, .NETFramework,Version=v4.8.1' instead of the project target framework 'net6.0'. This package may not be fully compatible with your project.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : error NU1202: Package FluentAssertions 1.3.0.1 is not compatible with net6.0 (.NETCoreApp,Version=v6.0). Package FluentAssertions 1.3.0.1 supports:
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : error NU1202:   - netframework35 (.NetFramework 3.5,Version=v0.0)
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : error NU1202:   - netframework40 (.NetFramework 4.0,Version=v0.0)
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj : error NU1202:   - silverlight40 (Silverlight 4.0,Version=v0.0)
  Failed to restore C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f\BenchmarkDotNet.Autogenerated.csproj (in 1.05 sec).
  1 of 2 projects are up-to-date for restore.

// BenchmarkDotNet has failed to build the auto-generated boilerplate code.
// It can be found in C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\025b51cf-82de-473c-b958-35a851effb1f
// Please follow the troubleshooting guide: https://benchmarkdotnet.org/articles/guides/troubleshooting.html

The final error NU1202 seems to be the one breaking the build (with a package version that we're not referencing).

The above errors do not exist with 0.13.5.
when providing --logBuildOutput switch with 0.13.5 version, below is what displayed:

C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\a6524095-fe67-4ed3-a217-ee169fe27529\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency Microsoft.Internal.Analyzers does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\a6524095-fe67-4ed3-a217-ee169fe27529\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency Microsoft.Internal.Analyzers.Rulesets does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\a6524095-fe67-4ed3-a217-ee169fe27529\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency Microsoft.VisualStudio.Threading.Analyzers does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\a6524095-fe67-4ed3-a217-ee169fe27529\BenchmarkDotNet.Autogenerated.csproj : warning NU1604: Project dependency StyleCop.Analyzers does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
  Restored C:\code\BenchmarkBuildRepo\out\Release-x64\Benchmark\net6.0\a6524095-fe67-4ed3-a217-ee169fe27529\BenchmarkDotNet.Autogenerated.csproj (in 836 ms).
  1 of 2 projects are up-to-date for restore.

[timcassell]

@olstakh Can you please try the latest nightly feed? https://benchmarkdotnet.org/articles/guides/nuget.html#nightly

BenchmarkDotNet.0.13.7-nightly.20230721

[olstakh]

@timcassell , i had to change your proposed version to 0.13.7-nightly.20230721**.42**, and it's working!
For my learning purposes - where can i learn what was the issue in 13.6 and how it was fixed?
Also, im assuming there's no good workaround meanwhile? We can't reference this nuget feed from our projects

[timcassell]

I suspect it was due to #2369. You can continue to use 0.13.5 until 0.13.7 is released.

[olstakh]

@timcassell , understood. the issue you mentioned doesn't seem to be related, the variable was used for convenience only, the repro works without it.
Is there a way to see what nightly versions builds are? I might try to binary search to determine which version fixed the issue and reverse engineer from that

[timcassell]

I'm not sure, but you can clone the repo and reference it locally.

[AndreyAkinshin]

@olstakh the full list of available nightly packages can be found at the bottom of this page: https://www.myget.org/feed/benchmarkdotnet/package/nuget/BenchmarkDotNet
The last number of the nightly version matches the build number of the publish-nightly workflow: https://github.com/dotnet/BenchmarkDotNet/actions/workflows/publish-nightly.yaml so that you can easily find the relevant commit.

[olstakh]

Thanks, @AndreyAkinshin ! I found the version that was not failing with this repo any more, it's 0.13.7-nightly.20230717.35
Cross checking with the other link, am i correct in correlating this commit to the fix then?
And again, double checking, due to the nature of this commit, im assuming there are no workarounds meanwhile? The reason im asking is because we're affected by #2264 in 0.13.5

[timcassell]

The only workaround would be to remove all properties from <PackageReference>es, including in imported props/targets/projs. That's probably impossible in some cases.

Expect 0.13.7 to be released next week.