Manual updates 20250318 nuget.config for new pipelines

[moljac]

1. Use of Nuget.config from dotnet/android for repo builds
2. Cake Tasks for downloading and installing Nugets needed for builds

[moljac]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Copilot]

Pull Request Overview

This PR updates the NuGet configuration and build scripts to support new pipeline requirements by manually downloading NuGet packages and cleaning them up post-build. Key changes include:

• Introduces a new Cake task in nuget-install.cake for downloading necessary NuGet packages.
• Adds cleanup steps in build-and-package.cake to remove NuGet packages from the output directory.
• Updates utilities.cake and binderate.cake with new package versions and directory setup.

Reviewed Changes

Copilot reviewed 5 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
build/cake/nuget-install.cake	New Cake task for manually downloading NuGet packages.
build/cake/build-and-package.cake	Adds deletion of downloaded packages to prevent CI validation issues.
build.cake	Loads and runs the new nuget-install task.
utilities.cake	Upgrades package versions and includes commented-out alternative addin declarations.
build/cake/binderate.cake	Ensures the output directory exists for binderate use.

Files not reviewed (2)

• NuGet.config: Language not supported
• tests/common/NuGet.config: Language not supported

[jpobst]

This breaks all of our unit tests, example:

NU1100: Unable to resolve 'Microsoft.NET.ILLink.Tasks (>= 8.0.14)' for 'net8.0-android34.0'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\ToolCache\dotnet\library-packs, darc-pub-dotnet-android-82d8938, darc-pub-dotnet-android-e7876a4, darc-pub-dotnet-emsdk-91b783e, darc-pub-dotnet-runtime-ef07c4f, dotnet-eng, dotnet-public, dotnet-tools, dotnet10, dotnet10-transport, dotnet9, dotnet9-transport, Local Output, xamarin.android util.
NU1100: Unable to resolve 'Microsoft.NET.ILLink.Tasks (>= 8.0.14)' for 'net8.0-android34.0/android-arm'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\ToolCache\dotnet\library-packs, darc-pub-dotnet-android-82d8938, darc-pub-dotnet-android-e7876a4, darc-pub-dotnet-emsdk-91b783e, darc-pub-dotnet-runtime-ef07c4f, dotnet-eng, dotnet-public, dotnet-tools, dotnet10, dotnet10-transport, dotnet9, dotnet9-transport, Local Output, xamarin.android util.
NU1100: Unable to resolve 'Microsoft.NET.ILLink.Tasks (>= 8.0.14)' for 'net8.0-android34.0/android-arm64'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\ToolCache\dotnet\library-packs, darc-pub-dotnet-android-82d8938, darc-pub-dotnet-android-e7876a4, darc-pub-dotnet-emsdk-91b783e, darc-pub-dotnet-runtime-ef07c4f, dotnet-eng, dotnet-public, dotnet-tools, dotnet10, dotnet10-transport, dotnet9, dotnet9-transport, Local Output, xamarin.android util.
NU1100: Unable to resolve 'Microsoft.NET.ILLink.Tasks (>= 8.0.14)' for 'net8.0-android34.0/android-x64'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\ToolCache\dotnet\library-packs, darc-pub-dotnet-android-82d8938, darc-pub-dotnet-android-e7876a4, darc-pub-dotnet-emsdk-91b783e, darc-pub-dotnet-runtime-ef07c4f, dotnet-eng, dotnet-public, dotnet-tools, dotnet10, dotnet10-transport, dotnet9, dotnet9-transport, Local Output, xamarin.android util.
NU1100: Unable to resolve 'Microsoft.NET.ILLink.Tasks (>= 8.0.14)' for 'net8.0-android34.0/android-x86'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\ToolCache\dotnet\library-packs, darc-pub-dotnet-android-82d8938, darc-pub-dotnet-android-e7876a4, darc-pub-dotnet-emsdk-91b783e, darc-pub-dotnet-runtime-ef07c4f, dotnet-eng, dotnet-public, dotnet-tools, dotnet10, dotnet10-transport, dotnet9, dotnet9-transport, Local Output, xamarin.android util.

I'm also not sure we actually want to make this change. If we stick to publicly released preview versions of .NET 10, do we need to do all of this extra work?

cc: @jonathanpeppers ^^

[jonathanpeppers]

What is this PR trying to do? I'm not following why you need a copy of dotnet/android's NuGet.config?

From the discussion here, I thought you should use NuGet.org and that's it?

• https://discord.com/channels/732297728826277939/732297837953679412/1356630082461896877

[moljac]

What is this PR trying to do?

Workaround for security policies.

This is new pipeline and has some restrictions we didn't have before.

Few minutes ago I added back:

    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />    

and as result I get:

Secure Supply Chain Analysis (auto-injected by policy)

[View raw log](https://devdiv.visualstudio.com/0bdbc590-a062-4c3f-b0f6-9383f67865ee/_apis/build/builds/11326615/logs/43)

Starting: Secure Supply Chain Analysis (auto-injected by policy)
==============================================================================
Task         : Secure Supply Chain Analysis
Description  : A task to scan for vulnerabilities in your software supply chain. Formerly "NuGet Security Analysis".
Version      : 0.2.209
Author       : Microsoft Corporation
Help         : See https://aka.ms/sscatask for more information.
==============================================================================
Telemetry ID: 27ece493-7a70-4bdb-97d1-fd0373d8839a
For more information please visit: https://aka.ms/sscatask
> Starting Multifeed Nuget Security Analysis:
##[warning]NuGet.config - Multiple feeds declared. (https://aka.ms/cfs/nuget)
##[warning]tests/common/NuGet.config - Multiple internal feeds declared, but none with upstreams.
> Starting Multifeed Corext Analysis:
> Starting Multifeed Python Security Analysis:
> Starting CFS NuGet Analysis:
##[warning]NuGet.config - CFS0013: Package source has value that is not an Azure Artifacts feed. (https://aka.ms/cfs/nuget)
> Starting CFS NPM Analysis:
> Starting CFS Maven Analysis:
> Starting CFS Cargo Analysis:
> Starting CFS CoreXT Analysis:
> Starting CFS CDPx Analysis:
> Starting DockerFile Analysis:
> Starting Kubernetes Deployment File Analysis:
> Starting Helm Charts Analysis:
> Starting Pipeline Configuration Security Analysis:
Azure Artifacts Configuration Analysis found 1 package configuration file in the repository which do not comply with Microsoft package feed security policies. The specific problems and links to their mitigations are listed above. If you need further assistance, please visit https://aka.ms/cfs/detectors .
##[error]NuGet Security Analysis found 1 NuGet package configuration file in the repository which do not comply with Microsoft package feed security policies. The specific problems are listed above. Please visit https://aka.ms/cfs/nuget for more details.
Finishing: Secure Supply Chain Analysis (auto-injected by policy)

Link:

https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=11326615&view=logs&j=784e4eae-0a8d-50ee-7be1-df4337debdeb&t=87ccd997-13b2-5249-5901-18acd7ad4971&l=28

That is the reason I used dotnet/android's nuget.config.

I'm not following why you need a copy of dotnet/android's NuGet.config?

There is no nuget.org in dotnet/android's NuGet.config and I simply copied it. BTW when I copied it - net10 preview was still not released, so I was building dotnet/android 1st.

From the discussion here, I thought you should use NuGet.org and that's it?

• https://discord.com/channels/732297728826277939/732297837953679412/1356630082461896877

For dotnet workload install it worked.

There is much much more

Our tests fail locally with:

========================================
all-packages-tests
========================================
  Determining projects to restore...
  Restored /Users/Shared/Projects/d/dotnet/android-libraries/dev--moljac--mu-20241209-net10-removal-of-generator-workarounds/tests/allpackages/AllPackagesTests.csproj (in 276 ms).
  AllPackagesTests -> /Users/Shared/Projects/d/dotnet/android-libraries/dev--moljac--mu-20241209-net10-removal-of-generator-workarounds/tests/allpackages/bin/Debug/net8.0/AllPackagesTests.dll
Test run for /Users/Shared/Projects/d/dotnet/android-libraries/dev--moljac--mu-20241209-net10-removal-of-generator-workarounds/tests/allpackages/bin/Debug/net8.0/AllPackagesTests.dll (.NETCoreApp,Version=v8.0)
VSTest version 17.11.1 (arm64)

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
  Failed TestAndroidDotNetAllGPSPackages [1 s]
  Error Message:
   Command 'new android' failed with exit code 103.
Output:


Error:
No templates or subcommands found matching: 'android'.

To list installed templates similar to 'android', run:
   dotnet new list android
To search for the templates on NuGet.org, run:
   dotnet new search android


For details on the exit code, refer to https://aka.ms/templating-exit-codes#103

It is because test use CliWrap to create app dotnet new android for wrokload restore.

If I run dotnet new android I get:

No templates or subcommands found matching: 'android'.

To list installed templates similar to 'android', run:
   dotnet new list android
To search for the templates on NuGet.org, run:
   dotnet new search android


For details on the exit code, refer to https://aka.ms/templating-exit-codes#103

There is also cake script that downloads few nugets I was not able to migrate to internal feeds. Namely some packages are whitelisted and some are not.

I hope this shines some light.

[moljac]

I'm also not sure we actually want to make this change. If we stick to publicly released preview versions of .NET 10, do we need to do all of this extra work?

This has nothing to do with .NET 10. Simply security policies for new pipelines.