stdenv: pURL implementation

[h0nIg]

purl implementation based on drv.src, which will get referenced by the main derivation (if not specified otherwise)

Backport reasons for 25.05

In my opinion, this change can get backported to 25.05 as well (without the release note). Otherwise we meet again in 5 + x months (guessing 5+6 = 11, 1 year ahead), after maintainers start adopting their derivations and later start thinking about further enhancements.

My company (using nixpkgs 25.05) is actively using pURL and can give the learnings back to the community already today (once merged to 25.05).

In addition CVE like this jq CVE were not detected properly, therefore an additional reason for backporting / benefiting today:

The security tracker https://discourse.nixos.org/t/nixpkgs-supply-chain-security-project/34345/30 can benefit as well, because once a regular bump of software on master is done (without keeping a CVE in mind and determining "stable needs a fix as well"), nixpkgs stable fix is not triggered automatically.
Example: #409300 libarchive was bumped without CVE reference (edit to the PR description was adjusted after additional issue was created ONLY), just by coincidence it was backported as well.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• Nixpkgs 25.11 Release Notes (or backporting 24.11 and 25.05 Nixpkgs Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
• NixOS 25.11 Release Notes (or backporting 24.11 and 25.05 NixOS Release notes)
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other contributing documentation in corresponding paths.

---

Add a 👍 reaction to pull requests you find important.

[drupol]

Ping @pombredanne ^^

[SuperSandro2000]

Lets go 🎉

Can we merge this or did I miss something in the long thread?

[h0nIg]

Lets go 🎉

Can we merge this or did I miss something in the long thread?

@SuperSandro2000 yes please go ahead and merge, @emilazy asked @YorikSar to take another look and he did that yesterday evening. thank you!

[nixpkgs-ci]

Backport failed for release-25.05, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin release-25.05
git worktree add -d .worktree/backport-421125-to-release-25.05 origin/release-25.05
cd .worktree/backport-421125-to-release-25.05
git switch --create backport-421125-to-release-25.05
git cherry-pick -x 4e2614fc0709ff77e40a8f39e2744239ee371826 0a69474ed34ef6a4e82804b4b2d844deb126a1ab 2e46d00d76d3c9690e9713a9c2686c328e3779da c78e6a235962eb272981ea6b16939034c0fde575 64a6ca1114355caca991817cba83c4beb18136e2 22dbee80107516b858abd3d7a45c149a316a78d8 1f173d017207dc039a1c2494fd88c20d757d864c cadcde9f7f04c239c0e187903d524ae57afce569 25f90d7d20c46acd8eca5a8bf1b7f558e0efda02 83b6d2e657e2bbc19d55c48b0a888988014ac805 87977474f1802bb0a5dbc1e5ad60ce7f04624cc7 81dc446ee36274f737a05755af92b74e70e0c07d 3ddee85a175472d063063a3423524f668ed31b86 f7cbf2374b500cc2b87dbba11baa9b4ea03d6086 bacccc39a9cfd80b62940002f0c656add2aa3619 028af7c17dacf56953cafd8a19aaecd12edf7921 0ef545933fb1a707b70cb94b475a07343aa9ae7e

[arianvp]

I might have merged this too eagerly. I think we should've probably squashed the commits into one commit prior to merging to ease backporting. I'm sorry 😔

[YorikSar]

Now that we’re already in 25.11 release cycle I don’t think backport makes much sense.

[emilazy]

@h0nIg Please squash commits that address review feedback in future for the sake of bisectability and clean history, per the commit conventions linked in the PR checklist.

[dramforever]

This broke nixpkgs-review on aarch64-linux with a thunderbird-bin error, because on aarch64-linux there's no thunderbird-bin, so thunderbird-bin.src errors. I think nixpkgs-review tries to grab the full meta which then errors because meta now depends on src.

Yes this really is the only package that has this problem. To reproduce: nix eval --json -f /path/to/nixpkgs --argstr system aarch64-linux thunderbird-bin.meta

How should we fix this?

cc @lovesegfault

[h0nIg]

This broke nixpkgs-review on aarch64-linux with a thunderbird-bin error, because on aarch64-linux there's no thunderbird-bin, so thunderbird-bin.src errors. I think nixpkgs-review tries to grab the full meta which then errors because meta now depends on src.

Yes this really is the only package that has this problem. To reproduce: nix eval --json -f /path/to/nixpkgs --argstr system aarch64-linux thunderbird-bin.meta

How should we fix this?

cc @lovesegfault

thunderbird-bin is not supported on aarch64-linux, is this really a problem? Will check anyhow

nix-repl> legacyPackages.aarch64-linux.thunderbird-bin.meta.identifiers
error:
       … while evaluating the attribute 'aarch64-linux.thunderbird-bin.meta.identifiers'
         at /nix/store/3k2h4x5hgfgia2fmc6lky178jghrqq06-source/pkgs/stdenv/generic/check-meta.nix:675:7:
          674|
          675|       identifiers =
             |       ^
          676|         let

       … in the left operand of the update (//) operator
         at /nix/store/3k2h4x5hgfgia2fmc6lky178jghrqq06-source/pkgs/stdenv/generic/check-meta.nix:761:9:
          760|         v1
          761|         // {
             |         ^
          762|           inherit v1 purlParts;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: attribute 'aarch64-linux' missing
       at /nix/store/3k2h4x5hgfgia2fmc6lky178jghrqq06-source/pkgs/applications/networking/mailreaders/thunderbird-bin/default.nix:40:10:
           39|
           40|   arch = mozillaPlatforms.${stdenv.hostPlatform.system};
             |          ^
           41|

nixpkgs/pkgs/applications/networking/mailreaders/thunderbird-bin/default.nix

Line 76 in d81b27c

platforms = builtins.attrNames mozillaPlatforms;

nixpkgs/pkgs/applications/networking/mailreaders/thunderbird-bin/default.nix

Lines 40 to 46 in d81b27c

	mozillaPlatforms = {
	i686-linux = "linux-i686";
	x86_64-linux = "linux-x86_64";
	# bundles are universal and can be re-used for both darwin architectures
	aarch64-darwin = "mac";
	x86_64-darwin = "mac";
	};

[RossComputerGuy]

Imo, that should be throwing an unsupported platform error rather than a missing attribute error.

[wolfgangwalther]

(copied from Matrix):

I'm skeptical that this only happens for thunderbird-bin. I think there are plenty of packages where a src is not available on every platform. I assume that one can't read meta for these packages on other platforms now anymore. That seems quite bad.

(I did not really look into this PR, but reading meta must be possible, even on unsupported platforms!)

[h0nIg]

(copied from Matrix):

I'm skeptical that this only happens for thunderbird-bin. I think there are plenty of packages where a src is not available on every platform. I assume that one can't read meta for these packages on other platforms now anymore. That seems quite bad.

(I did not really look into this PR, but reading meta must be possible, even on unsupported platforms!)

you can read thunderbird-bin.meta, but you can not read thunderbird-bin.meta.identifiers

[wolfgangwalther]

you can read thunderbird-bin.meta, but you can not read thunderbird-bin.meta.identifiers

Since meta.identifiers is part of meta, I must be able to read it, too.

[dramforever]

In practice this really breaks nixpkgs-review, so something must be fixed

[dramforever]

So the actual thing this breaks is config.checkMeta = true which is set by nixpkgs-review and causes thunderbird-bin.meta.available to error out on aarch64-linux.

AFAICT the thunderbird-bin variants are the only packages affected, but note that that doesn't mean it's the right solution to fix thunderbird-bin. It might mean we shouldn't have made meta depend on src to begin with, or maybe we should have been checking this in CI, or...

[wolfgangwalther]

For visibility: This was reverted in #453322, because of the issues mentioned above and in #453291.

[h0nIg]

For visibility: This was reverted in #453322, because of the issues mentioned above and in #453291.

I would like to summarize the 2 cases we hit:

• tools like nixpkgs-review, which ignore certain meta informations and which are willingly accessing meta even if it might be broken. They need to enable checkMeta=true as well to run into issues, which is turned off by default. We hit one package "thunderbird-bin", which did not evaluate properly and was fixed in the meantime: thunderbird-bin: throw on unsupported system #453333
• tools which just want to get listed on search.nixos.org, like cplex, which do not provide a src and which are not buildable. They throw an exception, since the src is not available (needs override).: stdenv: pURL implementation - fix checkMeta #453291 (comment)

We should thank @wolfgangwalther taking care about CI, he outlined conditions how this change should get included for the brought audience next time: #453322 (comment)

I would like to hear @YorikSar, which gave his feedback and which encouraged me to move the logic into mkderivation and which offers the feature for everyone / languages, which was a bit too ambitious.

options which i see, to get this included into 25.11.:

option 1: go back to previous implementation 8797747 -1 commit, where we explicitly configure languages which are known to play by the rules. We lose general support with this - but we prevent issues which caused the need of the revert
option 2: feature-toggle the inheritance from drv.src.meta to drv.meta. People using SBOM tools can enable this on purpose. People can still benefit once they find drv.src with their SBOM analysis tool (doing recursive drv analysis from outside of nixpkgs - since they no longer need to guess)

everyone: WDYT?

[YorikSar]

I would like to hear @YorikSar, which gave his feedback and which encouraged me to move the logic into mkderivation and which offers the feature for everyone / languages, which was a bit too ambitious.

Even with the previous approach, we're introducing this dependency between meta and src, except before the change it was spread across different language ecosystems. I'm sure we can find or will run into similar issues with src in some Go/Python/Ruby packages as well.

I guess, we could start by just adding PURLs to fetch* functions so that as many src attributes contain them as possible and let tools decide what to do with that. Note that tools will have to recursively check if src has its own src with PURL (I counted over 300 such packages).
Maybe we could add a function that copies PURL from src to let maintainers write something like meta.identifiers.purl = lib.purlFromSrc src? It would be quite annoying to have to put this in each package though.

[h0nIg]

lets try another round, this time I would like to prevent fallout through a feature flag and still enable people to gather experience & maintain their non default data (e.g. jq example, where fetchurl is used instead of fetchFromGithub) - as outlined by @YorikSar above regarding the fetchers aspect only

#454333