Skip to content

Supplement Hydra data with meta.mainProgram - #318

Open
samestep wants to merge 2 commits into
nix-community:masterfrom
samestep:nix-env-meta-main-program
Open

Supplement Hydra data with meta.mainProgram#318
samestep wants to merge 2 commits into
nix-community:masterfrom
samestep:nix-env-meta-main-program

Conversation

@samestep

@samestep samestep commented Jun 5, 2026

Copy link
Copy Markdown

This PR resolves #317, enabling nix-locate and downstream tools like comma to see packages that Hydra doesn't build:

$ nix-locate --at-root --whole-name /bin/nh
nh-unwrapped.out                             14,301,472 x /nix/store/j5lpv7cdjxs35if2b4snqkgxc4q0ya6a-nh-unwrapped-4.3.2/bin/nh
nh.out                                                0 x /nix/store/k9c04vx63x91fa3k147g5hi7k0ppns80-nh-4.3.2/bin/nh

I would strongly recommend reviewing the two commits of this PR separately.

The first commit rewrites the parser for nix-env --query --xml. I did this because I found the existing parser extremely unwieldy and difficult to modify. The rewrite uses Serde via quick-xml, is significantly less code, and is roughly 4x faster than before. While the new double usage of .collect in listings::fetch may look a bit suspicious, the memory usage is not noticeably different from before.

The second commit implements the actual feature: passing the --meta flag to that nix-env --query command so that we can extract meta.mainProgram when it's available. That is then threaded through along with the store path, and in cases where Hydra doesn't have it, a synthesized /bin/$mainProgram file listing is created. This whole thing can be turned off via nix-index --no-main-program, which simply doesn't add the --meta flag and so no meta.mainProgram entries are created in the first place.

A few notes/questions I have for the reviewer(s):

  1. Is it alright for these two commits to be put together into this one PR, or would the first commit need to be merged as a separate PR first? Or is the parser rewrite simply a no-go and I should change this PR to fit into the existing parser structure?

  2. Should this PR modify CHANGELOG.md?

  3. Is it fine that we say the size of the executable file is zero bytes, and give an empty string for the nar? It seems like the latter is only used for debugging (and thus would never be used in this case anyway), but I wanted to double-check. I believe the former is only displayed by nix-locate (see the example output above); it does look slightly weird, so I wonder if it'd be worth it to change FileNode::Regular's size field from u64 to Option<u64>.

  4. This PR doesn't use quick-xml in such a way that the error messages include line or column numbers. Is that fine? If not, I could make the code slightly uglier and use Reader::error_position to report the byte offset of the deserialization error. Or if we really need line and column specifically, I suppose we could keep track of where all the newlines are when we stream the data, and then convert the byte offset using those. I wasn't sure how much this matters, though, so I kept the code simple initially.

  5. While the rewritten parser from the first commit is about 4x faster than the previous parser, adding the --meta flag causes the XML output to be 10x bigger, and in the end, when we feed that bigger XML into the new parser, it's about 3x slower than feeding the smaller XML into the original parser. I'm guessing this is fine since parsing this XML isn't really a bottleneck?

  6. I don't love that now nixpkgs::query_packages and listings::fetch each have two boolean flags passed positionally. But I'm also not sure it's worth it to create a struct in this case.

@nixos-discourse

Copy link
Copy Markdown

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-index-cant-see-wrapper-packages/78059/7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

nix-index can't see wrapper packages

2 participants