Incompatible with nixos-unstable

[johannesloetzsch]

At the moment ǹix-servedoesn't work with ǹixos-unstable any longer.

When accessing a narinfo-file while signing is enabled, nix-serve fails with:

Died at /nix/store/ix4wzwra6yacms4ykyixbpfzj329ippv-perl-5.34.0-env/lib/perl5/site_perl/5.34.0/x86_64-linux-thread-multi/Nix/Manifest.pm line 236

Also curl localhost:5000/nix-cache-info shows, that StoreDir is not correctly evaluated.

The easiest way to reproduce this bug, is setting inputs.nixpkgs.url = "nixpkgs/nixos-unstable"; in flake.nix and running

nix run .#nix-serve

If further information is required to fix this, please let me know.
Thanks for all your wonderful work at the nix ecosystem :)

[t184256]

nixpkgs.overlays = [ (_: super: { nix-serve = super.nix-serve.override { nix = super.nix_2_3; }; } ) ]; fixes that, so I presume it's Nix 2.4.

[joncol]

@t184256 : Trying the above gives me the correct StoreDir: /nix/store in /nix-cache-info, but I still get internal server error when trying to GET some .narinfo URL. Do you mean the above fix should solve this?

[t184256]

Yes, that's what I meant, I'm able to use my cache now without internal server errors. Are there more bugs at play here in your case?

[joncol]

Maybe I did some other mistake? My setup is here: https://gitlab.com/joncol/nixos-flake/-/blob/main/flake.nix#L37-40.

If you spot some obvious mistake, I'd be very happy to hear it...

[joncol]

@t184256: Are you using services.nix-serve { enable: true; } to run nix-serve, or some other method?

[joncol]

@t184256: OK, I found your config, and putting the overlay just above the services.nix-serve section seems to work:

  nixpkgs.overlays = [
    (_: super: {
      nix-serve = super.nix-serve.override { nix = super.nix_2_3; };
    })
  ];

  services.nix-serve = {
    enable = true;
    port = 5000;
    secretKeyFile = "/var/cache-priv-key.pem";
  };

I don't really understand how overrides and overlays work yet, so I'm mostly cargo-culting.

[joncol]

@t184256: Now I can actually access the .narinfo endpoints. Current problem is that when trying to use the cached builds (from another machine, B), I get messages like:

warning: substituter 'http://my-nixos-host:5000' does not have a valid signature for path '/nix/store/ybd313lj4ava4l8zrk1f22c177bmzpfd-emacs-gcc-20211122.0'

(Trying to access http://my-nixos-host:5000/ybd313lj4ava4l8zrk1f22c177bmzpfd.narinfo from the same machine B works...)

[joncol]

Sorry for the noise. This last error was just me not using the right syntax for the public key on machine B. Now it works! Thanks for the "fix"!

[px-ben]

I encountered the same issue after moving to Nix 2.4. I raised the following PR with a fix: NixOS/nix#5634

[johannesloetzsch]

Thanks @t184256 and @px-ben :)

[knedlsepp]

For some reason the UID of nix-serve changed and I had to chown nix-serve /var/cache-priv-key.pem on the latest NixOS 21.11 upgrade. Will these UIDs change again on reboot?

[johannesloetzsch]

For some reason the UID of nix-serve changed and I had to chown nix-serve /var/cache-priv-key.pem on the latest NixOS 21.11 upgrade. Will these UIDs change again on reboot?

Hello @knedlsepp,
that should be a problem independent from this Issue.
I encountered it as well, when I followed the documentation at https://nixos.wiki/wiki/Binary_Cache.

It seems like a systemd.tmpfiles.rules is required at configuration.nix. You should be able to use this, when you adjust it to your paths:

systemd.tmpfiles.rules = [ "C /run/cache-priv-key.pem 400 nix-serve root - /var/cache-priv-key.pem" ];

If I'm right, we may want put a solution into the module or at least fix the documentation.