Skip to content

Implement dynamic dependencies and remove extra inputs hack#38

Merged
elpdt852 merged 1 commit intomainfrom
feature/dynamic-dep
Sep 6, 2025
Merged

Implement dynamic dependencies and remove extra inputs hack#38
elpdt852 merged 1 commit intomainfrom
feature/dynamic-dep

Conversation

@elpdt852
Copy link
Contributor

@elpdt852 elpdt852 commented Sep 1, 2025

Fix #19 🎉

Allows of arbitrarily deep dynamic dependencies, leveraging all the features of dynamic derivations like dynamicOutputs and nix-computed-output placeholders. This lets us remove the "extra-inputs" hack for compiling CppNix where they had bison parser.y generate .cc source files that also need header dependency discovery.

We introduce a "dynamic-task" derivation that detects that when (1) it has "deps = gcc", i.e. it has header dependency discovery, and (2) it has an input that is built rather than opaque, i.e. it is an output from another derivation, then we must have an intermediary derivation that depends on these built inputs and does header discovery and modifies the task derivation to include these discovered inputs.

There's still some work to do, for example we should do discovery in a task per built input rather than together so we can cache discovery more granularly. CppNix has generated inputs that are often shared across build targets, so there is a bit of wasted discovery atm.

With these changes, it makes it possible to build CppNix end to end without any hacks.

ninja-build-src-nix-nix.p-store-info.cc.o> nix-ninja-task: Finished! Copying 1 build outputs to derivation output paths
ninja-build-src-nix-nix.p-store.cc.o> nix-ninja-task: Finished! Copying 1 build outputs to derivation output paths
ninja-build-src-nix-nix.p-why-depends.cc.o> nix-ninja-task: Finished! Copying 1 build outputs to derivation output paths
ninja-build-src-nix-nix.p-upgrade-nix.cc.o> nix-ninja-task: Finished! Copying 1 build outputs to derivation output paths
ninja-build-src-nix-nix.p-verify.cc.o> nix-ninja-task: Finished! Copying 1 build outputs to derivation output paths
ninja-build-src-nix-nix.p-unix_daemon.cc.o> nix-ninja-task: Finished! Copying 1 build outputs to derivation output paths
example-nix> nix-ninja-task: Setup source directory in /build/source/build
example-nix> nix-ninja-task: Linking target src/nix/nix
example-nix> nix-ninja-task: Running: /bin/sh -c "g++  -o src/nix/nix src/nix/nix.p/add-to-store.cc.o src/nix/nix.p/app.cc.o src/nix/nix.p/self-exe.cc.o src/nix/nix.p/build.cc.o src/nix/nix.p/bundle.cc.o src/nix/nix.p/cat.cc.o src/nix/nix.p/config-check.cc.o src/nix/nix.p/config.cc.o src/nix/nix.p/copy.cc.o src/nix/nix.p/crash-handler.cc.o src/nix/nix.p/derivation-add.cc.o src/nix/nix.p/derivation-show.cc.o src/nix/nix.p/derivation.cc.o src/nix/nix.p/develop.cc.o src/nix/nix.p/diff-closures.cc.o src/nix/nix.p/dump-path.cc.o src/nix/nix.p/edit.cc.o src/nix/nix.p/env.cc.o src/nix/nix.p/eval.cc.o src/nix/nix.p/flake.cc.o src/nix/nix.p/fmt.cc.o src/nix/nix.p/hash.cc.o src/nix/nix.p/log.cc.o src/nix/nix.p/ls.cc.o src/nix/nix.p/main.cc.o src/nix/nix.p/make-content-addressed.cc.o src/nix/nix.p/man-pages.cc.o src/nix/nix.p/nar.cc.o src/nix/nix.p/optimise-store.cc.o src/nix/nix.p/path-from-hash-part.cc.o src/nix/nix.p/path-info.cc.o src/nix/nix.p/prefetch.cc.o src/nix/nix.p/profile.cc.o src/nix/nix.p/realisation.cc.o src/nix/nix.p/registry.cc.o src/nix/nix.p/repl.cc.o src/nix/nix.p/run.cc.o src/nix/nix.p/search.cc.o src/nix/nix.p/sigs.cc.o src/nix/nix.p/store-copy-log.cc.o src/nix/nix.p/store-delete.cc.o src/nix/nix.p/store-gc.cc.o src/nix/nix.p/store-info.cc.o src/nix/nix.p/store-repair.cc.o src/nix/nix.p/store.cc.o src/nix/nix.p/upgrade-nix.cc.o src/nix/nix.p/verify.cc.o src/nix/nix.p/why-depends.cc.o src/nix/nix.p/unix_daemon.cc.o src/nix/nix.p/nix-build_nix-build.cc.o src/nix/nix.p/nix-collect-garbage_nix-collect-garbage.cc.o src/nix/nix.p/nix-copy-closure_nix-copy-closure.cc.o src/nix/nix.p/nix-env_nix-env.cc.o src/nix/nix.p/nix-env_user-env.cc.o src/nix/nix.p/nix-instantiate_nix-instantiate.cc.o src/nix/nix.p/nix-store_dotgraph.cc.o src/nix/nix.p/nix-store_graphml.cc.o src/nix/nix.p/nix-store_nix-store.cc.o src/nix/nix.p/build-remote_build-remote.cc.o src/nix/nix.p/nix-channel_nix-channel.cc.o -Wl,--as-needed -Wl,--no-undefined '-Wl,-rpath,$ORIGIN/../libutil:$ORIGIN/../libstore:$ORIGIN/../libexpr:$ORIGIN/../libfetchers:$ORIGIN/../libflake:$ORIGIN/../libmain:$ORIGIN/../libcmd' -Wl,--start-group src/libutil/libnixutil.so src/libstore/libnixstore.so src/libexpr/libnixexpr.so src/libfetchers/libnixfetchers.so src/libflake/libnixflake.so src/libmain/libnixmain.so src/libcmd/libnixcmd.so /nix/store/9fz37cc2hai6n2d17fdr3gcdr67x3853-libarchive-3.8.1-lib/lib/libarchive.so /nix/store/3dzksnrd6a17b7qlrhrvxmbs4mlc2n9c-boehm-gc-8.2.8/lib/libgc.so -lpthread -ldl -Wl,--end-group -pthread"
example-nix> nix-ninja-task: Fixed RPATH for src/nix/nix
example-nix> nix-ninja-task: Finished! Copying 1 build outputs to derivation output paths

$ ./result-src-nix-nix/src/nix/nix --version
nix (Nix) 2.27.1

@hinshun hinshun force-pushed the feature/dynamic-dep branch 3 times, most recently from d6bffa1 to c682937 Compare September 6, 2025 07:43
@hinshun hinshun force-pushed the feature/dynamic-dep branch from c682937 to 66f4393 Compare September 6, 2025 08:04
@elpdt852 elpdt852 marked this pull request as ready for review September 6, 2025 08:15
@elpdt852 elpdt852 merged commit 8da02bd into main Sep 6, 2025
7 checks passed
@elpdt852 elpdt852 deleted the feature/dynamic-dep branch September 6, 2025 08:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dynamic derivations for dependency inference

2 participants