feat(oxcaml): instantiate parameterized libraries #12561
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Follow up on #12392 and #12336: This PR adds the ability to instantiate parameterized libraries in library/executable dependencies. This feature requires compiler support which is only available in OxCaml atm.
In stanza dependencies, the new form
(my_lib arg1 arg2 ... :as new_name)
specifies that the parameterized librarymy_lib
should be instantiated with a list of arguments (which must implement the parameters requested by the lib). The arguments are identified by the parameter interface they implement, such that the order in which they are provided doesn't matter. (It's nominal not positional. We sort them internally in dune, but only to ensure we don't duplicate efforts to instantiate both(f x y)
and(f y x)
, since they are the same.)The optional
:as new_name
allow renaming the instantiation in user code, which is useful when the same library is instantiated multiple times (otherwise if missing, by defaultMy_lib
is used as the name of the instantiation). In the alias file, each instantiation is translated to amodule New_name = My_lib(Param1)(Arg1)(Param2)(Arg2)(* ... *) [@jane.non_erasable.instances]
. (Note that eachArgN
implementsParamN
, i.e.(Arg1 : Param1)
, so this is not a real functor application, hence the OxCaml attribute.)While libraries can do a partial application of their parameterized dependencies (as long as their list of parameters is a superset of missing dependency parameters), executables must fully instantiate their dependencies. Computing the transitively instantiated libraries requires passing the applied arguments to each of parameterized dependencies that requires this parameter. This impacts the existing transitive closures performed by dune, since now the same library may be required multiple times but with different arguments.
To link the executables, all the (transitively) instantiated libraries must also be built by the compiler with the new OxCaml command
ocamlopt -instantiate mylib.cmx arg1.cmx arg2.cmx ... -o mylib-Arg1-Arg2...cmx
. Here a partial instantiation is not allowed (all arguments must be specified). Furthermore, this should be done for all modules ofmylib
, in their topological order. We create an archive at the end to package everything.Instantiated modules use dashes in their filenames to "parenthesize" their arguments, which can themselves contain dashes if they are the result of another instantiation. The number of dashes indicates the depth of the application. For example, a filename
f-g--h---x-y
corresponds to the applicationF(G(H(X)))(Y)
. We reuse this idea for the name of the folders where we instantiate each module of a fully instantiated parameterized dependency. A dependency(mylib arg1 arg2)
(fully applied) is instantiated in the _build subfolder.parameterized/mylib!arg1!arg2
. We can't use dash separators here because it's already allowed in library names and would be ambiguous, so instead we arbitrarily picked the exclamation mark instead... but let us know if there's a better choice!Finally for opam installations, we export all the dependencies arguments (but not their concrete instantiation, since this will done by user executables when needed).
While I still have a couple of TODOs to address minor issues (or so I hope), this PR seems to be working... so I would appreciate an early review to check I haven't done anything stupid :)
Fix #12088