-
Notifications
You must be signed in to change notification settings - Fork 30
docs: Document Guppy libraries #1552
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: mr/feat/compile-library
Are you sure you want to change the base?
Changes from 13 commits
af7a656
319877c
6e4f9c3
18f39f9
1ae986f
ebcb017
bd70682
17bda6b
5303842
1dd6653
42f709a
f3a54d5
ee67cc0
9b504cb
af04b70
7b79531
31c6d35
43d1c20
94e24ee
be02ee0
288e410
8439503
ff7df03
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| build |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| # Configuration file for the Sphinx documentation builder. | ||
| # See https://www.sphinx-doc.org/en/master/usage/configuration.html | ||
|
|
||
| project = "Guppy Libraries" | ||
| copyright = "2026, Quantinuum" | ||
| author = "Quantinuum" | ||
|
|
||
| extensions = [ | ||
| "sphinx.ext.napoleon", | ||
| "sphinx.ext.autodoc", | ||
| "sphinx.ext.coverage", | ||
| "sphinx.ext.viewcode", | ||
| "sphinx.ext.intersphinx", | ||
| "sphinx_copybutton", | ||
| "myst_parser", | ||
| ] | ||
|
|
||
| html_theme = "furo" | ||
|
|
||
| html_title = "Guppy library docs" | ||
|
|
||
| html_theme_options = {} | ||
|
|
||
| html_static_path = [] | ||
|
|
||
| exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"] | ||
|
|
||
| intersphinx_mapping = { | ||
| "python": ("https://docs.python.org/3/", None), | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # Guppy Libraries Documentation | ||
|
|
||
| An introductory tale into how Guppy libraries (are supposed to) work, how to write them, and how to use them. | ||
|
|
||
| ```{note} | ||
| This docs collection concerns Guppy libraries usage for independently compiling a set of functions to a Hugr package, | ||
| which can be independently distributed and e.g. cached for optimization purposes. If you are just looking to open | ||
| source your Guppy code, the standard packaging mechanisms of Python suffice, and you can ignore this page. | ||
| ``` | ||
|
|
||
| - Foreword | ||
|
|
||
| ```{toctree} | ||
| :maxdepth: 2 | ||
|
|
||
| overview | ||
| proposals | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,160 @@ | ||
| # Overview | ||
|
|
||
| ## Defining a library | ||
|
|
||
| A Guppy library is, at its simplest, a collection of functions that are compiled together into a Hugr package. | ||
| In contrast to the usual executable Hugr packages (like those produced from compiling single entrypoint functions), such | ||
| library Hugr packages feature (usually multiple) *public* functions, possibly with arguments and non-``None`` return | ||
| types. | ||
|
|
||
| A developer may define a bunch of functions beyond this collection, for example for internally sharing functionality | ||
| between the public functions. Furthermore, developers may want to create different libraries / Hugr packages from a | ||
| single codebase, requiring to export different public functions in each such library. | ||
| In Guppy, a library is defined by calling the ``guppy.library(...)`` function, passing the functions to be exported as | ||
| public. | ||
| Any functions reachable via those functions are still included in the Hugr package, but with private visibility. | ||
|
|
||
| For example, a library may be defined as follows: | ||
|
|
||
| ```python | ||
| from guppylang import guppy | ||
|
|
||
|
|
||
| @guppy | ||
| def my_func() -> None: | ||
| pass | ||
|
|
||
|
|
||
| @guppy | ||
| def another_func(x: int) -> None: | ||
| pass | ||
|
|
||
|
|
||
| @guppy | ||
| def a_third_func(x: int) -> int: | ||
| return x + 1 | ||
|
|
||
|
|
||
| # Creates the library object, but does not compile it | ||
| library = guppy.library( | ||
| my_func, | ||
| another_func, | ||
| a_third_func, | ||
| ) | ||
| ``` | ||
|
|
||
| For private functions, the name their Hugr nodes receive is not important. | ||
| However, public functions need to have a stable (and unique) name for the Hugr to be valid and useful. | ||
| Guppy includes a default name inference mechanism based on the fully qualified name of the function (including any | ||
| parent types and the module path), but it is also possible to explicitly specify the name (e.g. to avoid conflicts, make | ||
| a backward-compatible change, or to provide a more user-friendly name) using the `link_name` argument of the `@guppy` | ||
| decorator: | ||
|
|
||
| ```python | ||
| from guppylang import guppy | ||
|
|
||
|
|
||
| @guppy(link_name="my.custom.link.name") | ||
| def my_func() -> None: | ||
| pass | ||
| ``` | ||
|
|
||
| The following are not yet supported: | ||
|
|
||
| - Exporting generic functions of any kind (see the [proposal](proposals/generic.md)) | ||
| - Automatically collecting functions to be included in a library (see the [proposal](proposals/collection.md)) | ||
|
|
||
| ## Compiling a library and creating stubs | ||
|
|
||
| Once a library object is created, it can be compiled into a Hugr package, which can be distributed and used | ||
| independently of the source code: | ||
|
|
||
| ```python | ||
| from guppylang import guppy | ||
| from hugr.package import Package | ||
|
|
||
| library = guppy.library(...) # As above | ||
|
|
||
| # Hugr package contains any specified functions as public, and otherwise reachable functions as private | ||
| hugr_package: Package = library.compile() | ||
| with open("library.hugr", "wb") as f: | ||
| f.write(hugr_package.to_bytes()) | ||
| ``` | ||
|
|
||
| However, this Hugr package does not expose an interface that can be programmed against by consumers of the library *in | ||
| Guppy*. For this, the library author must define *stubs* for all exposed functions, utilising `@guppy.declare(...)` | ||
| decorators. | ||
|
|
||
| For the example above, this would look like: | ||
|
|
||
| ```python | ||
| from guppylang import guppy | ||
|
|
||
|
|
||
| @guppy.declare | ||
| def my_func() -> None: ... | ||
|
|
||
|
|
||
| @guppy.declare | ||
| def another_func(x: int) -> None: ... | ||
|
|
||
|
|
||
| @guppy.declare | ||
| def a_third_func(x: int) -> int: ... | ||
| ``` | ||
|
|
||
| Creation of these stubs is a manual process; it is currently not possible to automatically generate stubs for a | ||
| library or validate that definitions faithfully implement their corresponding stubs (see the | ||
| [proposal](proposals/stubs.md) for both). Furthermore, it is currently not possible to define the stubs, and | ||
| subsequently reference the stubs in the library function definitions for easier consistency checks (see the | ||
| [proposal](proposals/impls.md) for that). | ||
|
|
||
| Finally, these stubs (in `*.pyi` files) should be distributed using regular Python packaging mechanisms, so that users | ||
| of the library can install and program against them. This distribution may or may not contain the Hugr package as well. | ||
|
|
||
| ## Using a library | ||
|
|
||
| Let's call the library in the example above `super_guppy`. | ||
| That is, the library author has published a distribution containing the stubs above, to be imported `from super_guppy`. | ||
| A consumer of the library has installed that distribution using their favourite package manager (either from an index, | ||
| or by downloading the stub repository). | ||
|
|
||
| The consumer may now program against the API as follows: | ||
|
|
||
| ```python | ||
| from guppylang import guppy | ||
| from guppylang.std.builtins import result | ||
| from super_guppy import my_func, a_third_func | ||
|
|
||
|
|
||
| @guppy | ||
| def consumer_func() -> None: | ||
| my_func() | ||
| result("library_call", a_third_func(5)) | ||
|
|
||
|
|
||
| @guppy | ||
| def main() -> None: | ||
| consumer_func() | ||
| ``` | ||
|
|
||
| In this case, the consumer aims to create an executable Hugr package (e.g. by calling `main.compile()` and creating a | ||
| package with a single, argument-less entrypoint). However, the created Hugr package is incomplete: It lacks the function | ||
| bodies of the library functions, and thus cannot be executed. | ||
|
|
||
| Thus, `hugr-py` MUST provide means to link the library Hugr package into the consumer Hugr package. For convenience, the | ||
maximilianruesch marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| library Hugr package may be provided to the consumer program executor, so that it can be automatically linked in before | ||
| lowering to QIR. For example, using selene, this may look like: | ||
ss2165 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ```python | ||
| main.emulator(n_qubits=0, libraries=[hugr_package]).with_shots(100).run() | ||
| ``` | ||
|
|
||
| Currently, Hugr packages have to be manually downloaded / imported from whatever distribution mechanism the library | ||
| author chose. In the future, library authors may opt to distribute Hugr packages in Python wheels as well, and have | ||
| consuming code auto-collect these from the Python environment (see the [proposal](proposals/discovery.md) for this). | ||
|
|
||
| ```{note} | ||
| A consumer of `super_guppy` may as well be another library author. Dependency of a library should be specified as usual | ||
| in Python requirements by depending on the header distributions (or through other common mechanisms). | ||
| ``` | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| # Proposals | ||
|
|
||
| A collection of proposals for QOL features related to Guppy libraries. | ||
|
|
||
| ```{toctree} | ||
| :glob: | ||
|
|
||
| proposals/* | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| # (Proposal) More ways to define a library / auto-collection | ||
|
|
||
| Currently, functions must be manually fed into `guppy.library(...)` to be included in the library. This proposal | ||
| concerns additional ways to define (members of) a library, and automatically collect functions to be included in it. | ||
|
|
||
| A potential addition would be a `@guppy(export=...)` argument, that registers the function to be included in *the* | ||
| library in some global Guppy store (like the `DEF_STORE`). A function could then be included simply by adding that | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. a "global library" is a bit confusing - this still would require some notion of a specific library the function is exported in. The string method below addresses this, but passing in library objects rather than strings seems preferable
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This seems very similar to how GuppyModules were handled. |
||
| argument, e.g.: | ||
|
|
||
| ```python | ||
| from guppylang import guppy | ||
|
|
||
|
|
||
| @guppy(export=True) | ||
| def included() -> None: | ||
| ... | ||
|
|
||
|
|
||
| @guppy # or @guppy(export=False) for explicit exclusion | ||
| def not_included() -> None: | ||
| ... | ||
| ``` | ||
|
|
||
| This is similar to a `public` / `private` modifier, and indirectly determines the visibility of the function in the Hugr | ||
| package. It may be added to `@guppy.struct` and `@guppy.enum` as well, to include the functions on these constructs in | ||
| the library as public as well. | ||
|
|
||
| A call to `guppy.library` could then be simplified to an explicit opt-in to auto-collection: | ||
|
|
||
| ```python | ||
| from guppylang import guppy | ||
|
|
||
| # ... the functions above ... | ||
|
|
||
| # Includes `included`, but not `not_included` as a public member. | ||
| library = guppy.library(auto_collect=True) | ||
| ``` | ||
|
|
||
| ## Optional: Keys | ||
|
|
||
| In addition to supporting `True | False | None` for the `export` argument, it may be useful to support string keys as | ||
| well, to allow for more fine-grained control over the produced packages in which the function is included (concerning | ||
| both the visibility in the produced Hugr package and the stubs that are generated, e.g. | ||
| through [the stub proposal](stubs.md)). | ||
|
|
||
| For example, one may want to include certain (core) functions in all versions of the library, but specify certain sets | ||
| of functions to be included (akin to `extras` from Python packages): | ||
|
|
||
| ```python | ||
| from guppylang import guppy | ||
|
|
||
|
|
||
| @guppy(export=True) | ||
| def included_in_all_libs() -> None: | ||
| ... | ||
|
|
||
|
|
||
| @guppy(export='key1') | ||
| def included_when_key_1() -> None: | ||
| ... | ||
|
|
||
|
|
||
| @guppy(export='key2') | ||
| def included_when_key_2() -> None: | ||
| ... | ||
|
|
||
|
|
||
| @guppy(export='key3') | ||
| def included_when_key_3() -> None: | ||
| ... | ||
| ``` | ||
|
|
||
| Auto-collection may then take a range of keys to include: | ||
|
|
||
| ```python | ||
| from guppylang import guppy | ||
|
|
||
| # ... the functions above ... | ||
|
|
||
| # Includes all functions with export=True, export='key1', and export='key2', but not export='key3' | ||
| library = guppy.library(auto_collect=['key1', 'key2']) | ||
| ``` | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| # (Proposal) Automatic discovery of Hugr packages in installed distributions | ||
|
|
||
| This proposal concerns how Hugr packages are distributed and discovered. | ||
| At a high level, Hugr packages should be distributed as part of Python wheels, and discovered by Guppy at runtime | ||
| without the need for manual configuration or other custom user code. | ||
|
|
||
| The proposal is to leverage the | ||
| Python [entry points mechanism](https://packaging.python.org/en/latest/specifications/entry-points/) for this (as one of | ||
| the common drivers behind | ||
| [plugin discovery](https://packaging.python.org/en/latest/guides/creating-and-discovering-plugins/#using-package-metadata) | ||
| in Python). Library authors can define an entry point in the group `guppylang.hugr` in their `pyproject.toml`, and | ||
| specify a loader function that returns a list of paths to Hugr packages to be loaded. | ||
|
|
||
| For example, assuming the library was compiled to a `library.hugr` binary file, a library author may include the | ||
| following in a top-level `loader.py` inside their distribution: | ||
|
|
||
| ```python | ||
| from pathlib import Path | ||
|
|
||
|
|
||
| def load_hugrs() -> list[Path]: | ||
| return [Path(__file__).parent / "library.hugr"] | ||
| ``` | ||
|
|
||
| Additionally, they would include the `library.hugr` file, and the following configuration in the `pyproject.toml` of | ||
| their distribution (e.g. `super-guppy-hugr`): | ||
|
|
||
| ```toml | ||
| [project] | ||
| name = "super-guppy-hugr" | ||
| version = "0.1.0" | ||
| description = "HUGR distribution for {self.metadata.name}" | ||
|
|
||
| [project.entry-points."guppylang.hugr"] | ||
| hugr_loader = "loader:load_hugrs" | ||
| ``` | ||
|
|
||
| ```{note} | ||
| Hugr packages may also be included in stub distributions, and discovered in the same way by including the loader and | ||
| the entry point configuration. They do not have to be separate distributions. | ||
| ``` | ||
|
|
||
| `guppylang` can now auto-discover Hugr packages by looking for entry points in the `guppylang.hugr` group, and calling | ||
| the corresponding loader. An example implementation of this discovery mechanism may look like the following: | ||
|
|
||
| ```python | ||
| from pathlib import Path | ||
| from importlib.metadata import entry_points | ||
|
|
||
|
|
||
| def discover_hugr_packages() -> list[Path]: | ||
| eps = entry_points(group='guppylang.hugr') | ||
| hugr_packages = [] | ||
| for ep in eps: | ||
| # Can use `ep.dist.name` to get the distribution name, if needed for logging, debugging, or filtering. | ||
| loader_func = ep.load() | ||
| hugr_packages.extend(loader_func()) | ||
|
|
||
| return hugr_packages | ||
| ``` | ||
|
|
||
| Finally, consumers of the library would simply install the `super-guppy-hugr` distribution, and call the discovery | ||
| function before executing their Hugr code, to ensure that the library Hugr package is linked in: | ||
|
|
||
| ```python | ||
| from guppylang.library import discover_hugr_packages | ||
|
|
||
| main = ... # Define some consuming entry point | ||
|
|
||
| main.emulator(n_qubits=..., libraries=discover_hugr_packages()).with_shots(100).run() | ||
| ``` | ||
|
|
||
| Optionally, omission of `libraries` may lead to internal calls to automatic discovery, and other means of configuring | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. what about instead just refer to library by name, which is then looked up in the discovered set, seems more familiar to the search path/
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think all three should be allowed (pass hugr, pass name (which is looked up), and passing nothing which means all known hugrs get thrown in). |
||
| the used Hugr packages may be provided as well (e.g. environment variables, or a configuration file). | ||
|
|
||
| ## Open questions | ||
|
|
||
| - How does a user sort out which Hugr packages to use with multiple Hugr packages for the same stubs installed? | ||
| It is unclear how this case can be discovered (e.g. for a nice error message if they do not choose). | ||
Uh oh!
There was an error while loading. Please reload this page.