docs: update README and DEVELOPMENT.md to reflect current repo structure (#14)

Agent-Logs-Url: https://github.com/Quantinuum/hugrverse-env/sessions/0be68f6f-4b24-4d08-9e80-f28c454e0612

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: jake-arkinstall <65358059+jake-arkinstall@users.noreply.github.com>

Author: Copilot

DEVELOPMENT.md

@@ -10,7 +10,7 @@ sandboxed environment that CI uses.
   (e.g. [QEMU binfmt](https://github.com/multiarch/qemu-user-static) or
   Docker Desktop's built-in emulation). Building natively is strongly
   recommended for AArch64 because emulation will be very slow for a full
-  LLVM build.
+  LLVM or tket build.
 
 ## Linux (manylinux) builds
 
@@ -19,6 +19,9 @@ images. They do not embed the build scripts; instead the repository root is
 mounted at `/host` at runtime so that edits are picked up immediately without
 rebuilding the image.
 
+Each build script lives at `deps/<dependency>/<platform>.sh` and takes a single
+argument: the path where the output `.tar.gz` should be written.
+
 ### x86_64
 
 ```bash
@@ -28,15 +31,18 @@ docker build \
   -t hugrverse-env-manylinux-x86_64 \
   .
 
-# 2. Run the full build (output lands in ./artifacts/)
+# 2. Run a single dependency build (e.g. llvm)
 mkdir -p artifacts
 docker run --rm \
   -v "$(pwd):/host" \
   hugrverse-env-manylinux-x86_64 \
-  /host/builds/manylinux_2_28_x86_64/build.sh \
-  /host/artifacts/hugrverse_env_manylinux_2_28_x86_64.tar.gz
+  bash /host/deps/llvm/manylinux_2_28_x86_64.sh \
+       /host/artifacts/hugrenv-llvm-manylinux_2_28_x86_64.tar.gz
 ```
 
+Replace `llvm` with `tket` (or any other dependency) to build that component
+instead.
+
 ### AArch64 (native AArch64 host recommended)
 
 ```bash
@@ -49,8 +55,8 @@ mkdir -p artifacts
 docker run --rm \
   -v "$(pwd):/host" \
   hugrverse-env-manylinux-aarch64 \
-  /host/builds/manylinux_2_28_aarch64/build.sh \
-  /host/artifacts/hugrverse_env_manylinux_2_28_aarch64.tar.gz
+  bash /host/deps/llvm/manylinux_2_28_aarch64.sh \
+       /host/artifacts/hugrenv-llvm-manylinux_2_28_aarch64.tar.gz
 ```
 
 > **Tip:** pass `-e VERBOSE=1` to enable verbose output from build sub-steps,
@@ -65,56 +71,66 @@ build:
 docker run --rm -it \
   -v "$(pwd):/host" \
   hugrverse-env-manylinux-x86_64 \
-  -c 'cd /host && bash'
+  bash
 ```
 
-You can then run individual sub-scripts (e.g.
-`bash builds/manylinux_2_28_x86_64/llvm/build.sh`) to test in isolation.
+You can then run individual scripts (e.g.
+`bash /host/deps/llvm/manylinux_2_28_x86_64.sh /tmp/hugrenv-llvm-manylinux_2_28_x86_64.tar.gz`)
+to test in isolation.
 
 ## macOS builds
 
-macOS build scripts run natively; no Docker is needed.
+macOS build scripts run natively; no Docker is needed. The `MACOSX_DEPLOYMENT_TARGET`
+environment variable should be set to `11.0` to match what CI uses.
 
 ```bash
+export MACOSX_DEPLOYMENT_TARGET=11.0
+
 # ARM64
-bash builds/macosx_11_0_arm64/build.sh /tmp/hugrverse_env_macosx_11_0_arm64.tar.gz
+bash deps/llvm/macosx_11_0_aarch64.sh /tmp/hugrenv-llvm-macosx_11_0_aarch64.tar.gz
 
 # x86_64 (Intel Mac)
-bash builds/macosx_11_0_x86_64/build.sh /tmp/hugrverse_env_macosx_11_0_x86_64.tar.gz
+bash deps/llvm/macosx_11_0_x86_64.sh /tmp/hugrenv-llvm-macosx_11_0_x86_64.tar.gz
 ```
 
-The scripts will install any missing dependencies via Homebrew.
+Replace `llvm` with `tket` to build the tket dependency. The scripts will
+install any missing dependencies via Homebrew.
 
 ## Windows builds
 
-Open a **Developer PowerShell for VS 2022** (or run `.\builds\win_amd64\build.ps1`
-from a terminal that already has the MSVC environment loaded):
+Windows scripts are standard Bash scripts that must be run from a shell that
+has the MSVC environment loaded (e.g. Git Bash launched from a **Developer
+Command Prompt for VS 2022**, or after sourcing `vcvars64.bat`). CI uses the
+`ilammy/msvc-dev-cmd` action to set this up automatically.
 
-```powershell
-.\builds\win_amd64\build.ps1 -OutputPath C:\Temp\hugrverse_env_win_amd64.zip
+```bash
+bash deps/llvm/win_amd64.sh /tmp/hugrenv-llvm-win_amd64.tar.gz
+bash deps/tket/win_amd64.sh /tmp/hugrenv-tket-win_amd64.tar.gz
 ```
 
-> The MSVC environment can be loaded in a regular PowerShell session with the
-> `ilammy/msvc-dev-cmd` action logic, or by sourcing
+> The MSVC environment can be loaded in a regular PowerShell session by
+> sourcing
 > `"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvars64.bat"`.
 
-## Adding a new component
+## Adding a new dependency
 
-1. Create `builds/<platform>/<component>/build.sh` (or `.ps1`).
-2. Source it from the platform's parent `build.sh` / `build.ps1`.
-3. Test locally using the methods above before opening a PR.
-4. Update `README.md` to document the new component.
+1. Create a `deps/<dependency>/` directory with a `<platform>.sh` script for
+   each supported platform. Each script must accept the output tarball path as
+   its first argument.
+2. Test locally using the methods above before opening a PR.
+3. Update `README.md` to document the new component.
 
 ## Verifying the archive
 
 After a local build, inspect the archive to confirm its contents:
 
 ```bash
 # Linux / macOS
-tar -tzf artifacts/hugrverse_env_manylinux_2_28_x86_64.tar.gz | head -20
+tar -tzf artifacts/hugrenv-llvm-manylinux_2_28_x86_64.tar.gz | head -20
 
 # Windows
-unzip -l artifacts\hugrverse_env_win_amd64.zip | head -20
+tar -tzf /tmp/hugrenv-llvm-win_amd64.tar.gz | head -20
 ```
 
-Installed LLVM binaries should appear under `opt/llvm/bin/` in the archive.
+LLVM binaries should appear under `hugrverse/bin/` in the archive; tket
+libraries under `hugrverse/lib/`.

README.md

@@ -3,81 +3,127 @@
 CI environment bootstrapping for hugrverse projects.
 
 This repository builds and releases pre-compiled tooling environments used by
-other repositories in the Quantinuum hugrverse. Each release contains a
-compressed archive (`.tar.gz` or `.zip`) of pre-built tools for a specific
-target platform. Downstream projects download these archives during their CI
-bootstrap phase so that they never need to compile heavyweight dependencies
-from scratch.
+other repositories in the Quantinuum hugrverse. The goal is to provide
+libraries that are as compatible as possible across the platforms that matter
+most to Rust/Python build pipelines. On Linux, libraries are built inside
+[manylinux 2.28](https://github.com/pypa/manylinux) containers, giving broad
+binary compatibility across a wide range of distributions. On macOS the
+deployment target is set to 11.0.
+
+Each release attaches a compressed archive (`hugrenv-<dep>-<platform>.tar.gz`)
+for every supported dependency/platform combination, plus a `hugrenv.lock` file
+that records the release version and the Nix-style SHA-256 hash of every
+archive.
 
 ## What is built
 
 | Component | Version | Description |
 |-----------|---------|-------------|
 | LLVM      | 21.1.8  | Clang + LLD + LLVM core libraries |
+| tket      | 2.16.0  | Quantinuum tket quantum compiler + C API |
 
 ## Supported platforms
 
-| Platform tag              | Runner                        | Archive format |
-|---------------------------|-------------------------------|----------------|
-| `manylinux_2_28_x86_64`   | Docker `quay.io/pypa/manylinux_2_28_x86_64` on `ubuntu-latest` | `.tar.gz` |
-| `manylinux_2_28_aarch64`  | Docker `quay.io/pypa/manylinux_2_28_aarch64` on `ubuntu-24.04-arm` | `.tar.gz` |
-| `macosx_11_0_aarch64`     | `macos-14`                    | `.tar.gz` |
-| `macosx_11_0_x86_64`      | `macos-15-intel`              | `.tar.gz` |
-| `win_amd64`               | `windows-latest` + MSVC       | `.zip` |
+| Platform tag              | Runner                                                          | Notes |
+|---------------------------|-----------------------------------------------------------------|-------|
+| `manylinux_2_28_x86_64`   | Docker `quay.io/pypa/manylinux_2_28_x86_64` on `ubuntu-latest` | manylinux 2.28 — broad Linux compatibility |
+| `manylinux_2_28_aarch64`  | Docker `quay.io/pypa/manylinux_2_28_aarch64` on `ubuntu-24.04-arm` | manylinux 2.28 — broad Linux compatibility |
+| `macosx_11_0_aarch64`     | `macos-14`                                                      | `MACOSX_DEPLOYMENT_TARGET=11.0` |
+| `macosx_11_0_x86_64`      | `macos-15-intel`                                                | `MACOSX_DEPLOYMENT_TARGET=11.0` |
+| `win_amd64`               | `windows-latest` + MSVC                                         | |
+
+## Using hugrverse-env in your repository
+
+### 1. Obtain a `hugrenv.lock` file
+
+Every [GitHub release](https://github.com/Quantinuum/hugrverse-env/releases)
+attaches a `hugrenv.lock` file alongside the compiled archives. Download the
+lock file for the version you want to pin and **commit it to your repository**
+(e.g. at the repository root as `hugrenv.lock`). The lock file records the
+release version and a content hash for every archive so downstream workflows
+can verify what they download.
+
+### 2. Install hugrenv packages in GitHub Actions
+
+Use the `.github/actions/install-hugrenv` composite action from this
+repository. It reads your committed `hugrenv.lock`, detects the current runner
+platform, downloads the requested packages from the matching release, extracts
+them, and sets the relevant environment variables (`LLVM_SYS_211_PREFIX`,
+`LIBCLANG_PATH`, `TKET_C_API_PATH`, `LD_LIBRARY_PATH` / `DYLD_LIBRARY_PATH`,
+`PATH`, etc.) for subsequent steps.
+
+```yaml
+- name: Install hugrenv
+  uses: Quantinuum/hugrverse-env/.github/actions/install-hugrenv@main
+  with:
+    packages: llvm,tket   # comma-separated; defaults to "llvm,tket"
+    lockfile: hugrenv.lock  # relative path to your lock file; default is "hugrenv.lock"
+```
+
+The action supports all five platforms listed above and works on Linux, macOS,
+and Windows runners without any additional setup.
 
 ## Repository layout
 
 ```
-builds/
-  <platform>/        # One directory per target platform
-    build.sh         # Parent script: builds all components, then bundles output
-    version.txt      # Managed by release-please; records the current version
-    llvm/
-      build.sh       # Builds LLVM from source into /opt/llvm (or C:\hugrverse\llvm)
+deps/
+  llvm/
+    manylinux_2_28_x86_64.sh   # Builds LLVM for each platform
+    manylinux_2_28_aarch64.sh
+    macosx_11_0_aarch64.sh
+    macosx_11_0_x86_64.sh
+    win_amd64.sh
+  tket/
+    manylinux_2_28_x86_64.sh   # Builds tket + C API for each platform
+    manylinux_2_28_aarch64.sh
+    macosx_11_0_aarch64.sh
+    macosx_11_0_x86_64.sh
+    win_amd64.sh
 docker/
   manylinux_2_28_x86_64.Dockerfile   # Local-testing image for x86_64 Linux
   manylinux_2_28_aarch64.Dockerfile  # Local-testing image for AArch64 Linux
 .github/
+  actions/
+    install-hugrenv/
+      action.yml    # Composite action: reads lock file, downloads & installs packages
   workflows/
-    build.yml             # Builds each platform; uploads artifact on release
-    release-please.yml    # Creates release PRs and tags via release-please
-release-please-config.json       # Monorepo release-please configuration
-.release-please-manifest.json    # Current version for each package
+    build.yml       # Builds changed targets on PRs; uploads artifacts
+    merge.yml       # Promotes merged-PR artifacts to main-* on merge to main
+    release.yml     # Bundles main-* artifacts, generates hugrenv.lock, publishes release
 ```
 
 ## How CI works
 
-1. **Build workflow** (`build.yml`) — triggered on every pull request, push to
-   `main`, published GitHub release, and manually via `workflow_dispatch`.  
+1. **Build** (`build.yml`) — triggered on pull requests and manually via
+   `workflow_dispatch`.
+   * On a PR, only scripts that changed since the base commit are built.
    * Linux targets are built inside the official `quay.io/pypa/manylinux_2_28_*`
-     Docker images to ensure binary compatibility with the manylinux ABI.
-   * macOS targets run natively on the appropriate GitHub-hosted runners.
-   * The Windows target runs with the MSVC toolchain on `windows-latest`.
-   * On a release event the resulting archives are also uploaded directly to
-     the GitHub release.
-
-2. **Release workflow** (`release-please.yml`) — triggered on every push to
-   `main`.  
-   * Uses [release-please](https://github.com/googleapis/release-please) in
-     monorepo mode with five independent packages, one per platform.
-   * When commits affecting a platform directory are merged to `main`,
-     release-please opens a release PR for that platform.
-   * Merging the release PR creates a tag (e.g.
-     `hugrverse-env-manylinux_2_28_x86_64-v1.2.3`) and a GitHub release.
-   * The build workflow then triggers on the published release and attaches the
-     compiled archive.
-
-## Adding a new component
-
-1. Create a `builds/<platform>/<component>/build.sh` (or `.ps1` for Windows)
-   that downloads, compiles, and installs the component to `/opt/<component>`
-   (or `C:\hugrverse\<component>` on Windows).
-2. Call the new script from `builds/<platform>/build.sh` and, if needed, add
-   the install directory to the `tar` / `Compress-Archive` invocation that
-   creates the bundle.
-3. Repeat for every platform that should include the component.
+     Docker images to ensure manylinux binary compatibility.
+   * macOS targets run natively; `MACOSX_DEPLOYMENT_TARGET` is set to `11.0`.
+   * The Windows target uses the MSVC toolchain via `ilammy/msvc-dev-cmd`.
+   * Each built archive is uploaded as a PR artifact named
+     `pr-<head-sha>-<dep>-<platform>`.
+
+2. **Merge** (`merge.yml`) — triggered when a PR is merged to `main`.
+   * Finds the PR build artifacts by head SHA and re-uploads them as
+     `main-<dep>-<platform>` artifacts (retained 90 days).
+
+3. **Release** (`release.yml`) — triggered manually via `workflow_dispatch`.
+   * Downloads all `main-*` promoted artifacts.
+   * Generates a `hugrenv.lock` containing the version and Nix-style SHA-256
+     hashes for every archive.
+   * Creates a GitHub release tagged `v<version>` and attaches all archives
+     plus the lock file.
+
+## Adding a new dependency
+
+1. Create `deps/<dependency>/` and add a `<platform>.sh` script for each
+   supported platform. Each script accepts an output tarball path as its first
+   argument and produces a `hugrenv-<dependency>-<platform>.tar.gz`.
+2. Repeat for every platform that should include the dependency.
+3. Update this README to document the new component.
 
 ## Local development
 
-See [DEVELOPMENT.md](DEVELOPMENT.md) for instructions on testing changes
-locally using Docker.
+See [DEVELOPMENT.md](DEVELOPMENT.md) for instructions on testing build-script
+changes locally using Docker (Linux) or running scripts natively (macOS/Windows).