Skip to content

Commit cde63fd

Browse files
One shared package selection for all distributions (vinca extends, ros2-* names, no kilted) (#8)
See #8 for the full description. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 02f5345 commit cde63fd

1,216 files changed

Lines changed: 16950 additions & 21634 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.claude/skills/robostack-debug-build/SKILL.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ bash -x conda_build.sh 2>&1 | less
6666
- **gtest / test failures**: add the missing dependency in `dependencies.yaml`, or
6767
disable the tests when safe; no custom shims.
6868
- **Qt plugins (rviz, rtabmap)**: make sure CMake finds the intended Qt major version.
69-
- **Missing dependency**: `shared/dependencies.yaml` (all distributions) or
69+
- **Missing dependency**: `shared/patch/dependencies.yaml` (all distributions) or
7070
`distros/<d>/patch/dependencies.yaml` (one), keyed by ROS package name:
7171
```yaml
7272
<ros_pkg>:
@@ -75,9 +75,10 @@ bash -x conda_build.sh 2>&1 | less
7575
```
7676
- **A conda-forge package is shadowed**: when a rosdep key that is also a ROS package
7777
maps to a conda-forge package (e.g. `tl_expected` -> `cpp-expected`), vinca may build
78-
the ROS package under the conda-forge name. Add it to `packages_skip_by_deps`
79-
(`shared/vinca.yaml`), and evict the stale build from the PR cache
80-
(`evict_cache` in `distros/<d>/ci.yaml`, plain names work).
78+
the ROS package under the conda-forge name. vinca skips ROS packages mapped in
79+
`shared/robostack.yaml` automatically; if it still happens, check the mapping, and
80+
evict a stale build from the PR cache (`evict_cache` in `distros/<d>/ci.yaml`, plain
81+
names work).
8182
- **CMake args for one package**: `additional_cmake_args` in `pkg_additional_info.yaml`.
8283

8384
## 5. Make it stick
Lines changed: 36 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -1,51 +1,54 @@
11
---
22
name: robostack-package-selection
3-
description: Add, exclude or restrict ROS packages in RoboStack vinca.yaml files (shared/ and distros/<d>/), including platform-only packages, dependency skips and the build gap report. Use when a package should be added, removed, made Linux-only, or when recipes/artifacts don't line up.
3+
description: Add, exclude or restrict ROS packages in RoboStack vinca.yaml files (shared/ selection, packages_exclude / packages_skip per distribution), including platform-only packages and the build gap report. Use when a package should be added, removed, made Linux-only, or when recipes/artifacts don't line up.
44
---
55

66
# Package selection (vinca.yaml)
77

88
`distros/<d>/work/vinca.yaml` is assembled from `shared/vinca.yaml` plus
9-
`distros/<d>/vinca.yaml`. The three package lists are concatenated; for other keys
10-
the distribution wins. A package selected for every distribution goes into
11-
`shared/vinca.yaml`.
9+
`distros/<d>/vinca.yaml` (`tools/robostack.py`, `merge_vinca`): the package lists are
10+
combined, vinca applies them.
1211

13-
## Add packages
12+
## Selecting packages
13+
14+
All selections are in **`shared/vinca.yaml`** (`packages_select_by_deps`), with the
15+
broadest platform condition that works:
16+
17+
```yaml
18+
packages_select_by_deps:
19+
- some_package
20+
- if: linux
21+
then:
22+
- some_linux_only_package
23+
```
24+
25+
vinca skips a selected package that a distribution's rosdistro doesn't have ("not in
26+
available packages anymore"), so a package that only some distributions have is
27+
still selected in `shared/`. Add packages with the bot or:
1428

1529
```bash
16-
pixi run rs add-package foxglove_bridge rqt_gauges # every distribution that has them
30+
pixi run rs add-package foxglove_bridge rqt_gauges # lists what each distribution builds
1731
pixi run rs add-package foxglove_bridge humble jazzy --preview
1832
```
1933

20-
This is the bot's `add-package`. It inserts in sorted position, chooses `shared/` when
21-
every distribution gets the same packages, and lists what will be built on linux-64.
22-
Names are ROS package names; dashes and underscores both work.
34+
## Exceptions per distribution (or for all, in shared/)
2335

24-
## The three ways to exclude a package
36+
Both keys take plain names (everywhere) or `- if: <condition>` / `then:` blocks (on
37+
those platforms only), and both drop the package from the shared selection there:
2538

26-
vinca's revision is pinned in `pixi.toml`. If that pin moves, re-check
27-
`vinca/main.py` and `vinca/resolve.py`.
39+
- **`packages_exclude`**: never built, and dropped from the dependencies of every other
40+
package (vinca's `packages_skip_by_deps` + `packages_remove_from_deps`). The usual
41+
choice: a package that doesn't build on a platform, and nothing needs it.
42+
- **`packages_skip`**: never built, but dependents keep depending on it (vinca's
43+
`packages_skip_by_deps` only), e.g. when an already published build must be used
44+
(rolling's `rviz_visual_tools`, which `moveit_visual_tools` needs).
45+
46+
ROS packages whose name `shared/robostack.yaml` maps to a conda-forge package (e.g.
47+
`tl_expected` -> `cpp-expected`) are skipped automatically, so the conda-forge package
48+
is used and not shadowed by a ROS build of the same name.
2849

29-
1. **`packages_select_by_deps` under a platform condition**: the primary way to keep a
30-
package's own recipe off a platform. Every listed name is selected
31-
unconditionally, so not listing it for a platform is what excludes it:
32-
```yaml
33-
packages_select_by_deps:
34-
- if: linux
35-
then:
36-
- some_linux_only_pkg
37-
```
38-
2. **`packages_skip_by_deps`**: only stops transitive pull-in (`ignore_pkgs` for
39-
`distro.get_depends()`). It doesn't stop a package listed directly in (1). It's
40-
also the tool for ROS packages that map to conda-forge in `shared/robostack.yaml`
41-
(`tl_expected`, `sophus`), so that vinca doesn't build them under the conda-forge
42-
name.
43-
3. **`packages_remove_from_deps`**: removes the package's own recipe *and* strips it
44-
from every other package's host/run dependencies, together and inseparably. Use
45-
it only if nothing selected legitimately needs it; otherwise use (1) + (2).
46-
47-
Keep (2) and (3) coherent with the platform conditions in (1). A truly platform-only
48-
package goes behind a condition, not into a comment.
50+
vinca rejects the old `packages_skip_by_deps` / `packages_remove_from_deps` keys. vinca's revision is pinned in `pixi.toml`; if that pin moves, re-check
51+
`vinca/main.py` and `vinca/resolve.py`.
4952

5053
## Recipes vs. artifacts
5154

@@ -65,5 +68,5 @@ Regenerate recipes after editing `vinca.yaml` before expecting the report to cha
6568
## rosdep mappings
6669

6770
`shared/robostack.yaml` maps rosdep keys to conda packages (per platform);
68-
`shared/packages-ignore.yaml` lists keys to drop. Changes there affect every
71+
`robostack: []` drops a key. Changes there affect every
6972
distribution: check them with `generate-recipes` for at least two distributions.

‎.claude/skills/robostack-patch/SKILL.md‎

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,28 +7,28 @@ description: Create, refresh, place, port and validate source patches for RoboSt
77

88
## Where they live
99

10-
- `distros/<d>/patch/ros-<d>-<pkg>.patch` (all platforms), or with a platform suffix
10+
- `distros/<d>/patch/ros2-<pkg>.patch` (all platforms), or with a platform suffix
1111
`.osx.patch`, `.linux.patch`, `.win.patch`, `.unix.patch`. `<pkg>` uses dashes.
1212
- One patch per package and platform. Merge new hunks into the existing file instead
1313
of adding variants.
1414
- vinca picks them up by name when generating recipes; nothing else needs wiring.
15-
`pixi run rs check` rejects names of other distributions.
15+
`pixi run rs check` rejects other names.
1616
- `distros/<d>/patch/dependencies.yaml` is for dependency fixes, not source patches
17-
(`shared/dependencies.yaml` for all distributions).
17+
(`shared/patch/dependencies.yaml` for all distributions).
1818

1919
## Create a patch from work-directory edits
2020

2121
```bash
2222
WORK=$(ls -td distros/<d>/work/output/bld/rattler-build_ros2-<pkg>_*/work | head -1)
2323
cd "$WORK"
2424
# edit the sources, rebuild with `bash conda_build.sh` (after `source build_env.sh`)
25-
rattler-build create-patch --directory . --name ros-<d>-<pkg> \
25+
rattler-build create-patch --directory . --name ros2-<pkg> \
2626
--exclude "*.o,*.so,*.dylib,*.a,*.pyc,__pycache__,build/" --dry-run
27-
rattler-build create-patch --directory . --name ros-<d>-<pkg> \
27+
rattler-build create-patch --directory . --name ros2-<pkg> \
2828
--exclude "*.o,*.so,*.dylib,*.a,*.pyc,__pycache__,build/"
2929
```
3030

31-
Move the result to `distros/<d>/patch/ros-<d>-<pkg>.patch`, merging it with an
31+
Move the result to `distros/<d>/patch/ros2-<pkg>.patch`, merging it with an
3232
existing patch for that package if there is one. Then:
3333

3434
```bash
@@ -59,7 +59,8 @@ it, in which case delete the patch.
5959
- Port only to an existing package with a compatible source. Compare the versions in
6060
`distros/<d>/rosdistro_snapshot.yaml` and check with `check-patches`. Never copy a
6161
patch only because a file of that name exists elsewhere.
62-
- Rename to the target distribution (`ros-<other>-<pkg>.patch`).
62+
- The file name stays the same in every distribution (`ros2-<pkg>.patch`), so
63+
`diff distros/{jazzy,humble}/patch/ros2-<pkg>.patch` shows how two distributions differ.
6364
- After the PR's first build, add the package to `evict_cache` in the target's
6465
`ci.yaml` if CI already cached it.
6566
- Fixes that belong upstream: open a PR on the package's repository and note it in the
Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ body:
1616
id: distribution
1717
attributes:
1818
label: ROS distribution
19-
options: [rolling, lyrical, kilted, jazzy, humble]
19+
options: [rolling, lyrical, jazzy, humble]
2020
validations:
2121
required: true
2222

@@ -34,9 +34,9 @@ body:
3434
attributes:
3535
label: Installed packages
3636
description: |
37-
Please share your installed packages by running `mamba list` and entering the output below:
37+
Please share your installed packages by running `pixi list` in your project directory and entering the output below:
3838
_Note:_ This will be automatically formatted as code.
39-
placeholder: "mamba list"
39+
placeholder: "pixi list"
4040
render: shell
4141
validations:
4242
required: true
@@ -46,9 +46,9 @@ body:
4646
attributes:
4747
label: Environment info
4848
description: |
49-
Please share your environment by running `mamba info` and entering the output below:
49+
Please share your environment by running `pixi info` in your project directory and entering the output below:
5050
_Note:_ This will be automatically formatted as code.
51-
placeholder: "mamba info"
51+
placeholder: "pixi info"
5252
render: shell
5353
validations:
5454
required: true

.github/ISSUE_TEMPLATE/package-request.yml renamed to .github/ISSUE_TEMPLATE/2_package_request.yaml

Lines changed: 4 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,10 @@ body:
55
- type: markdown
66
attributes:
77
value: |
8-
robostack-bot opens a pull request that adds the package to the build, and
9-
replies here with what will be built (per distribution, including new
10-
dependencies). A maintainer reviews and merges it; the issue closes with it.
8+
robostack-bot opens a pull request that adds the package to the build of every
9+
distribution that has it, and replies here with what will be built (per
10+
distribution, including new dependencies). A maintainer reviews and merges it;
11+
the issue closes with it.
1112
- type: input
1213
id: package_name
1314
attributes:
@@ -16,16 +17,6 @@ body:
1617
placeholder: foxglove_bridge, rqt_gauges
1718
validations:
1819
required: true
19-
- type: dropdown
20-
id: distributions
21-
attributes:
22-
label: Distributions
23-
description: "`all`: every distribution (that has the package). Select one or more."
24-
multiple: true
25-
options: [all, rolling, lyrical, kilted, jazzy, humble]
26-
default: 0
27-
validations:
28-
required: true
2920
- type: checkboxes
3021
id: operating_systems
3122
attributes:

.github/ISSUE_TEMPLATE/robostack-bot.yml renamed to .github/ISSUE_TEMPLATE/3_robostack_bot.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ body:
1111
1212
| command | who | what it does |
1313
|---|---|---|
14-
| `add-package` | anyone | Add ROS packages to the package selection of the chosen distributions that have them; the PR lists what will be built. A maintainer reviews and merges it. |
14+
| `add-package` | anyone | Add ROS packages to the shared package selection (built by every distribution that has them); the PR lists what will be built. A maintainer reviews and merges it. |
1515
| `update-rosdistro-snapshot` | maintainers | Refresh `rosdistro_snapshot.yaml` to the latest rosdistro sync (master for rolling) and bump the build number (highest released + 1, per-package build numbers removed) and mutex: a full rebuild. The PR lists the version bumps. Runs daily when a new sync is tagged. |
1616
| `find-stale-packages` | maintainers | List published packages built against pins that no longer match, with the build-number snippet to rebuild them. Replies only. |
1717
| `update-conda-forge-pinning` | maintainers | Move the shared conda-forge pinning to the latest version; re-renders `conda_build_config.yaml` and bumps the build number and mutex of the distributions whose pins changed. Runs weekly. |
@@ -34,7 +34,7 @@ body:
3434
label: Distributions
3535
description: "`all`: every distribution (for add-package: every one that has the package). Ignored by update-conda-forge-pinning."
3636
multiple: true
37-
options: [all, rolling, lyrical, kilted, jazzy, humble]
37+
options: [all, rolling, lyrical, jazzy, humble]
3838
default: 0
3939
validations:
4040
required: true

‎.github/workflows/automerge.yaml‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
name: Auto-merge
2+
3+
# Adding the "automerge" label to a pull request turns on GitHub's auto-merge: the
4+
# pull request is merged as soon as the required checks ("All green") pass.
5+
# Removing the label turns it off again. Pull requests without the label are never
6+
# merged automatically.
7+
#
8+
# pull_request_target so that the token can merge pull requests from forks too; the
9+
# pull request's code is never checked out or run here.
10+
11+
on:
12+
pull_request_target:
13+
types: [labeled, unlabeled]
14+
15+
permissions:
16+
contents: write
17+
pull-requests: write
18+
19+
jobs:
20+
automerge:
21+
if: github.event.label.name == 'automerge'
22+
runs-on: ubuntu-latest
23+
steps:
24+
# The merge has to come from the bot (or a PAT): a merge made with the
25+
# workflow's own token would not trigger the main build workflow.
26+
- name: Bot token
27+
id: app
28+
if: vars.ROBOSTACK_BOT_APP_ID != ''
29+
uses: actions/create-github-app-token@v2
30+
with:
31+
app-id: ${{ vars.ROBOSTACK_BOT_APP_ID }}
32+
private-key: ${{ secrets.ROBOSTACK_BOT_PRIVATE_KEY }}
33+
- name: Turn auto-merge on or off
34+
env:
35+
GH_TOKEN: ${{ steps.app.outputs.token || secrets.GHA_PAT || github.token }}
36+
PR: ${{ github.event.pull_request.html_url }}
37+
ACTION: ${{ github.event.action }}
38+
run: |
39+
if [ "$ACTION" = labeled ]; then
40+
gh pr merge "$PR" --auto --merge
41+
else
42+
gh pr merge "$PR" --disable-auto
43+
fi
Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -52,10 +52,14 @@ jobs:
5252
- uses: actions/checkout@v6
5353
with:
5454
persist-credentials: false
55-
- uses: prefix-dev/setup-pixi@v0.10.2
55+
# git auto-maintenance can leave a lock file in the git clones pixi caches
56+
# (e.g. vinca's), which breaks every later run restoring that cache
57+
- name: Disable git auto-maintenance
58+
run: git config --global maintenance.auto false && git config --global gc.auto 0 && git config --global core.longpaths true
59+
- uses: prefix-dev/setup-pixi@v0.11.0
5660
with:
5761
frozen: true
58-
pixi-version: v0.75.0
62+
pixi-version: v0.81.0
5963
- id: plan
6064
env:
6165
GITHUB_TOKEN: ${{ github.token }} # GitHub API: ros/rosdistro sync tags
@@ -102,10 +106,14 @@ jobs:
102106
- uses: actions/checkout@v6
103107
with:
104108
token: ${{ steps.app.outputs.token || secrets.GHA_PAT || github.token }}
105-
- uses: prefix-dev/setup-pixi@v0.10.2
109+
# git auto-maintenance can leave a lock file in the git clones pixi caches
110+
# (e.g. vinca's), which breaks every later run restoring that cache
111+
- name: Disable git auto-maintenance
112+
run: git config --global maintenance.auto false && git config --global gc.auto 0 && git config --global core.longpaths true
113+
- uses: prefix-dev/setup-pixi@v0.11.0
106114
with:
107115
frozen: true
108-
pixi-version: v0.75.0
116+
pixi-version: v0.81.0
109117
cache-write: false # the plan job saves the pixi environment cache
110118
- name: robostack-bot ${{ matrix.command }} ${{ matrix.distro }}
111119
id: bot
Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# After a merge: regenerate the staged build workflows of every distribution the
22
# merge touched, one build branch per distribution and platform
3-
# (buildbranch_<distro>_<platform>, workflows .github/workflows/build_<distro>_*.yml).
3+
# (buildbranch_<distro>_<platform>, workflows .github/workflows/build_<distro>_*.yaml).
44
# Pushing them starts the builds, which upload to each distribution's channel.
55
#
66
# Pushing workflow files needs the robostack-bot app (or secrets.GHA_PAT); without a
@@ -69,10 +69,14 @@ jobs:
6969
with:
7070
persist-credentials: false
7171
fetch-depth: 0
72-
- uses: prefix-dev/setup-pixi@v0.10.2
72+
# git auto-maintenance can leave a lock file in the git clones pixi caches
73+
# (e.g. vinca's), which breaks every later run restoring that cache
74+
- name: Disable git auto-maintenance
75+
run: git config --global maintenance.auto false && git config --global gc.auto 0 && git config --global core.longpaths true
76+
- uses: prefix-dev/setup-pixi@v0.11.0
7377
with:
7478
frozen: true
75-
pixi-version: v0.75.0
79+
pixi-version: v0.81.0
7680
# one job saves the shared pixi environment cache (same key for all)
7781
cache-write: ${{ matrix.distro == fromJSON(needs.plan.outputs.distros)[0] }}
7882
- name: Generate and push the build branches
@@ -94,14 +98,13 @@ jobs:
9498
retry() { for attempt in 1 2 3; do "$@" && return 0; echo "attempt ${attempt} failed: $*" >&2; sleep 15; done; return 1; }
9599
for platform in linux-64 osx-64 osx-arm64 win-64 linux-aarch64; do
96100
git reset --hard "$GITHUB_SHA" && git clean -fdx -e .pixi
97-
batch=10; [ "$platform" == "win-64" ] && batch=25 # https://github.com/RoboStack/robostack.github.io/issues/105
98101
retry pixi run -v rs "$DISTRO" generate-recipes --platform "$platform"
99-
retry pixi run -v rs "$DISTRO" gha --platform "$platform" --batch-size "$batch"
100-
if ! ls .github/workflows/build_"$DISTRO"_*.yml >/dev/null 2>&1; then
102+
retry pixi run -v rs "$DISTRO" gha --platform "$platform" --batch-size 25
103+
if ! ls .github/workflows/build_"$DISTRO"_*.yaml >/dev/null 2>&1; then
101104
echo "$DISTRO $platform: nothing to build"
102105
continue
103106
fi
104-
cp .github/workflows/build_"$DISTRO"_*.yml "$RUNNER_TEMP/workflows/"
107+
cp .github/workflows/build_"$DISTRO"_*.yaml "$RUNNER_TEMP/workflows/"
105108
if [ "$PUSH" == "true" ]; then
106109
branch="buildbranch_${DISTRO}_${platform//-/_}"
107110
git add -f . ':!.pixi'

0 commit comments

Comments
 (0)