Skip to content

Commit 63fb3d3

Browse files
committed
ci: move lance dependency bump flow into skill
1 parent 533e0ee commit 63fb3d3

6 files changed

Lines changed: 315 additions & 139 deletions

File tree

.agents/skills/README.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Agent Skills
2+
3+
This directory contains repo-scoped code agent skills for the lance-duckdb project.
4+
5+
Each skill is a folder that contains a required `SKILL.md` and optional bundled resources.
6+
7+
Codex discovers skills from `.agents/skills` in the current working directory and parent directories.
Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
---
2+
name: lance-duckdb-update-lance-dependency
3+
description: Update lance-duckdb to a specific Lance release or tag. Use when bumping Lance Rust dependencies in the lance-duckdb repository, including Cargo.toml, Cargo.lock, validation, branch creation, commit, push, and PR creation when requested.
4+
---
5+
6+
# lance-duckdb Update Lance Dependency
7+
8+
## Scope
9+
10+
Use this skill in the `lance-format/lance-duckdb` repository when updating Lance Rust dependencies to a specific Lance version or tag.
11+
12+
Inputs can be a version (`7.2.0-beta.1`), a tag (`v7.2.0-beta.1`), a tag ref (`refs/tags/v7.2.0-beta.1`), or `latest`.
13+
14+
## Workflow
15+
16+
1. Confirm the worktree status with `git status --short`.
17+
2. Resolve the target Lance version:
18+
19+
- If the input is `latest`, empty, or omitted, run:
20+
21+
```bash
22+
python3 ci/check_lance_release.py
23+
```
24+
25+
Parse the JSON output. If `needs_update` is not `true`, stop without creating a PR. Otherwise use `latest_tag`.
26+
27+
- If the input is explicit, use it directly.
28+
29+
3. Compute update metadata without changing files:
30+
31+
```bash
32+
python3 ci/update_lance_dependency.py "$TAG_OR_VERSION" --metadata-only
33+
```
34+
35+
Before making changes, check for an existing open PR with the emitted `pr_title`:
36+
37+
```bash
38+
gh pr list --search "\"$PR_TITLE\" in:title" --state open --limit 1 --json number,url,title
39+
```
40+
41+
If a matching open PR exists, stop and report it instead of creating a duplicate.
42+
43+
4. Run the deterministic update entrypoint:
44+
45+
```bash
46+
python3 ci/update_lance_dependency.py "$TAG_OR_VERSION"
47+
```
48+
49+
This updates the direct Lance Rust crate versions in `Cargo.toml`, removes repository-local Lance `[patch.crates-io]` entries if present, refreshes `Cargo.lock` with precise Cargo updates, and prints JSON metadata containing `branch_name`, `commit_message`, and `pr_title`.
50+
51+
5. Run validation:
52+
53+
```bash
54+
cargo fmt --all
55+
cargo check --manifest-path Cargo.toml
56+
cargo clippy --manifest-path Cargo.toml --all-targets
57+
```
58+
59+
If Cargo reports incompatible Arrow/DataFusion requirements, inspect the target Lance release requirements and update the pinned Arrow/DataFusion versions in `Cargo.toml`, then rerun `python3 ci/update_lance_dependency.py "$TAG_OR_VERSION"` and the validation commands. Fix real diagnostics and rerun validation until it succeeds.
60+
61+
6. Inspect `git status --short` and `git diff` to ensure only the Lance dependency update and required compatibility fixes are present.
62+
63+
7. If the task only asks to prepare local changes, stop here and report the changed files and validation result.
64+
65+
8. If the task asks to publish the update, create a branch using the printed `branch_name`, stage all relevant files, and commit using the printed `commit_message`. Do not amend or rewrite existing commits.
66+
67+
9. Push to `origin`. Before creating the PR, check that the current token has push permission:
68+
69+
```bash
70+
gh api repos/lance-format/lance-duckdb --jq .permissions.push
71+
```
72+
73+
If the remote branch already exists for the same generated branch name, delete the remote ref with `gh api -X DELETE repos/lance-format/lance-duckdb/git/refs/heads/$BRANCH_NAME`, then push. Do not force-push.
74+
75+
10. Create a PR targeting `main` with the printed `pr_title`. If there is no PR template, keep the body to two or three concise sentences: state the Lance dependency bump, note any required compatibility fixes, and link the triggering Lance tag or release.
76+
77+
11. Read back the remote PR title after creation. If it is not a Conventional Commit title, fix it immediately.
78+
79+
## GitHub Actions
80+
81+
This workflow is intentionally manual. When this skill is used from GitHub Actions, `TAG`, `GH_TOKEN`, and `GITHUB_TOKEN` may already be set. Resolve `latest` first when `TAG` is empty. Once an explicit tag or version is known, use:
82+
83+
```bash
84+
python3 ci/update_lance_dependency.py "$TAG" --github-output "$GITHUB_OUTPUT"
85+
```
86+
87+
Then use the emitted `branch_name`, `commit_message`, and `pr_title` values for branch, commit, and PR creation.

.github/workflows/codex-update-lance-dependency.yml

Lines changed: 13 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -13,17 +13,12 @@
1313
name: Codex Update Lance Dependency
1414

1515
on:
16-
workflow_call:
17-
inputs:
18-
tag:
19-
description: "Tag name from Lance (e.g., v2.0.0)"
20-
required: true
21-
type: string
2216
workflow_dispatch:
2317
inputs:
2418
tag:
25-
description: "Tag name from Lance (e.g., v2.0.0)"
26-
required: true
19+
description: "Tag name from Lance. Leave empty to use the latest Lance release that needs an update."
20+
required: false
21+
default: ""
2722
type: string
2823

2924
permissions:
@@ -37,7 +32,7 @@ jobs:
3732
steps:
3833
- name: Show inputs
3934
run: |
40-
echo "tag = ${{ inputs.tag }}"
35+
echo "tag = ${{ inputs.tag || 'latest' }}"
4136
4237
- name: Checkout Repo
4338
uses: actions/checkout@v5
@@ -63,18 +58,6 @@ jobs:
6358
with:
6459
components: rustfmt, clippy
6560

66-
- name: Validate stable release tag
67-
env:
68-
TAG: ${{ inputs.tag }}
69-
run: |
70-
set -euo pipefail
71-
VERSION="${TAG#refs/tags/}"
72-
VERSION="${VERSION#v}"
73-
if ! [[ "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
74-
echo "Only stable Lance versions are allowed (got: ${TAG})" >&2
75-
exit 1
76-
fi
77-
7861
- name: Configure git user
7962
run: |
8063
git config user.name "lance-community"
@@ -88,46 +71,21 @@ jobs:
8871
OPENAI_API_KEY: ${{ secrets.CODEX_TOKEN }}
8972
run: |
9073
set -euo pipefail
91-
VERSION="${TAG#refs/tags/}"
92-
VERSION="${VERSION#v}"
93-
BRANCH_NAME="codex/update-lance-${VERSION//[^a-zA-Z0-9]/-}"
74+
TARGET_TAG="${TAG:-latest}"
9475
9576
cat <<EOF >/tmp/codex-prompt.txt
9677
You are running inside the lance-duckdb repository on a GitHub Actions runner.
97-
Update the Lance Rust dependencies to version ${VERSION} (semver format) and prepare a pull request for maintainers to review.
9878
99-
Follow these steps exactly:
100-
1. Update versions in Cargo.toml:
101-
- In the [dependencies] section, update the versions for these crates to "${VERSION}":
102-
lance, lance-arrow, lance-core, lance-index, lance-linalg, lance-namespace, lance-namespace-impls, lance-table.
103-
- Keep existing features/flags unchanged.
104-
- Do not add or keep a [patch.crates-io] section for Lance crates; dependencies must come from crates.io stable releases only.
105-
2. Update Cargo.lock to match the new versions.
106-
- Prefer precise updates like:
107-
cargo update -p lance --precise ${VERSION}
108-
cargo update -p lance-arrow --precise ${VERSION}
109-
... (repeat for each of the crates above)
110-
- If Cargo complains about incompatible Arrow/DataFusion versions, update the pinned Arrow/DataFusion versions in Cargo.toml to match the Lance ${VERSION} dependency requirements, then update Cargo.lock again.
111-
3. Run these checks:
112-
- cargo fmt --all
113-
- cargo check --manifest-path Cargo.toml
114-
- cargo clippy --manifest-path Cargo.toml --all-targets
115-
4. Inspect "git status --short" and "git diff" to confirm the dependency update and any required fixes.
116-
5. Create and switch to a new branch named "${BRANCH_NAME}" (replace any duplicated hyphens if necessary).
117-
6. Stage all relevant files with "git add -A". Commit using the message "chore: update lance dependency to v${VERSION}".
118-
7. Push the branch to origin. If the remote branch already exists, delete it first with "gh api -X DELETE repos/lance-format/lance-duckdb/git/refs/heads/${BRANCH_NAME}" then push with "git push origin ${BRANCH_NAME}". Do NOT use "git push --force" or "git push -f".
119-
8. env "GH_TOKEN" is available; use "gh" tools for GitHub operations like creating pull requests.
120-
9. Create a pull request targeting "main" with title "chore: update lance dependency to v${VERSION}".
121-
- First, write the PR body to /tmp/pr-body.md using a heredoc (cat <<'EOF' > /tmp/pr-body.md).
122-
- The body should summarize the dependency bump, key resolver changes (if any), the commands run in step 3, and link the triggering tag (${TAG}).
123-
- Then run "gh pr create --body-file /tmp/pr-body.md".
124-
10. Display the PR URL, "git status --short", and a concise summary of the commands run and their results.
79+
Use \$lance-duckdb-update-lance-dependency with target "${TARGET_TAG}".
12580
12681
Constraints:
127-
- Use bash commands.
128-
- Do not merge the PR.
129-
- If any command fails, diagnose and fix the issue instead of aborting.
130-
- For compatibility issues, consult lance-format/lance at tag v${VERSION} for the expected Rust dependency versions.
82+
- Use env "GH_TOKEN" for GitHub operations.
83+
- Do not merge the pull request.
84+
- Do not force-push.
85+
- Do not create a duplicate pull request if an open PR already exists for the target Lance version.
86+
- If any command fails, diagnose and fix the root cause instead of aborting.
87+
- For compatibility issues, consult lance-format/lance at the target tag for the expected Rust dependency versions.
88+
- After creating the PR, display the PR URL, "git status --short", and a concise summary of the commands run and their results.
13189
EOF
13290
13391
printenv OPENAI_API_KEY | codex login --with-api-key

.github/workflows/lance-release-timer.yml

Lines changed: 0 additions & 80 deletions
This file was deleted.

README.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -174,12 +174,13 @@ See the SQL reference for full parameter documentation: [docs/sql.md#search](doc
174174

175175
Issues and PRs are welcome. High-impact areas include pushdown, parallelism/performance, type coverage, and better diagnostics.
176176

177-
### Automation: Lance dependency bumps
177+
### Manual Lance dependency bumps
178178

179-
This repository includes a GitHub Actions workflow pair that opens a PR when a newer tag is detected in `lance-format/lance`:
179+
This repository includes a manual GitHub Actions workflow for preparing Lance dependency bump PRs:
180180

181-
- `.github/workflows/lance-release-timer.yml`: polls for a newer Lance tag and triggers the Codex workflow.
182-
- `.github/workflows/codex-update-lance-dependency.yml`: uses Codex CLI to update `Cargo.toml` / `Cargo.lock`, run basic Rust checks, and open a PR.
181+
- `.github/workflows/codex-update-lance-dependency.yml`: manually runs Codex CLI with the repo-scoped `$lance-duckdb-update-lance-dependency` skill.
182+
- `.agents/skills/lance-duckdb-update-lance-dependency/SKILL.md`: defines the shared workflow for latest-release resolution, duplicate PR handling, dependency updates, validation, and PR creation.
183+
- `ci/update_lance_dependency.py`: provides the deterministic dependency update and metadata entrypoint used by the skill.
183184

184185
Required repository secrets:
185186

0 commit comments

Comments
 (0)