ci: unified docs orchestrator workflow and metadata sync script#26285
Open
ci: unified docs orchestrator workflow and metadata sync script#26285
Conversation
mrpollo
force-pushed
the
mrpollo/docs_metadata
branch
from
0f7baa9 to
f58e205
Compare
Link Check ResultsNo broken links found! |
mrpollo
force-pushed
the
mrpollo/docs_metadata
branch
3 times, most recently
from
6ec679a to
1116bb8
Compare
|
No flaws found |
hamishwillee
previously approved these changes
Jan 29, 2026
Contributor
There was a problem hiding this comment.
@mrpollo The approach looks good, however I am unable to test it effectively locally. I think it would be great to do this suggestion #26285 (comment) and then merge this and iterate.
Approving to make that possible.
context: our docs need this metadata, this script generates the metadata then
moves the files to the required destination.
docs:
update_uorb_graphs.sh — generate, compare, and sync uORB graph JSONs
Usage:
./scripts/update_uorb_graphs.sh [--test-only] [--debug]
Options:
--test-only Run generation and comparison only; exit 1 if diffs found, without copying files
--debug Echo debug info for missing or differing files
Examples:
# CI mode: fail if docs need updates
./scripts/update_uorb_graphs.sh --test-only
# Developer mode: regenerate and sync JSONs
./scripts/update_uorb_graphs.sh
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
check if metadata needs to be rebuilt! Signed-off-by: Ramon Roche <mrpollo@gmail.com>
more debug tools Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
metadata_msg_docs.sh — generate and sync uORB message reference documentation Usage: Tools/ci/metadata_msg_docs.sh [--test-only] [--debug] Options: --test-only Run make target and comparison; exit 1 if diffs found, without copying files --debug Show full make output and debug info for file comparisons Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
metadata_parameters.sh — generate and sync PX4 parameter reference documentation Usage: Tools/ci/metadata_parameters.sh [--test-only] [--debug] Options: --test-only Run make target and comparison; exit 1 if diffs found, without copying file --debug Show full make output and debug info for comparison Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
metadata_airframe.sh — generate and sync PX4 airframe reference documentation Usage: Tools/ci/metadata_airframe.sh [--test-only] [--debug] Options: --test-only Run make target and comparison; exit 1 if diffs found, without copying file --debug Show full make output and debug info for comparison Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
metadata_modules.sh — generate and sync PX4 module reference documentation Usage: Tools/ci/metadata_modules.sh [--test-only] [--debug] Options: --test-only Run make target and comparison; exit 1 if diffs found, without copying files --debug Show full make output and debug info for file comparisons Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
test is broken Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Add conditional metadata generation for PRs that modify both docs and source files. This fixes broken link checker reports when a PR adds a new module and documents it simultaneously - previously the link checker would fail because metadata files are only generated on push to main. Changes: - Add docs-orchestrator.yml workflow with: - detect-changes job to check if PR touches source paths - pr-metadata-regen job that generates metadata and uploads as artifact - link-check job that downloads metadata artifact when available - build-site job for VitePress site generation - deploy-aws and crowdin-upload jobs for push events - Prettier formatting step before auto-commit on push - Add Prettier for markdown formatting: - Add prettier ^3.2.0 as devDependency in docs/package.json - Add docs/.prettierrc with prose-preserving config - Remove normalize_whitespace from metadata_sync.sh (Prettier handles whitespace normalization now) Signed-off-by: Ramon Roche <mrpollo@gmail.com>
mrpollo
force-pushed
the
mrpollo/docs_metadata
branch
from
9566995 to
ef1f2f4
Compare
Contributor
Author
|
@hamishwillee I've addressed all your comments:
Let me know if you're happy to merge so we can start enjoying the benefits! 🚀 |
hamishwillee
reviewed
Feb 4, 2026
| run: | | ||
| cd docs | ||
| yarn install --frozen-lockfile | ||
| yarn prettier --write "en/**/*.md" |
Contributor
There was a problem hiding this comment.
I think this will prettier absolutely everything, not just modified files. This will affect efficiency, but it not a bad thing.
hamishwillee
previously approved these changes
Feb 4, 2026
Contributor
There was a problem hiding this comment.
Thanks @mrpollo - looks good. The only "issue" I can see is that prettier runs on everything, and had not been run on this PR. So decided to do it so all changes in other PRs going forwards are deltas.
We probably should run this in v1.17 too, because it's metadata is likely to be out of date.
But, there is a depencency error https://github.com/PX4/PX4-Autopilot/actions/runs/21689629726/job/62545775038?pr=26285
error Your lockfile needs to be updated, but yarn was run with `--frozen-lockfile`.
I think we can fix this by removing the json package lock file - trying that now
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Consolidates 7 docs-related workflows into a single orchestrator and replaces 6 metadata scripts with a unified
metadata_sync.sh.Architecture
PR Flow:
Push Flow (main/release branches):
Changes
New unified metadata script (
Tools/ci/metadata_sync.sh):--generate,--sync,--verboseNew docs orchestrator workflow (
.github/workflows/docs-orchestrator.yml):New Makefile target:
msg_docs- generates uORB message documentationNew documentation:
docs/en/test_and_ci/continous_integration.md- Documents the Docs CI pipelineDeleted workflows (replaced by orchestrator):
docs_deploy.yml,docs_deploy_aws.yml,docs_flaw_checker.ymldocs_pr_comment.yml,docs_crowdin_upload.yml,docs_metadata_check.ymlfailsafe_sim.ymlDeleted scripts (replaced by metadata_sync.sh):
metadata_parameters.sh,metadata_airframe.sh,metadata_modules.shmetadata_msg_docs.sh,metadata_uorb_graph.sh,metadata_failsafe_web.shKept:
docs_crowdin_download.yml(separate scheduled trigger for translations)Benefits
Test plan