Skip to content

feat(ci): bifurcate CI into hosted and self-hosted jobs #330

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Coldaine wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat(ci): bifurcate CI into hosted and self-hosted jobs #330

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7a0ca5e
feat(ci): bifurcate CI into hosted and self-hosted jobs
Dec 25, 2025
2cabb85
fix(ci): correct bifurcation - self-hosted for hardware tests only
Dec 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
221 changes: 72 additions & 149 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ jobs:
run: bash scripts/ci/setup-whisper-cache.sh

# Security scanning for vulnerabilities and license compliance
security_audit:
security:
name: Security Audit
runs-on: ubuntu-latest
steps:
Expand All @@ -123,153 +123,103 @@ jobs:
- name: Run cargo deny
run: cargo deny check

# Build, check, and test with multiple Rust versions
unit_tests_hosted:
name: Unit Tests & Golden Master (Hosted)
lint:
name: Lint (fmt + clippy)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v5.0.0
- name: Set up Rust toolchain
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
toolchain: stable
components: rustfmt, clippy
- name: Check formatting
run: cargo fmt --all -- --check
- name: Run clippy
run: cargo clippy --all-targets --locked

docs:
name: Documentation
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v5.0.0
- name: Set up Rust toolchain
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
toolchain: stable
- name: Build documentation
run: cargo doc --workspace --no-deps --locked

# Build and unit tests on GitHub-hosted (stateless, parallelizable)
build_and_unit_tests:
name: Build & Unit Tests (Hosted)
runs-on: ubuntu-latest
needs: [setup-whisper-dependencies]
strategy:
matrix:
rust-version: [stable] # Use stable only
steps:
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v5.0.0

- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y xdotool wget unzip gcc g++ make xvfb openbox dbus-x11 wl-clipboard xclip ydotool x11-utils wmctrl pkg-config pulseaudio libasound2-dev libgtk-3-dev libatspi2.0-dev libxtst-dev python3-pip python3-venv
sudo apt-get install -y pkg-config libasound2-dev libgtk-3-dev python3-pip python3-venv

- name: Set up Rust toolchain
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
toolchain: ${{ matrix.rust-version }}
components: rustfmt, clippy
override: true

# Only run formatting and linting on stable
- name: Check formatting (advisory)
if: matrix.rust-version == 'stable'
run: |
set +e
cargo fmt --all -- --check
status=$?
if [ "$status" -ne 0 ]; then
echo "::warning::cargo fmt detected formatting differences. Please run 'cargo fmt --all' locally before committing."
fi
exit 0

- name: Run clippy
if: matrix.rust-version == 'stable'
run: cargo clippy --all-targets --locked
toolchain: stable

- name: Type check
run: cargo check --workspace --all-targets --locked

- name: Build
run: cargo build --workspace --locked

# Only build docs and run tests on stable
- name: Build documentation
if: matrix.rust-version == 'stable'
run: cargo doc --workspace --no-deps --locked

- name: Run unit and integration tests (skip E2E)
if: matrix.rust-version == 'stable'
- name: Run unit tests
env:
WHISPER_MODEL_PATH: ${{ needs.setup-whisper-dependencies.outputs.model_path }}
WHISPER_MODEL_SIZE: ${{ needs.setup-whisper-dependencies.outputs.model_size }}
run: |
echo "=== Environment Validation ==="
echo "WHISPER_MODEL_PATH: $WHISPER_MODEL_PATH"
echo "WHISPER_MODEL_SIZE: $WHISPER_MODEL_SIZE"
echo "Model directory contents:"
ls -la "$WHISPER_MODEL_PATH" || echo "Model directory not accessible"
echo "=== Running Tests ==="
cargo test --workspace --locked --
echo "=== Running Workspace Unit Tests ==="
cargo test --workspace --locked

- name: Run Golden Master pipeline test
if: matrix.rust-version == 'stable'
env:
WHISPER_MODEL_PATH: ${{ needs.setup-whisper-dependencies.outputs.model_path }}
WHISPER_MODEL_SIZE: ${{ needs.setup-whisper-dependencies.outputs.model_size }}
run: |
echo "=== Running Golden Master Test ==="
# Install Python dependencies for Golden Master
pip install faster-whisper
export PYTHONPATH=$(python3 -c "import site; print(site.getsitepackages()[0])")
cargo test -p coldvox-app --test golden_master -- --nocapture

# Moonshine E2E skipped on GitHub-hosted: PyTorch+CUDA deps (4GB+) exceed disk space
# These tests run on self-hosted via moonshine_check job instead
- name: Skip Moonshine E2E Tests (runs on self-hosted)
if: matrix.rust-version == 'stable'
run: |
echo "::notice::Moonshine E2E tests skipped on GitHub-hosted (disk space). See moonshine_check job."

# GUI groundwork check integrated here
- name: Detect and test Qt 6 GUI
if: matrix.rust-version == 'stable'
run: |
# Qt6 might not be easily available on ubuntu-latest without extra actions, skipping for now or adding if needed
echo "Skipping Qt6 check on hosted runner"

- name: Upload test artifacts on failure
if: failure()
uses: actions/upload-artifact@v6
with:
name: test-artifacts-build-${{ matrix.rust-version }}
name: test-artifacts-build
path: |
target/debug/deps/
target/debug/build/
retention-days: 7

text_injection_tests:
name: Hardware Integration Tests (Self-Hosted)
# Hardware tests ONLY - requires live display (self-hosted laptop)
hardware_tests:
name: Hardware Tests (Self-Hosted)
runs-on: [self-hosted, Linux, X64, fedora, nobara]
needs: [setup-whisper-dependencies]
timeout-minutes: 30
timeout-minutes: 15
env:
# Runner's live KDE Wayland session with XWayland
DISPLAY: ":0"
WAYLAND_DISPLAY: "wayland-0"
RUST_LOG: debug
RUST_TEST_TIME_UNIT: 10000
RUST_TEST_TIME_INTEGRATION: 30000
SCCACHE_DIR: $HOME/.cache/sccache
# Build optimizations
CARGO_INCREMENTAL: "1"
RUSTFLAGS: "-C link-arg=-fuse-ld=mold"
steps:
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v5.0.0
- uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # v1
with:
toolchain: stable

# Setup sccache via justfile (installs if missing, enables RUSTC_WRAPPER)
- name: Setup sccache
run: |
# Install just if not available
if ! command -v just >/dev/null 2>&1; then
cargo install just --locked
fi

# Run setup-sccache recipe (idempotent - installs if missing)
just setup-sccache

# Enable sccache wrapper after installation
SCCACHE_BIN=""
if command -v sccache >/dev/null 2>&1; then
SCCACHE_BIN="$(command -v sccache)"
elif [[ -x "$HOME/.cargo/bin/sccache" ]]; then
SCCACHE_BIN="$HOME/.cargo/bin/sccache"
fi

if [[ -n "$SCCACHE_BIN" ]]; then
"$SCCACHE_BIN" --start-server || true
echo "RUSTC_WRAPPER=$SCCACHE_BIN" >> "$GITHUB_ENV"
echo "sccache enabled: $SCCACHE_BIN"
fi

- uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

- name: Setup ColdVox
Expand Down Expand Up @@ -305,85 +255,50 @@ jobs:
echo "::warning::X11 not accessible via xset"
fi

- name: Validate test prerequisites
- name: Validate hardware prerequisites
run: |
set -euo pipefail
echo "=== Test Environment Validation ==="
echo "=== Hardware Test Environment Validation ==="
echo "DISPLAY: $DISPLAY"
echo "XAUTHORITY: ${XAUTHORITY:-not set}"
echo "WAYLAND_DISPLAY: ${WAYLAND_DISPLAY:-not set}"

# Check if XAUTHORITY file exists
if [[ -f "${XAUTHORITY:-}" ]]; then
echo "XAUTHORITY file exists: $XAUTHORITY"
else
echo "::warning::XAUTHORITY file not found at ${XAUTHORITY:-unset}"
fi

# Verify X server is accessible
if ! xset -q >/dev/null 2>&1; then
echo "::error::Cannot connect to X server on display $DISPLAY"
echo "Trying xhost diagnostics..."
xhost 2>&1 || true
exit 1
fi
echo "X server is accessible."
echo "Available text injection backends:"
command -v xdotool >/dev/null && echo " - xdotool: $(xdotool --version 2>/dev/null || echo 'available')"
echo "Available backends:"
command -v xdotool >/dev/null && echo " - xdotool: available"
command -v ydotool >/dev/null && echo " - ydotool: available"
command -v enigo >/dev/null && echo " - enigo: available (Rust crate)"
echo "GTK development libraries:"
pkg-config --exists gtk+-3.0 && echo " - GTK+ 3.0: available" || echo " - GTK+ 3.0: not found"
echo "System audio:"
command -v alsa-info >/dev/null && echo " - ALSA: available" || echo " - ALSA: not found"
echo "=== Validation Complete ==="

- name: Test with real-injection-tests feature
- name: Run real injection tests
run: |
dbus-run-session -- bash -lc '
# Set per-test timeout to prevent hanging
export RUST_TEST_TIME_UNIT="10000" # 10 second timeout per test
export RUST_TEST_TIME_INTEGRATION="30000" # 30 second for integration tests
# Note: atspi feature not enabled - AT-SPI Collection.GetMatches requires
# a full desktop session, not available in headless Xvfb environment.
# AT-SPI tests will skip gracefully.
export RUST_TEST_TIME_UNIT="10000"
export RUST_TEST_TIME_INTEGRATION="30000"
cargo test -p coldvox-text-injection \
--features real-injection-tests \
-- --nocapture --test-threads=1
'

- name: Build pipeline (default features)
run: |
dbus-run-session -- bash -c '
set -euo pipefail
echo "Testing default features..."
cargo test -p coldvox-text-injection --locked

echo "Testing without default features..."
cargo test -p coldvox-text-injection --no-default-features --locked

echo "Testing regex feature only..."
cargo test -p coldvox-text-injection --no-default-features --features regex --locked
'

# Build main app to ensure integration compiles
- name: Build main application
run: cargo build --locked -p coldvox-app

- name: Run Hardware Capability Checks
env:
COLDVOX_E2E_REAL_INJECTION: "1"
COLDVOX_E2E_REAL_AUDIO: "1"
run: |
echo "=== Running Hardware Capability Checks ==="
# We run the new hardware_check test file, enabling the ignored tests
cargo test -p coldvox-app --test hardware_check -- --nocapture --include-ignored

- name: Upload test artifacts on failure
if: failure()
uses: actions/upload-artifact@v6
with:
name: test-artifacts-text-injection
- name: Cleanup
if: always()
run: |
if [[ -n "${DBUS_SESSION_BUS_PID:-}" ]]; then
kill "$DBUS_SESSION_BUS_PID" 2>/dev/null || true
fi
name: test-artifacts-build-and-test
path: |
target/debug/deps/
target/debug/build/
Expand Down Expand Up @@ -441,9 +356,11 @@ jobs:
needs:
- validate-workflows
- setup-whisper-dependencies
- security_audit
- unit_tests_hosted
- text_injection_tests
- lint
- security
- docs
- build_and_unit_tests
- hardware_tests
- moonshine_check
if: always()
steps:
Expand All @@ -453,15 +370,21 @@ jobs:
echo "## CI Report" > report.md
echo "- validate-workflows: ${{ needs.validate-workflows.result }}" >> report.md
echo "- setup-whisper-dependencies: ${{ needs.setup-whisper-dependencies.result }}" >> report.md
echo "- security_audit: ${{ needs.security_audit.result }}" >> report.md
echo "- unit_tests_hosted: ${{ needs.unit_tests_hosted.result }}" >> report.md
echo "- text_injection_tests: ${{ needs.text_injection_tests.result }}" >> report.md
echo "- lint: ${{ needs.lint.result }}" >> report.md
echo "- security: ${{ needs.security.result }}" >> report.md
echo "- docs: ${{ needs.docs.result }}" >> report.md
echo "- build_and_unit_tests: ${{ needs.build_and_unit_tests.result }}" >> report.md
echo "- hardware_tests: ${{ needs.hardware_tests.result }}" >> report.md
echo "- moonshine_check: ${{ needs.moonshine_check.result }} (optional)" >> report.md

Copy link

Copilot AI Dec 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing whitespace detected on this line. Remove the trailing spaces for cleaner code formatting.

Suggested change

Copilot uses AI. Check for mistakes.
if [[ "${{ needs.setup-whisper-dependencies.result }}" != "success" ]]; then echo "::error::Setup Whisper dependencies failed."; exit 1; fi
if [[ "${{ needs.security_audit.result }}" != "success" ]]; then echo "::warning::Security audit failed - check for vulnerabilities."; fi
if [[ "${{ needs.unit_tests_hosted.result }}" != "success" ]]; then echo "::error::Build and check failed."; exit 1; fi
if [[ "${{ needs.text_injection_tests.result }}" != "success" ]]; then echo "::error::Text injection tests failed."; exit 1; fi
if [[ "${{ needs.lint.result }}" != "success" ]]; then echo "::error::Lint checks failed."; exit 1; fi
if [[ "${{ needs.security.result }}" != "success" ]]; then echo "::warning::Security audit failed - check for vulnerabilities."; fi
if [[ "${{ needs.docs.result }}" != "success" ]]; then echo "::warning::Documentation build failed."; fi
if [[ "${{ needs.build_and_unit_tests.result }}" != "success" ]]; then echo "::error::Build and Unit Tests failed."; exit 1; fi
if [[ "${{ needs.hardware_tests.result }}" != "success" ]]; then echo "::error::Hardware tests failed."; exit 1; fi
if [[ "${{ needs.moonshine_check.result }}" != "success" ]]; then echo "::warning::Moonshine check failed (optional)."; fi

Copy link

Copilot AI Dec 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing whitespace detected on this line. Remove the trailing spaces for cleaner code formatting.

Copilot uses AI. Check for mistakes.
echo "All critical stages passed successfully."
- name: Upload CI Report
uses: actions/upload-artifact@v6
Expand Down
17 changes: 7 additions & 10 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ Default: `silero`, `text-injection`

## CI Environment

> **Principle**: The self hosted runner on the laptop is used for hardware tests, CI for everything else.
> **Principle**: The laptop only does what only the laptop can do (hardware tests).

See [CI Architecture](docs/dev/CI/architecture.md) for full details.

Expand All @@ -163,17 +163,17 @@ See [CI Architecture](docs/dev/CI/architecture.md) for full details.
|------|-------------|
| **Live KDE Plasma session** | NO Xvfb needed. Use real `$DISPLAY`. |
| **Fedora-based** | `apt-get` does NOT exist. Use `dnf`. |
| **Weak hardware** | Don't run builds or unit tests here. |
| **Always available** | Auto-login, survives reboots. |
| **Warm sccache** | Incremental builds ~2-3 min. |

### CI Split

| Task | Runner | Why |
|------|--------|-----|
| `cargo fmt`, `cargo clippy` | GitHub-hosted | Fast, parallel, free |
| `cargo audit`, `cargo deny` | GitHub-hosted | Security checks, no build needed |
| `cargo build` | **Self-hosted** | Warm cache, THE build |
| Hardware tests | **Self-hosted** | Requires display/audio/clipboard |
| `cargo build`, `cargo test` | GitHub-hosted | Powerful hardware, parallelizable |
| Hardware tests (display/audio/clipboard) | **Self-hosted** | Requires live desktop session |

### DON'T (Common AI Mistakes)

Expand All @@ -188,12 +188,9 @@ See [CI Architecture](docs/dev/CI/architecture.md) for full details.
env:
DISPLAY: ":99"

# WRONG - delays self-hosted by 5-10 min
hardware:
needs: [lint, build]

# WRONG - wasted work, can't share artifacts with Fedora
- run: cargo build # On ubuntu-latest
# WRONG - overloads weak laptop with work GitHub can do
- run: cargo build # On self-hosted
- run: cargo test --workspace # On self-hosted
```

## PR Checklist
Expand Down
Loading
Loading