Skip to content

Conversation

@google-labs-jules
Copy link
Contributor

This change introduces a new test inventory document at docs/dev/testing/test_inventory.md. This document provides a comprehensive overview of the project's test suite, including a rating for each test based on its scope, value, speed, and dependencies.

Fixes #329


PR created automatically by Jules for task 4516107591645021813 started by @Coldaine

Adds a comprehensive inventory of all tests in the project, categorized by crate and type. Each test is rated on its scope, value, speed, and dependencies.

This document provides a valuable resource for understanding the project's test suite and identifying areas for improvement.
@google-labs-jules
Copy link
Contributor Author

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!


For security, I will only act on instructions from the user who triggered this task.

New to Jules? Learn more at jules.google/docs.

@qodo-code-review
Copy link

qodo-code-review bot commented Dec 25, 2025

CI Feedback 🧐

(Feedback updated until commit 2a1ca67)

A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

Action: Lint

Failed stage: Run clippy [❌]

Failure summary:

The action failed during the build process because the alsa-sys crate could not find the required
system library alsa. Specifically:
- The build script for alsa-sys v0.3.1 panicked at line 13 in
build.rs
- The pkg-config tool could not locate the alsa.pc file
- The PKG_CONFIG_PATH environment
variable is not set
- The system is missing the ALSA development library (libasound2-dev on
Ubuntu/Debian systems)

This is a missing system dependency issue, not a code issue. The ALSA library needs to be installed
on the runner before building.

Relevant error logs:
1:  ##[group]Runner Image Provisioner
2:  Hosted Compute Agent
...

165:  components: rustfmt, clippy
166:  ##[endgroup]
167:  ##[group]Run : set $CARGO_HOME
168:  �[36;1m: set $CARGO_HOME�[0m
169:  �[36;1mecho CARGO_HOME=${CARGO_HOME:-"$HOME/.cargo"} >> $GITHUB_ENV�[0m
170:  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
171:  env:
172:  CARGO_TERM_COLOR: always
173:  WHISPER_MODEL_SIZE: tiny
174:  MIN_FREE_DISK_GB: 10
175:  MAX_LOAD_AVERAGE: 5
176:  ##[endgroup]
177:  ##[group]Run : install rustup if needed
178:  �[36;1m: install rustup if needed�[0m
179:  �[36;1mif ! command -v rustup &>/dev/null; then�[0m
180:  �[36;1m  curl --proto '=https' --tlsv1.2 --retry 10 --retry-connrefused --location --silent --show-error --fail https://sh.rustup.rs | sh -s -- --default-toolchain none -y�[0m
181:  �[36;1m  echo "$CARGO_HOME/bin" >> $GITHUB_PATH�[0m
...

266:  �[36;1m    echo CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse >> $GITHUB_ENV�[0m
267:  �[36;1m  elif rustc +stable --version --verbose | grep -q '^release: 1\.6[67]\.'; then�[0m
268:  �[36;1m    touch "/home/runner/work/_temp"/.implicit_cargo_registries_crates_io_protocol || true�[0m
269:  �[36;1m    echo CARGO_REGISTRIES_CRATES_IO_PROTOCOL=git >> $GITHUB_ENV�[0m
270:  �[36;1m  fi�[0m
271:  �[36;1mfi�[0m
272:  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
273:  env:
274:  CARGO_TERM_COLOR: always
275:  WHISPER_MODEL_SIZE: tiny
276:  MIN_FREE_DISK_GB: 10
277:  MAX_LOAD_AVERAGE: 5
278:  CARGO_HOME: /home/runner/.cargo
279:  CARGO_INCREMENTAL: 0
280:  ##[endgroup]
281:  ##[group]Run : work around spurious network errors in curl 8.0
282:  �[36;1m: work around spurious network errors in curl 8.0�[0m
283:  �[36;1m# https://rust-lang.zulipchat.com/#narrow/stream/246057-t-cargo/topic/timeout.20investigation�[0m
...

367:  - /home/runner/work/ColdVox/ColdVox/crates/coldvox-vad-silero/Cargo.toml
368:  - /home/runner/work/ColdVox/ColdVox/crates/coldvox-vad/Cargo.toml
369:  ##[endgroup]
370:  ... Restoring cache ...
371:  No cache found.
372:  ##[group]Run cargo fmt --all -- --check
373:  �[36;1mcargo fmt --all -- --check�[0m
374:  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
375:  env:
376:  CARGO_TERM_COLOR: always
377:  WHISPER_MODEL_SIZE: tiny
378:  MIN_FREE_DISK_GB: 10
379:  MAX_LOAD_AVERAGE: 5
380:  CARGO_HOME: /home/runner/.cargo
381:  CARGO_INCREMENTAL: 0
382:  CACHE_ON_FAILURE: false
383:  ##[endgroup]
384:  ##[group]Run cargo clippy --all-targets --locked -- -D warnings
385:  �[36;1mcargo clippy --all-targets --locked -- -D warnings�[0m
386:  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
387:  env:
388:  CARGO_TERM_COLOR: always
389:  WHISPER_MODEL_SIZE: tiny
390:  MIN_FREE_DISK_GB: 10
391:  MAX_LOAD_AVERAGE: 5
392:  CARGO_HOME: /home/runner/.cargo
393:  CARGO_INCREMENTAL: 0
394:  CACHE_ON_FAILURE: false
395:  ##[endgroup]
...

439:  �[1m�[92m  Downloaded�[0m zerocopy-derive v0.8.31
440:  �[1m�[92m  Downloaded�[0m syn v2.0.111
441:  �[1m�[92m  Downloaded�[0m zvariant v5.8.0
442:  �[1m�[92m  Downloaded�[0m wayland-protocols v0.32.9
443:  �[1m�[92m  Downloaded�[0m tracing-subscriber v0.3.22
444:  �[1m�[92m  Downloaded�[0m tracing v0.1.44
445:  �[1m�[92m  Downloaded�[0m serde_json v1.0.146
446:  �[1m�[92m  Downloaded�[0m x11rb-protocol v0.13.2
447:  �[1m�[92m  Downloaded�[0m unicode-segmentation v1.12.0
448:  �[1m�[92m  Downloaded�[0m strength_reduce v0.2.4
449:  �[1m�[92m  Downloaded�[0m weezl v0.1.12
450:  �[1m�[92m  Downloaded�[0m wayland-sys v0.31.7
451:  �[1m�[92m  Downloaded�[0m ucd-trie v0.1.7
452:  �[1m�[92m  Downloaded�[0m typed-builder v0.20.1
453:  �[1m�[92m  Downloaded�[0m tokio-macros v2.6.0
454:  �[1m�[92m  Downloaded�[0m thiserror-impl v1.0.69
455:  �[1m�[92m  Downloaded�[0m tar v0.4.44
456:  �[1m�[92m  Downloaded�[0m strum_macros v0.26.4
457:  �[1m�[92m  Downloaded�[0m tokio v1.48.0
458:  �[1m�[92m  Downloaded�[0m strsim v0.11.1
459:  �[1m�[92m  Downloaded�[0m socks v0.3.4
460:  �[1m�[92m  Downloaded�[0m socket2 v0.6.1
461:  �[1m�[92m  Downloaded�[0m uuid v1.19.0
462:  �[1m�[92m  Downloaded�[0m typeid v1.0.3
463:  �[1m�[92m  Downloaded�[0m encoding_rs v0.8.35
464:  �[1m�[92m  Downloaded�[0m tiny-keccak v2.0.2
465:  �[1m�[92m  Downloaded�[0m time-macros v0.2.24
466:  �[1m�[92m  Downloaded�[0m time-core v0.1.6
467:  �[1m�[92m  Downloaded�[0m thread_local v1.1.9
468:  �[1m�[92m  Downloaded�[0m thiserror-impl v2.0.17
469:  �[1m�[92m  Downloaded�[0m thiserror v2.0.17
470:  �[1m�[92m  Downloaded�[0m thiserror v1.0.69
471:  �[1m�[92m  Downloaded�[0m termtree v0.5.1
...

506:  �[1m�[92m  Downloaded�[0m linux-raw-sys v0.4.15
507:  �[1m�[92m  Downloaded�[0m rand v0.9.2
508:  �[1m�[92m  Downloaded�[0m quick-xml v0.36.2
509:  �[1m�[92m  Downloaded�[0m nix v0.30.1
510:  �[1m�[92m  Downloaded�[0m rustc_version v0.4.1
511:  �[1m�[92m  Downloaded�[0m rtrb v0.3.2
512:  �[1m�[92m  Downloaded�[0m quote v1.0.42
513:  �[1m�[92m  Downloaded�[0m proc-macro2 v1.0.103
514:  �[1m�[92m  Downloaded�[0m openssl v0.10.75
515:  �[1m�[92m  Downloaded�[0m ndarray v0.16.1
516:  �[1m�[92m  Downloaded�[0m image v0.25.9
517:  �[1m�[92m  Downloaded�[0m tracing-appender v0.2.4
518:  �[1m�[92m  Downloaded�[0m toml v0.9.10+spec-1.1.0
519:  �[1m�[92m  Downloaded�[0m rust-ini v0.21.3
520:  �[1m�[92m  Downloaded�[0m rawpointer v0.2.1
521:  �[1m�[92m  Downloaded�[0m quick-error v2.0.1
522:  �[1m�[92m  Downloaded�[0m qt-build-utils v0.8.0
523:  �[1m�[92m  Downloaded�[0m voice_activity_detector v0.2.1
524:  �[1m�[92m  Downloaded�[0m rustls-pki-types v1.13.1
525:  �[1m�[92m  Downloaded�[0m regex v1.12.2
526:  �[1m�[92m  Downloaded�[0m realfft v3.5.0
527:  �[1m�[92m  Downloaded�[0m rand_chacha v0.9.0
528:  �[1m�[92m  Downloaded�[0m proc-macro-crate v3.4.0
529:  �[1m�[92m  Downloaded�[0m predicates-core v1.0.9
530:  �[1m�[92m  Downloaded�[0m quick-error v1.2.3
531:  �[1m�[92m  Downloaded�[0m ppv-lite86 v0.2.21
...

820:  �[1m�[92m   Compiling�[0m toml_datetime v0.7.5+spec-1.1.0
821:  �[1m�[92m    Checking�[0m allocator-api2 v0.2.21
822:  �[1m�[92m   Compiling�[0m crunchy v0.2.4
823:  �[1m�[92m    Checking�[0m foldhash v0.1.5
824:  �[1m�[92m   Compiling�[0m serde_json v1.0.146
825:  �[1m�[92m    Checking�[0m hashbrown v0.15.5
826:  �[1m�[92m   Compiling�[0m toml_edit v0.23.9
827:  �[1m�[92m   Compiling�[0m enumflags2_derive v0.7.12
828:  �[1m�[92m   Compiling�[0m tiny-keccak v2.0.2
829:  �[1m�[92m   Compiling�[0m proc-macro-crate v3.4.0
830:  �[1m�[92m    Checking�[0m mio v1.1.1
831:  �[1m�[92m   Compiling�[0m getrandom v0.3.4
832:  �[1m�[92m    Checking�[0m parking v2.2.1
833:  �[1m�[92m   Compiling�[0m zvariant_utils v3.2.1
834:  �[1m�[92m    Checking�[0m itoa v1.0.15
835:  �[1m�[92m   Compiling�[0m thiserror v2.0.17
836:  �[1m�[92m   Compiling�[0m async-trait v0.1.89
837:  �[1m�[92m   Compiling�[0m thiserror-impl v2.0.17
838:  �[1m�[92m    Checking�[0m concurrent-queue v2.5.0
839:  �[1m�[92m   Compiling�[0m zvariant_derive v5.8.0
840:  �[1m�[92m   Compiling�[0m getrandom v0.2.16
841:  �[1m�[92m    Checking�[0m fastrand v2.3.0
842:  �[1m�[92m   Compiling�[0m ucd-trie v0.1.7
843:  �[1m�[92m   Compiling�[0m typeid v1.0.3
844:  �[1m�[92m    Checking�[0m unicode-segmentation v1.12.0
845:  �[1m�[92m   Compiling�[0m pest v2.8.4
846:  �[1m�[92m   Compiling�[0m const-random-macro v0.1.16
847:  �[1m�[92m    Checking�[0m ryu v1.0.20
848:  �[1m�[92m   Compiling�[0m crc32fast v1.5.0
849:  �[1m�[92m    Checking�[0m const-random v0.1.18
850:  �[1m�[92m   Compiling�[0m alsa-sys v0.3.1
851:  �[1m�[92m   Compiling�[0m erased-serde v0.4.9
852:  �[1m�[92m   Compiling�[0m anyhow v1.0.100
853:  �[1m�[91merror�[0m: failed to run custom build command for `alsa-sys v0.3.1`
854:  Caused by:
...

872:  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
873:  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
874:  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-unknown-linux-gnu
875:  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_linux_gnu
876:  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
877:  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
878:  --- stderr
879:  thread 'main' (3558) panicked at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/alsa-sys-0.3.1/build.rs:13:18:
880:  pkg-config exited with status code 1
881:  > PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 pkg-config --libs --cflags alsa
882:  The system library `alsa` required by crate `alsa-sys` was not found.
883:  The file `alsa.pc` needs to be installed and the PKG_CONFIG_PATH environment variable must contain its parent directory.
884:  The PKG_CONFIG_PATH environment variable is not set.
885:  HINT: if you have installed the library, try setting PKG_CONFIG_PATH to the directory containing `alsa.pc`.
886:  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
887:  �[1m�[33mwarning�[0m: build failed, waiting for other jobs to finish...
888:  ##[error]Process completed with exit code 101.
889:  Post job cleanup.

This commit refactors the CI workflow to bifurcate jobs between GitHub-hosted and self-hosted runners, optimizing for performance and resource usage.

- Stateless jobs (lint, security, docs, unit tests, golden master) now run on GitHub-hosted runners.
- Hardware-dependent tests are now isolated to a dedicated self-hosted runner job.
- Caching for the Whisper model is implemented on the GitHub-hosted runner to improve performance.
- Full environment validation is preserved for the hardware tests to ensure reliability.

Additionally, this commit adds a comprehensive test inventory at `docs/dev/testing/test_inventory.md` to document the project's test suite.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement CI Bifurcation (Split Hosted vs Self-Hosted)

1 participant