End to end test action using Maestro

[TimoPtr]

Summary

First e2e test steps using the Android app.

A valid run https://github.com/home-assistant/android/actions/runs/30528453182/job/90824810474?pr=6917 takes 3mn of emulator time. The consumption is then *9 the numbers of emulator we have.

We are testing from API 29 to 37, bellow it doesn't work because of some limitation on the webview installed on the emulator.

The new workflow does run every morning on main using the dev docker image of Home-Assistant. The goal is to catch any regression while onboarding and interacting with the frontend. It's a direct test of the communication using the external bus by opening the native settings screen.

We are using
https://github.com/ludeeus/setup-homeassistant
https://emulator.wtf
https://maestro.dev/

Checklist

• New or updated tests have been added to cover the changes following the testing guidelines.
• The code follows the project's code style and best_practices.
• The changes have been thoroughly tested, and edge cases have been considered.
• Changes are backward compatible whenever feasible. Any breaking changes are documented in the changelog for users and/or in the code for developers depending on the relevance.

Link to pull request in documentation repositories

Developer Documentation: home-assistant/developers.home-assistant#

Any other notes

Part of #7036

[github-actions]

Test Results

0 tests   0 ✅  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit 5ebb4d2.

♻️ This comment has been updated with latest results.

[Copilot]

Pull request overview

Introduces an initial end-to-end (E2E) onboarding flow using Maestro, backed by a scheduled GitHub Actions workflow that boots a Home Assistant container and runs the flow across multiple Android API levels on emulator.wtf. It also hardens Improv BLE provisioning/scanning to avoid crashes on devices/emulators without a Bluetooth adapter.

Changes:

• Add a scheduled/manual GitHub Actions workflow to build the app, start Home Assistant, provision emulators, and run a Maestro onboarding flow across API levels.
• Add a Maestro flow script that onboards, logs in, opens the frontend settings, and navigates into native Companion App settings.
• Make Improv Bluetooth usage nullable (inert when no adapter) and extend unit tests to cover the no-adapter behavior.

Reviewed changes

Copilot reviewed 28 out of 29 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
app/src/test/kotlin/io/homeassistant/companion/android/frontend/improv/ImprovRepositoryImplTest.kt	Adds tests ensuring Improv scan/provision flows remain inert when no Bluetooth adapter is present.
app/src/main/kotlin/io/homeassistant/companion/android/frontend/improv/ImprovRepositoryImpl.kt	Makes ImprovManager nullable; scanning/provisioning become no-ops when Bluetooth is unavailable.
app/src/main/kotlin/io/homeassistant/companion/android/frontend/improv/ImprovManagerFactory.kt	Updates factory contract to allow returning null when Bluetooth adapter is missing.
app/src/main/kotlin/io/homeassistant/companion/android/frontend/improv/FrontendImprovModule.kt	Hilt provider now guards ImprovManager creation behind Bluetooth adapter availability.
.maestro/onboarding.yaml	Defines the Maestro onboarding + settings-navigation E2E flow.
.github/workflows/e2e.yml	Adds the E2E workflow that builds, starts HA, provisions emulators, and runs Maestro.
.github/e2e/homeassistant/configuration.yaml	Home Assistant configuration for the CI E2E container.
.github/e2e/homeassistant/.storage/repairs.issue_registry	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/person	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/onboarding	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/lovelace.map	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/lovelace_dashboards	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/http.auth	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/http	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/homeassistant.exposed_entities	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/frontend.system_data	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/core.uuid	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/core.restore_state	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/core.entity_registry	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/core.device_registry	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/core.config_entries	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/core.config	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/core.area_registry	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/core.analytics	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/bluetooth.passive_update_processor	Seeds HA storage state for E2E instance.
.github/e2e/homeassistant/.storage/auth_provider.homeassistant	Seeds HA auth provider state for E2E instance.
.github/e2e/homeassistant/.storage/auth	Seeds HA auth state for E2E instance (includes token material).
.github/e2e/homeassistant/.storage/assist_pipeline.pipelines	Seeds HA assist pipeline state for E2E instance.
.github/e2e/homeassistant/.gitignore	Ignores volatile HA runtime files within the E2E config directory.

Comments suppressed due to low confidence (2)

app/src/main/kotlin/io/homeassistant/companion/android/frontend/improv/ImprovRepositoryImpl.kt:219

• stopScanInternal() catches Exception, which also catches CancellationException. Even though this runs under NonCancellable, cancellation can still be thrown from inside called code and should propagate.

        try {
            manager?.stopScan()
        } catch (e: Exception) {
            Timber.w(e, "Cannot stop scanning")
        }

.github/workflows/e2e.yml:36

• The workflow environment variable fallback still hard-codes dev (inputs.home-assistant-version || 'dev'), which contradicts the intended default of testing the stable HA image. Align the fallback with the workflow_dispatch default so scheduled/PR runs also use stable unless overridden.

      E2E_FORWARD_IP: 172.16.0.2
      E2E_HOME_ASSISTANT_VERSION: ${{ inputs.home-assistant-version || 'dev' }}
      # The hostname must not start with something resembling a dictionary