Skip to content

Fix macOS input device selection without voice processing in AudioEngineDevice - #261

Draft
hiroshihorie wants to merge 2 commits into
m144_releasefrom
hiroshi/audioengine-mac-input-device
Draft

Fix macOS input device selection without voice processing in AudioEngineDevice#261
hiroshihorie wants to merge 2 commits into
m144_releasefrom
hiroshi/audioengine-mac-input-device

Conversation

@hiroshihorie

@hiroshihorie hiroshihorie commented Jul 3, 2026

Copy link
Copy Markdown
Member

Problem

On macOS with voice processing disabled, selecting a non-default recording device always fails at InitRecording. Without voice processing, AVAudioEngine backs the input and output nodes with a single shared HAL I/O unit, and a HAL unit talks to exactly one CoreAudio device (kAudioOutputUnitProperty_CurrentDevice is one global slot). Per-direction device selection is therefore unsatisfiable in that shape:

  • The existing per-direction configuration (input on element 1) fails with kAudioUnitErr_InvalidPropertyValue (-10851) once the graph is wired.
  • Setting the device earlier "succeeds" but re-routes both directions: pointing the shared unit at an input-only device collapses the output side to 0 channels.

The voice processing path is unaffected: VPIO is a full-duplex unit designed for per-direction device control (it manages an aggregate internally), which is why the Swift SDK (VP on by default) never hit this. The Rust SDK runs this engine with voice processing off by default on macOS, where "select a microphone" failed out of the box.

Fix

A new engine setup step configures the shared unit's device in the non-VP path, before the graph is wired (the HAL unit rejects device changes afterwards, and the node formats read during enable must reflect the target device):

  • Same effective input/output device: set it directly on the shared unit.
  • Different devices: create a private CoreAudio aggregate device combining them — the canonical mechanism for duplex I/O across distinct physical devices — and set that. The output device is the clock master and drift compensation is enabled on the input sub device, so the HAL re-times mic samples onto the render clock. The aggregate is process-private (invisible to other apps), destroyed on engine release/recreation, and rollback-safe.
  • Both directions on system default: unchanged, the engine keeps following the default route.

One readiness wait proved necessary in testing: the I/O unit renegotiates its node formats asynchronously after a device change, so the engine waits (bounded ~1s, rollback on timeout) until channel counts are valid before wiring the graph. HAL-level aggregate composition turned out to be synchronous in practice - the stream counts are logged at creation for error attribution.

New helpers CreatePrivateAggregateDevice / DestroyAggregateDevice live in mac_audio_utils. The existing per-direction configuration is kept for the voice processing path, now explicitly gated to it.

Testing

Verified on macOS arm64 via the LiveKit Rust SDK's platform_audio exerciser against a local build:

  • The previously failing sequence (start recording with VP off, then switch to a non-default microphone) now works: aggregate created, formats renegotiated, recording continues on the new device.
  • Voice processing path regression-checked: device switching with VP on still works.
  • Full exerciser passes: lifecycle, device enumeration/selection/hot-swap, recording, AEC/AGC/NS reconfiguration, 16-thread concurrent churn (with repeated engine recreation and aggregate create/destroy cycles), and full teardown/reacquire cycles — zero errors.

Not yet verified: audio quality under long-session clock drift through the aggregate (drift compensation is enabled, but deserves a listen test), and the default-output-change-while-aggregated scenario (should recreate via the existing default device listeners).

…ineDevice

Without voice processing, AVAudioEngine's input and output nodes share a
single HAL I/O unit, so the previous per-direction device configuration
could not work: setting the input device on the shared unit failed with
kAudioUnitErr_InvalidPropertyValue once the graph was wired, and any
earlier set re-routed both directions, collapsing output formats when
the input device has no output streams. In practice selecting a
non-default recording device always failed InitRecording in this mode.

The shared unit is now configured in a dedicated step that runs before
the graph is wired, and when the effective input and output devices
differ, a private aggregate device combining them is created (output as
clock master, drift compensation on the input sub device) and set as the
unit's device. The aggregate is destroyed on engine release and
recreation. The existing per-direction configuration is kept for the
voice processing path, where the separate I/O units accept it.
The HAL-level stream readiness poll after aggregate creation was
subsumed by the I/O unit format wait in the engine, which is where
readiness actually matters for consumers. Replace the loop with a
one-shot stream count log for error attribution.

Also switch the deprecated Master sub device key spelling to Main,
which maps to the same underlying dictionary key.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant