Skip to content

Emscripten Updates from Jonathan Frank - #8558

Open
danoli3 wants to merge 8 commits into
openframeworks:masterfrom
danoli3:emscripten-combined
Open

danoli3 wants to merge 8 commits into
openframeworks:masterfrom
danoli3:emscripten-combined

Conversation

@danoli3

@danoli3 danoli3 commented Sep 11, 2026 •

Copy link
Copy Markdown
Member

Combined Emscripten PRs, cherry-picked with original author (Jonathhhan) and adapted for 6.0.6:

@danoli3 danoli3 changed the title Emscripten: combine 7211+7261+7553+7727 with 6.0.6 update Emscripten Updates Sep 11, 2026
@danoli3 danoli3 changed the title Emscripten Updates Emscripten Updates from Jonathan Frank Sep 11, 2026
@danoli3
danoli3 force-pushed the emscripten-combined branch from 0b26f61 to 64763cc Compare September 11, 2026 11:21
@danoli3

danoli3 commented Sep 11, 2026

Copy link
Copy Markdown
Member Author

Dropping WASM_WORKERS in light of -s AUDIO_WORKLET=1 - fixes MAIN_MODULE issues

@danoli3
danoli3 marked this pull request as draft September 12, 2026 04:32
Jonathhhan and others added 6 commits October 13, 2026 13:01
Cherry-picked from Jonathhhan/Emscripten-glInfoExample-fix
- Wrap GL limits and file handling in #ifndef TARGET_EMSCRIPTEN
- Original PR: openframeworks#7211
Cherry-picked from Jonathhhan/Emscripten_audioDeviceSelect_draft
- Add library_audioDeviceSelect.js for audio device enumeration
- Update library_html5audio.js
- Add --post-js for audioDeviceSelect in config.emscripten.default.mk
- Original PR: openframeworks#7261
Cherry-picked from Jonathhhan/Emscripten_recordVideo
- Add recordVideo.js and fix-webm-duration.js for canvas recording
- Update library_html5audio.js (MediaStreamAudioDestinationNode)
- Add --pre-js for recordVideo/fix-webm in config.emscripten.default.mk
- Original PR: openframeworks#7553
Cherry-picked from Jonathhhan/audioWorklet-3
- Update html5audio to audioWorklet (AudioContext 44100, MediaStreamAudioDestinationNode, emscriptenRegisterAudioObject)
- Rewrite ofxEmscriptenSoundStream to use emscripten_create_wasm_audio_worklet_node
- Update sound examples to 128 buffer, fix mutex to std::mutex
- Add USE_PTHREADS, AUDIO_WORKLET, WASM_WORKERS flags for 6.x (adapted for 6.0.6)
- Original PR: openframeworks#7727
WASM_WORKERS=1 + AUDIO_WORKLET=1 with MAIN_MODULE=1 fails on 6.0.6.
Use MAIN_MODULE=2 (dynamic) which allows WASM_WORKERS.
Keeps WASM_WORKERS perf for audioWorklet while allowing MAIN_MODULE.
The previous commit's MAIN_MODULE=2 does not help: emscripten rejects
WASM_WORKERS with any MAIN_MODULE ("dynamic linking is not supported").
Removing MAIN_MODULE exposed the problems it was hiding:

- config: drop MAIN_MODULE (EXPORT_ALL still exports every symbol for the
  JS libraries), link library_html5audio.js (the html5audio_* functions
  were undefined; MAIN_MODULE let that link), drop WEBAUDIO_DEBUG (its
  logging calls dbg(), which only exists with ASSERTIONS), and default
  EMBIND_AOT to 0 as the comment says: EMBIND_AOT fails with
  "expected to find pattern <<< EMBIND_AOT_INVOKERS >>>" when there are
  no EMSCRIPTEN_BINDINGS, and OF core has none.
- library_html5audio.js: html5audio_stream_create depends on $emAudio;
  emscripten 6.0.11's $emscriptenGetAudioObject reads emAudio without
  declaring it, so it was undefined.
- ofxEmscriptenSoundPlayer: re-enable the html5audio calls disabled in
  cd463ee. ofxEmscriptenAudioContext() compiled to
  "context = initialized = true", so the stream got audio context 1,
  which does not exist.
- recordVideo.js / fix-webm-duration.js: --pre-js code also runs in the
  audio worklet, which has no document or window. The undeclared
  drawTexture__deps assignment and window.ysFixWebmDuration threw there,
  so the worklet processor never started and there was no sound.
@danoli3
danoli3 force-pushed the emscripten-combined branch from 2ad406e to afa5f09 Compare October 13, 2026 00:55
@danoli3

danoli3 commented Oct 13, 2026

Copy link
Copy Markdown
Member Author

Rebased on master and fixed: Emscripten audio now builds and plays

Rebase: the template typo fix is dropped, because it's already on master (#8559). The glInfoExample conflict with #8579 is resolved by keeping both the TARGET_EMSCRIPTEN guard and master's MSVC freopen_s path. Jonathan's commits keep their author.

New commit afa5f09, fixing why this didn't build (CI build-emscripten was failing) and why it made no sound once built:

  1. MAIN_MODULE: emscripten rejects WASM_WORKERS with any MAIN_MODULE, so =2 didn't help. Removed it; EXPORT_ALL still exports every symbol for the JS libraries. MAIN_MODULE had been hiding the next three problems.
  2. library_html5audio.js wasn't linked, so the html5audio_* functions were undefined. It's now passed as --js-library. It also now depends on $emAudio: emscripten 6.0.11's $emscriptenGetAudioObject reads emAudio without declaring that dependency.
  3. EMBIND_AOT now defaults to 0, as the config comment already says. With no EMSCRIPTEN_BINDINGS (OF core has none) the link fails with expected to find pattern <<< EMBIND_AOT_INVOKERS >>>. Projects with their own bindings can still set it to 1.
  4. WEBAUDIO_DEBUG=1 removed: its logging calls dbg(), which doesn't exist in Release builds (ReferenceError: dbg is not defined).
  5. The audio context is back: ofxEmscriptenSoundPlayer's html5audio calls were disabled in cd463eeb6, so ofxEmscriptenAudioContext() compiled to context = initialized = true. The stream got audio context 1, which doesn't exist.
  6. The --pre-js files now work in the audio worklet. recordVideo.js (drawTexture__deps = …, document.createElement) and fix-webm-duration.js (window.…) threw there, so the worklet processor never started. Errors in the worklet don't reach the page console; this one only showed up by attaching to the worklet target.

Tested: emsdk 6.0.11, headless Chrome on macOS, served with COOP/COEP headers.

Example Build Runtime
audioOutputExample ✅ context running, worklet processor running, output peak 0.05 (volume 0.1), no errors
soundBufferExample ✅ output peak 0.40, no errors
audioInputExample ✅ input callback running (3512 buffers in 10 s, fake mic)
soundPlayerExample ✅ context running, no errors (playback not exercised headless)
glInfoExample ✅ no errors

Still to look at:

  • audioOutputExample plays its tone, but its waveform panes stay empty.
  • Video recording wasn't exercised.
  • Nothing was tested on a real browser with a microphone or speakers.

@danoli3

danoli3 commented Oct 13, 2026

Copy link
Copy Markdown
Member Author

The empty waveform panes in audioOutputExample aren't from this PR: open ofPolyline / ofBeginShape outlines don't draw on WebGL on master either. That's fixed separately in #8585, with before/after on WebGL, iOS and Android.

@danoli3
danoli3 marked this pull request as ready for review October 13, 2026 06:25
- recordVideo.js: add OF's audio track only when an audio context exists.
  Apps with no sound stream / player (e.g. graphicsExample) failed with
  "Cannot read properties of undefined (reading 'stream')", and the stop
  call then failed on the undefined MediaRecorder.
- recordVideo.js: stopping when not recording is a no-op, and the
  download is named after the recorded container (Video.webm on Chrome,
  it was always Video.mp4).
- library_html5audio.js: restore AUDIO.contextStream (a
  MediaStreamAudioDestinationNode fed by the analyser) from the record
  video change; the audioWorklet rewrite dropped it, so recordings had no
  audio.
@danoli3

danoli3 commented Oct 13, 2026

Copy link
Copy Markdown
Member Author

Record video tested, with fixes in f4e02d8

Recording is started and stopped from the app the same way as in #7553: EM_ASM(recordVideo($0), isRecording);. For this test, a temporary r key toggle in graphicsExample, driven from headless Chrome with emsdk 6.0.11.

graphicsExample, 5 s recorded in the browser (VP9 WebM, 1024×768, 30 fps; shown here as a GIF):

Original recordings: graphicsExample-recordVideo.webm (video only) and audioOutputExample-recordVideo-with-audio.webm (VP9 + 44.1 kHz stereo PCM, the example's tone at about −23 dB peak).

Fixed (f4e02d8):

  • Apps with no audio couldn't record. recordVideo always added AUDIO.contextStream's audio track, so graphicsExample failed with Cannot read properties of undefined (reading 'stream'), and the stop call then failed on the undefined MediaRecorder. The audio track is now added only when an audio context exists, and stopping when not recording does nothing.
  • Recordings had no audio. The audioWorklet rewrite of library_html5audio.js dropped AUDIO.contextStream from the record video change. It's restored: a MediaStreamAudioDestinationNode fed by the analyser.
  • Wrong file name. The download was always called Video.mp4, but Chrome records WebM. It's now named after the recorded type (Video.webm).

Notes:

  • recordVideo.js is a --pre-js, so at -O3 the function is only kept in the build when the app references it from EM_ASM, as in the snippet above. Calling it from the browser console won't work.
  • The ofTexture overload, recordVideo(1, textureID), wasn't tested.

@danoli3

danoli3 commented Oct 14, 2026

Copy link
Copy Markdown
Member Author

The 4 examples running (WebGL2, headless Chrome, emsdk 6.0.11)

Built with make from this PR merged with current master (f9e8d95, which includes #8585, so the audio waveforms draw). The mouse is moved in a circle over the canvas while recording.

Example Audio Errors
glInfoExample n/a 0
audioOutputExample context running, audioWorklet running, output peak 0.05 (volume 0.1) 0
audioInputExample audioWorklet input, 2354 buffers received 0
soundBufferExample output peak 0.47 0

The input example's microphone is a 330 Hz tone injected through getUserMedia, because headless Chrome's fake capture device is silent.

glInfoExample

audioOutputExample (mouse x pans, y sets the frequency)

audioInputExample

soundBufferExample

PROJECT_EMSCRIPTEN_WASM = 32 (default) or 64 in a project's config.make,
or make PROJECT_EMSCRIPTEN_WASM=64 (EMSCRIPTEN_MEMORY64 still selects 64).

- libs come from libs/*/lib/emscripten/WASM or WASM64 (ABI_PATH); the
  recursive lib/emscripten search linked both when both were installed
- wasm64 builds the core into lib/emscripten/WASM64 and objects into
  their own WASM64 folders, so 32 and 64 builds don't overwrite each other;
  wasm32 paths are unchanged
- -sMEMORY64=1 on compile, on the core's relocatable link (emcc -r) and
  on the app link; the old EMSCRIPTEN_MEMORY64 link flags were overwritten
  by the later PLATFORM_LDFLAGS assignment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants