Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions docs/todos/160-compiler-buffer-loop-strategy-config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
---
title: 'Plan: Configurable Buffer Generation Strategy (Loop vs Unrolled)'
priority: Medium
effort: 6-10 hours
created: 2025-11-06
status: Planned
completed: null
---

# Plan: Configurable Buffer Generation Strategy (Loop vs Unrolled)

## Goals
- Make buffer generation compile-time configurable between looped and unrolled implementations.
- Add compile-time `bufferSize` support with loop as the default strategy.
- Extract loop generation into a helper to keep `packages/compiler/src/index.ts` smaller.

## Current Behavior (Context)
- The compiler exports three wasm functions: `init`, `cycle`, and `buffer`.
- `buffer` is currently generated as a flat, unrolled sequence of `call cycle` repeated 128 times in `packages/compiler/src/index.ts` (inside the `createCodeSection` call that builds `buffer`).
- Runtime code calls `buffer()` once per audio buffer render and reads/writes audio buffers from wasm memory.
- There is no compile-time `bufferSize` or `bufferStrategy` option in `CompileOptions` yet.

## Proposed API
- Add to `CompileOptions`:
- `bufferSize?: number` (default 128)
- `bufferStrategy?: 'loop' | 'unrolled'` (default 'loop')

## Implementation Plan
1. Add `bufferSize` and `bufferStrategy` to `CompileOptions` in `packages/compiler/src/types.ts` with defaults applied in the compiler entrypoint.
2. Create `packages/compiler/src/wasmBuilders/` and add a helper (e.g., `createBufferFunctionBody.ts`) that returns the buffer function body byte array. It should accept `bufferSize` and `bufferStrategy` and emit either:
- Unrolled: `new Array(bufferSize).fill(call(1)).flat()` (existing behavior).
- Loop: a counted loop that calls `cycle` `bufferSize` times using a local i32 counter and `br_if`.
3. Update `packages/compiler/src/index.ts` to use the helper instead of inlining the buffer function body.
4. Update snapshots/tests that embed the `buffer` function bytecode or export list to reflect looped default.
5. Add tests to cover:
- Default behavior: `bufferStrategy = 'loop'` and `bufferSize = 128`.
- Explicit `bufferStrategy = 'unrolled'` with a custom `bufferSize`.
- A custom `bufferSize` with loop strategy.
6. Update docs that discuss buffer size and loop unrolling so the new default and options are clear.

## Open Questions
- Exact naming: `bufferStrategy` vs `bufferLoopMode` vs `bufferUnroll` (current leaning: `bufferStrategy`).
- Should there be validation or warnings for very large `bufferSize` values? (current direction: no cap)

## Implementation Details (Loop Strategy)
- Use wasm locals and control flow helpers already present in `packages/compiler/src/wasmUtils`:
- Locals: `createLocalDeclaration`, `localGet`, `localSet`
- Consts: `i32const`
- Control flow: `block`, `loop`, `br_if`, `Instruction.I32_SUB`
- Suggested loop shape (pseudocode):
- `local.set $i (i32.const bufferSize)`
- `block`
- `loop`
- `call $cycle`
- `local.set $i (local.get $i - 1)`
- `local.get $i`
- `br_if 0` (continue loop if counter != 0)
- `end`
- `end`
- Keep `buffer` signature unchanged (`[] -> []`).
- The `buffer` function index remains 2 (after `init` and `cycle`) in the export section.

## Files Likely to Change
- `packages/compiler/src/types.ts` (add compile-time options)
- `packages/compiler/src/index.ts` (use helper for buffer body)
- `packages/compiler/src/wasmBuilders/createBufferFunctionBody.ts` (new helper)
- `packages/compiler/tests/**` and snapshots referencing `buffer` body
- `docs/brainstorming_notes/005-audio-buffer-cycle-length-configuration.md` and `docs/todos/054-benchmark-unrolled-vs-normal-loop-audio-buffer-filler.md` (update defaults and new options)

## Risks
- Snapshot churn in compiler instruction tests.
- Performance regression risk vs unrolled implementation (mitigate with follow-up benchmarks).
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ title: 'TODO: Add Comment Code Blocks'
priority: Medium
effort: 1-2d
created: 2026-01-02
status: Open
completed: null
status: Completed
completed: 2026-01-03
---

# TODO: Add Comment Code Blocks
Expand Down
4 changes: 2 additions & 2 deletions packages/editor/packages/editor-state/src/effects/compiler.ts
Original file line number Diff line number Diff line change
Expand Up @@ -132,12 +132,12 @@ export default async function compiler(store: StateManager<State>, events: Event

if (state.initialProjectState?.compiledWasm && state.initialProjectState.compiledModules) {
try {
state.compiler.codeBuffer = decodeBase64ToUint8Array(state.initialProjectState.compiledWasm);
state.compiler.compiledModules = state.initialProjectState.compiledModules;
store.set('compiler.codeBuffer', decodeBase64ToUint8Array(state.initialProjectState.compiledWasm));
log(state, 'Pre-compiled WASM loaded and decoded successfully', 'Loader');
} catch (err) {
state.compiler.codeBuffer = new Uint8Array();
state.compiler.compiledModules = {};
store.set('compiler.codeBuffer', new Uint8Array());
Comment on lines 133 to +140
Copy link

Copilot AI Jan 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inconsistent state mutation approach: line 127 uses direct assignment (state.compiler.codeBuffer = new Uint8Array()) while lines 136 and 140 use store.set(). For consistency and to ensure state management works correctly (subscriptions, tracking, etc.), all state modifications should use store.set(). The same applies to lines 125-126, 128-131 which also use direct assignment.

Copilot uses AI. Check for mistakes.
console.error('[Loader] Failed to decode pre-compiled WASM:', err);
error(state, 'Failed to decode pre-compiled WASM', 'Loader');
}
Expand Down
21 changes: 3 additions & 18 deletions packages/editor/packages/editor-state/src/effects/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ export default function configEffect(store: StateManager<State>, events: EventDi
* Each config block is compiled independently to allow proper error mapping.
* Errors are saved to codeErrors.configErrors with the creationIndex of the source block.
*/
async function forceCompileConfig(): Promise<void> {
async function rebuildConfig(): Promise<void> {
const compileConfig = state.callbacks.compileConfig;
if (!compileConfig) {
return;
Expand All @@ -90,19 +90,9 @@ export default function configEffect(store: StateManager<State>, events: EventDi
}
}

function rebuildConfig() {
// Check if compilation is disabled by config
if (state.compiler.disableAutoCompilation) {
log(state, 'Config compilation skipped: disableAutoCompilation flag is set', 'Config');
return;
}

forceCompileConfig();
}

// Wire up event handlers
// rebuildConfig runs BEFORE module compilation because blockTypeUpdater runs first
events.on('compileConfig', forceCompileConfig);
events.on('compileConfig', rebuildConfig);
store.subscribe('initialProjectState', () => {
if (state.initialProjectState?.compiledConfig) {
applyConfigToState(store, state.initialProjectState.compiledConfig);
Expand All @@ -124,15 +114,10 @@ export default function configEffect(store: StateManager<State>, events: EventDi
* @returns Promise resolving to the merged config object
*/
export async function compileConfigForExport(state: State): Promise<ConfigObject> {
// If compilation is disabled, return the stored compiled config if available
if (state.compiler.disableAutoCompilation) {
return state.compiledConfig || {};
}

// If no compileConfig callback, return empty object
const compileConfig = state.callbacks.compileConfig;
if (!compileConfig) {
return {};
return state.compiledConfig || {};
}

// Collect all config blocks
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -114,22 +114,15 @@ describe('disableAutoCompilation feature', () => {
});

describe('Config compilation', () => {
it('should skip config compilation when disableAutoCompilation is true', async () => {
it('should compile config even when disableAutoCompilation is true', async () => {
store.set('compiler.disableAutoCompilation', true);

configEffect(store, mockEvents);

store.set('graphicHelper.codeBlocks', [...mockState.graphicHelper.codeBlocks]);
await new Promise(resolve => setTimeout(resolve, 0));

expect(mockCompileConfig).not.toHaveBeenCalled();
expect(
mockState.console.logs.some(
log =>
log.message.includes('Config compilation skipped: disableAutoCompilation flag is set') &&
log.category === '[Config]'
)
).toBe(true);
expect(mockCompileConfig).toHaveBeenCalled();
});

it('should compile config normally when disableAutoCompilation is false', async () => {
Expand All @@ -145,16 +138,16 @@ describe('disableAutoCompilation feature', () => {
});

describe('Runtime-ready export', () => {
it('should skip config compilation for export when disableAutoCompilation is true', async () => {
it('should compile config for export even when disableAutoCompilation is true', async () => {
mockState.compiler.disableAutoCompilation = true;

const result = await compileConfigForExport(mockState);

expect(mockCompileConfig).not.toHaveBeenCalled();
expect(result).toEqual({});
expect(mockCompileConfig).toHaveBeenCalled();
expect(result).toEqual({ memorySizeBytes: 1048576 });
});

it('should return stored compiledConfig when disableAutoCompilation is true and config exists', async () => {
it('should compile config for export even when compiledConfig exists', async () => {
mockState.compiler.disableAutoCompilation = true;
mockState.compiledConfig = {
memorySizeBytes: 2097152,
Expand All @@ -163,11 +156,8 @@ describe('disableAutoCompilation feature', () => {

const result = await compileConfigForExport(mockState);

expect(mockCompileConfig).not.toHaveBeenCalled();
expect(result).toEqual({
memorySizeBytes: 2097152,
selectedRuntime: 1,
});
expect(mockCompileConfig).toHaveBeenCalled();
expect(result).toEqual({ memorySizeBytes: 1048576 });
});

it('should compile config for export when disableAutoCompilation is false', async () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,11 +79,11 @@ describe('applyConfigToState', () => {
expect(state.compiler.disableAutoCompilation).toBe(true);
});

it('should not change disableAutoCompilation when not in config', () => {
it('should reset disableAutoCompilation to false when not in config', () => {
const state = createMockState();
const store = createStateManager(state);
state.compiler.disableAutoCompilation = true;
applyConfigToState(store, { memorySizeBytes: 65536 });
expect(state.compiler.disableAutoCompilation).toBe(true);
expect(state.compiler.disableAutoCompilation).toBe(false);
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,6 @@ export function applyConfigToState(store: StateManager<State>, config: ConfigObj
}
}

if (typeof config.disableAutoCompilation === 'boolean') {
store.set('compiler.disableAutoCompilation', config.disableAutoCompilation);
}

if (typeof config.memorySizeBytes === 'number') {
store.set('compiler.compilerOptions.memorySizeBytes', config.memorySizeBytes);
}
store.set('compiler.disableAutoCompilation', config.disableAutoCompilation ?? false);
store.set('compiler.compilerOptions.memorySizeBytes', config.memorySizeBytes ?? 1048576); // 1MB default
}
2 changes: 2 additions & 0 deletions packages/runtime-main-thread-logic/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ export default function createMainThreadLogicRuntime(

const intervalTime = Math.floor(1000 / sampleRate);

clearInterval(interval);
interval = setInterval(() => {
if (!wasmApp) return;
const startTime = performance.now();
Expand All @@ -44,6 +45,7 @@ export default function createMainThreadLogicRuntime(
timeToExecuteLoopMs = endTime - startTime;
}, intervalTime);

clearInterval(statsInterval);
statsInterval = setInterval(() => {
onStats({
timerPrecisionPercentage: 100 - Math.abs(timerDriftMs / intervalTime) * 100,
Expand Down
8 changes: 5 additions & 3 deletions packages/runtime-web-worker-logic/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ let timerDriftMs: number;

async function init(memoryRef: WebAssembly.Memory, sampleRate: number, codeBuffer: Uint8Array) {
try {
clearInterval(interval);
clearInterval(statsInterval);

const wasmApp = await createModule(memoryRef, codeBuffer);

const intervalTime = Math.floor(1000 / sampleRate);

lastIntervalTime = performance.now();

clearInterval(interval);

interval = setInterval(() => {
const startTime = performance.now();
timerDriftMs = startTime - lastIntervalTime - intervalTime;
Expand All @@ -25,6 +25,8 @@ async function init(memoryRef: WebAssembly.Memory, sampleRate: number, codeBuffe
timeToExecuteLoopMs = endTime - startTime;
}, intervalTime);

clearInterval(statsInterval);

statsInterval = setInterval(() => {
self.postMessage({
type: 'stats',
Expand Down
5 changes: 2 additions & 3 deletions packages/runtime-web-worker-midi/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,6 @@ async function init(
compiledModules: CompiledModuleLookup
) {
try {
clearInterval(interval);
clearInterval(statsInterval);

const wasmApp = await createModule(memoryRef, codeBuffer);
memoryBuffer = wasmApp.memoryBuffer;

Expand All @@ -39,6 +36,7 @@ async function init(
const intervalTime = Math.floor(1000 / sampleRate);

lastIntervalTime = performance.now();
clearInterval(interval);
interval = setInterval(() => {
const startTime = performance.now();
timerDriftMs = startTime - lastIntervalTime - intervalTime;
Expand All @@ -50,6 +48,7 @@ async function init(
broadcastMidiMessages(midiNoteModules, memoryBuffer);
}, intervalTime);

clearInterval(statsInterval);
statsInterval = setInterval(() => {
self.postMessage({
type: 'stats',
Expand Down
1 change: 1 addition & 0 deletions src/examples/projects/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ export const projectMetadata: ProjectMetadata[] = [
slug: 'simpleCounterMainThread',
title: 'Simple Counter (Main Thread)',
},
{ slug: 'standaloneProject', title: 'Standalone Project Example' },
];

// For backwards compatibility
Expand Down
3 changes: 1 addition & 2 deletions src/examples/projects/simpleCounterMainThread.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,7 @@ const project: Project = {
'set',
'popScope',
'',
'scope "runtimeSettings"',
'scope 0',
'scope "runtimeSettings[0]"',
'scope "runtime"',
'push "MainThreadLogicRuntime"',
'set',
Expand Down
Loading