Upgrading to 0.12 with scripts on a different domain (vite, python)

[incidentist]

I had a bad time upgrading to 0.12. I ended up consulting many issues in this repo, but am documenting all the missteps in one place. The trouble mostly stemmed from the fact that my app in dev mode fetches index.html from the python backend (localhost:8000) but loads all scripts via the Vite server (localhost:5173) for speed. This causes CORS problems. The other main problems I had are:

1. Workers often fail silently.
2. It is not clear what the difference is between ffmpeg-core.js, ffmpeg-core.worker.js and worker.js. It is easy to get these mixed up.

Here's what worked for me:

    // Most assets can be pulled from any origin, so let's use a CDN
    const baseURL = 'https://cdn.jsdelivr.net/npm/@ffmpeg/[email protected]/dist/esm'
    // The root worker needs to be served from the same origin as the page to enable SharedArrayBuffer
    const workerBaseUrl = window.location.origin + '/static/ffmpeg';
    const ffmpeg = new FFmpeg();
    ffmpeg.on('log', console.log);

    // Download most ffmpeg files to local blobs
    const [coreURL, wasmURL, workerURL] = await Promise.all([
        toBlobURL(`${baseURL}/ffmpeg-core.js`, 'text/javascript'),
        toBlobURL(`${baseURL}/ffmpeg-core.wasm`, 'application/wasm'),
        toBlobURL(`${baseURL}/ffmpeg-core.worker.js`, 'text/javascript'),
    ]);
    await ffmpeg.load({ coreURL, wasmURL, workerURL, classWorkerURL: `${workerBaseUrl}/worker.js` });

Then, you need to copy worker.js as well as a few files it requires, to wherever assets are served from on your backend:

cp node_modules/@ffmpeg/ffmpeg/dist/esm/worker.js api/assets/ffmpeg
cp node_modules/@ffmpeg/ffmpeg/dist/esm/const.js api/assets/ffmpeg
cp node_modules/@ffmpeg/ffmpeg/dist/esm/errors.js api/assets/ffmpeg

You can probably concat all these into one file somehow. Note that if the worker.js code changes, you'll have to do this step again to update it.

THEN, your call to ffmpeg needs to manually set the # of threads. If you don't do this, you may see ffmpeg initialize in your logs, but no actually video processing will happen.

const ffmpegParams = [
  // ...other parameters
  '-threads', '3',
  // ...more parameters
];

I hope this is helpful. Thank you to everyone who gave input on these bugs on the various scattered issues I looked at (too many to list here). I think the low-hanging fruit that will make this stuff easier in the future is: 1) clearer docs about the different args to load(), and 2) worker.js onmessage makes a lot of assumptions about the structure of the data it receives. If it receives something structured differently, it fails silently. It should be louder.

[incidentist]

I will also note that I did not need this in my vite config:

optimizeDeps: {
 exclude: ['@ffmpeg/ffmpeg', '@ffmpeg/util']
 },

I admit, I'm unsure exactly what this does, so maybe if I include it it makes the bundle smaller or something. But leaving it out doesn't seem to break anything.