MVSep_MDX23_Colab.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TkR0saXIcvyC"
      },
      "source": [
        "# MVSep MDX23 Colab v2.1.1\n",
        "* Slow separations fixed, unified installation cells (28.12.23).\n",
        "* Chunks currently set to 100000, work with 3:00 track. Decrease it if you run out of memory.\n",
        "* With current settings it takes 35 minutes to separate 3:00 track. For faster separation set both overlaps to 0.8, as it's still good performance/quality parameter (0.96 max tested, with 300K before memory issues).\n",
        "<br><br>\n",
        "* Don't write file names in the input field below - only folder paths allowed (otherwise it will show just /content/MVSEP-MDX23-Colab_v2.1 error).\n",
        "* Batch processing - all audio files inside provided input folder will be separated.<br>\n",
        "* Not all EOFError errors are critical. Check your output folder after separation first.<br>\n",
        "* [v2.2 final](https://colab.research.google.com/github/jarredou/MVSEP-MDX23-Colab_v2/blob/v2.2/MVSep-MDX23-Colab.ipynb) (with faster optional only 2 stem output) - better SDR, more vocal residues\n",
        "\n",
        "Colab version of MDX23 algorithm from [MVSEP.COM](https://www.mvsep.com) with some tweaks:\n",
        "* Updated with new UVR-MDX': voc_ft & Instr-HQ3 models\n",
        "* Fixed high frequency bleed in vocals\n",
        "* Fixed volume compensation for MDX models\n",
        "* It receives input files from temp Colab disk (open file manager in the left to upload a file), not from GDrive, but you can change it to GDrive by setting /content/drive/MyDrive/input (be aware that the path in some cases is case-sensitive and you must create that folder on your own)\n",
        "* For a bit better quality, chunks_size can be set to 500K for ~2:41-3:58 tracks - 500K fails with 5 minute tracks\n",
        "* Replace inference.py by [that](https://raw.githubusercontent.com/Infisrael/MVSEP-MDX23-Colab_v2.1/main/inference.py) one if you still have memory issues with 5 minute tracks and chunk_size set to 300K (open file manager and go to Colab folder) - be aware that it disables Demucs denoiser and increases vocal residues.\n",
        "* Beta 2.2 1.5.1 [inference](https://cdn.discordapp.com/attachments/887455924845944873/1129470812580225164/inference_1.5.1_vocft_mod.py) - might be a bit cleaner (denoiser still on).\n",
        "* 2.2 Pre Beta 3 /wo v3 MDX yet [Colab](https://colab.research.google.com/github/jarredou/MVSEP-MDX23-Colab_v2/blob/597b5b7f653e4593a0a94938a3923077d66f8767/MVSep-MDX23-Colab.ipynb) and [inference](https://drive.google.com/file/d/1bpZKZynmdsYcriF-M8t8yLtVLm7zRz5U/view?usp=sharing) - might be even cleaner (denoiser still on) - delete two bigshifts and vocals only references in the cell to make it work.\n",
        "* Beta 2.2 1.5.2 (shifts 0, less muddy) - [inference](https://drive.google.com/file/d/1GKTShwgVKwyLNLssh2AmLiN5vEwrUPus/view?usp=sharing).\n",
        "* For both overlaps set to 0.95 and 500K chunks it takes 27 minutes to process 3:58 track with shifts 0 set in inference from above.\n",
        "* It will show a lot of warinings and exceptions (e.g. PySoundFile failed, EOFError) frequently, but separation can be successful with these errors (and unsuccessful too, if you set wrong parameters)\n",
        "* Instrum is inverted vocals stem\n",
        "* Instrum2 is the sum of drums+bass+other stems (muddier, smaller SDR)\n",
        "* It doesn't use Demucs denoiser disabled yet, so it's less noisy here than current 2.1 - that fix in 2.1 was introduced after the 2.1 release in 2.0 repo (fixes problems when no matter what chunks you use, you still get memory errors in e.g. Demucs 3/4 step)\n",
        "\n",
        "\n",
        "Credits:\n",
        "* [https://github.com/ZFTurbo/MVSEP-MDX23-music-separation-model](https://github.com/ZFTurbo/MVSEP-MDX23-music-separation-model)\n",
        "* Models by [Demucs](https://github.com/facebookresearch/demucs), [UVR GUI Team](https://github.com/Anjok07/ultimatevocalremovergui) - Anjok / Aufr33 & [Kimberley Jensen](https://github.com/KimberleyJensen)\n",
        "* Adaptation & tweaks by [jarredou](https://github.com/jarredou/MVSEP-MDX23-Colab_v2/)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "uWX5WOqjU0QC"
      },
      "outputs": [],
      "source": [
        "#@markdown #Installation\n",
        "#@markdown *Run this cell to install MVSep-MDX23*\n",
        "print('Installing... This will take 1 minute...')\n",
        "%cd /content\n",
        "from google.colab import drive\n",
        "drive.mount('/content/drive')\n",
        "!git clone https://github.com/deton24/MVSEP-MDX23-Colab_v2.1 &> /dev/null\n",
        "%cd /content/MVSEP-MDX23-Colab_v2.1\n",
        "!pip install -r requirements.txt &> /dev/null\n",
        "!pip install demucs\n",
        "!pip install onnxruntime-gpu\n",
        "# onnxruntime-gpu nightly fix for cuda12.2\n",
        "!python -m pip install ort-nightly-gpu --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ort-cuda-12-nightly/pypi/simple/\n",
        "print('Installation done !')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "V7n1nXKsU4sd"
      },
      "outputs": [],
      "source": [
        "#@markdown #Separation\n",
        "from pathlib import Path\n",
        "import glob\n",
        "\n",
        "%cd /content/MVSEP-MDX23-Colab_v2.1\n",
        "def console(t):\n",
        "    get_ipython().system(t)\n",
        "\n",
        "#file_path = '/content/drive/MyDrive/mvsep_dataset/' #@param {type:\"string\"}\n",
        "folder_path = '/content/drive/MyDrive/input' #@param {type:\"string\"}\n",
        "output_folder = '/content/drive/MyDrive/output' #@param {type:\"string\"}\n",
        "overlap_large = 0.9 #@param {type:\"slider\", min:0.1, max:1, step:0.05}\n",
        "overlap_small = 0.9 #@param {type:\"slider\", min:0.1, max:1, step:0.05}\n",
        "\n",
        "#@markdown *Use lower chunk_size if you have memory errors*\n",
        "chunk_size = 100000 #@param {type:\"slider\", min:50000, max:1000000, step:100000}\n",
        "\n",
        "#console(f'python inference.py --large_gpu --chunk_size {chunk_size} --input_audio \"{file_path}\" --output_folder \"./results/{filename}\"')\n",
        "for file_path in glob.glob(folder_path+\"/*\"):\n",
        "  filename =  Path(file_path).stem\n",
        "  Path(output_folder,filename).mkdir(parents=True, exist_ok=True)\n",
        "  console(f'python inference.py --large_gpu --chunk_size {chunk_size} --input_audio \"{file_path}\" --overlap_large {overlap_large} --overlap_small {overlap_small} --output_folder \"{output_folder}\"/\"{filename}\"')\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title (optional) Mixdown to 64 bit (might be slightly better instrum2 equivalent (32-bit float) - can be placebo in most cases; (change it to pcm_s16le for 16 bit). Write file name without input file extension in paths below\n",
        "\n",
        "!ffmpeg -i \"/content/drive/MyDrive/output/your track/your track_bass.wav\" -i \"/content/drive/MyDrive/output/your track/your track_other.wav\" -i \"/content/drive/MyDrive/output/your track/your track_drums.wav\" -filter_complex \"[0]volume=3[a];[1]volume=3[b];[2]volume=3[c];[a][b][c]amix=inputs=3:duration=longest\" -c:a pcm_f64le '/content/drive/MyDrive/output/output.wav'"
      ],
      "metadata": {
        "id": "0Ds39boPN-x5"
      },
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": []
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}