Easier midi usage

[polhomarkho]

Hello Victor!
Here is a merge request to have an easier access to midi functions and music stuff more generally.
The code is ready (I just want to review it myself a last time tomorrow) but I'm working on adding a good example script in the PHONK-examples directory.
I'm not sure if the code follows the "rules" of PHONK as it makes midi things quite high level and accessible but it's really midi oriented so please tell me if it's not what you want in your app 😅
Here is what I added:

• a chord parser to play chord easily
• a note player to avoid issue when two identical notes are played
• a new way to create notes from the js using the note name instead of midi pitch
• new midi widgets to create easily midi actions (note button, control slider...)

[victordiaz]

Hi @polhomarkho,
I'd happy to add this to PHONK, I dont have any concern to add it if it makes your experience easier working with PHONK and maybe for someone else in the future.
In general, anything that improves or add extra functionality that does not break other things I'm happy to include it :)

I'm wondering if the specific UI that you are adding here could be done in a javascript file. So we could have a helpers folder that could be accessed from any project. The helper folder would contain utilities that simplify certain workflows.

So you could do something like

const midiUI = import('helpers/midiUI')
const slider1 = midiUI.addCcSlider()

Anyway thats is only an idea :)

Ping me when you have an example that I can try with the new functionally and I will review the pull request with more detail!

[polhomarkho]

Hi @victordiaz, sorry for the delay! It took me some time to have everything working with the repo cleanup 😅

Here's an example to play with:

/*	
 *  Description simple midi controller to control a soft synth on a computer 
 *  by Paul-Emile Sublet <[email protected]>
 */

// globals
const redColor = '#FF0000';

var buttonPositionX;
var buttonPositionY;
const buttonSize = 0.19;
const buttonSpacing = 0.01;

// start program
const midi = media.startMidiController();
const inputs = midi.findAvailableMidiInputs();

if (!inputs.length) {
  ui.toast('no midi input available :(');
  app.close();
} else if (inputs.length === 1) {
  const selectedDeviceInputId = inputs[0].deviceInputId;
  midi.setupMidi(selectedDeviceInputId);
  displayUi();
} else {
  // Select a midi input to send data to the computer if multiple midi inputs are found
  ui.popup()
    .title('Choose a midi input:')
    .choice(inputs.map(function (input) {
      return input.name;
    }))
    .onAction(function (selectedElement) {
      const selectedElementIndex = selectedElement.answerId;
      const selectedDeviceInputId = inputs[selectedElementIndex].deviceInputId;
      midi.setupMidi(selectedDeviceInputId);
      displayUi();
    })
    .show();
}

function displayUi() {
  buttonPositionX = 0;
  buttonPositionY = 0;
  
  // chord using midi pitches
  midi.ui.addNoteButton(buttonPositionX, buttonPositionY, buttonSize, buttonSize)
    .withText('C maj from pitches')
    .withNotePitches(60, 64)
    .addNotePitch(67);  // for the sake of the demo, could be in `withNotePitches()`
  movePositionForNextButton();
  
  // chord using note names
  midi.ui.addNoteButton(buttonPositionX, buttonPositionY, buttonSize, buttonSize)
    .withText('Dm7 from note names')
    .withNoteNames('D4', 'F4', 'A4')
    .addNoteName('C5');  // for the sake of the demo, could be in `withNoteNames()`
  movePositionForNextButton();
  
  // complex chord using all note parameters
  midi.ui.addNoteButton(buttonPositionX, buttonPositionY, buttonSize, buttonSize)
    .withText("E4 ch2, G4 ch2 velo 20, B4 ch1")
    .withNotes(
      {
        name: 'E4',
        channel: 1
      },
      {
        name: 'G4',
        channel: 1,
        velocity: 20
      })
    .addNote({name: 'B4'});  // for the sake of the demo, could be in `withNotes()`
  movePositionForNextButton();
  
  // 5th degree chord from a mixolydian mode of C major
  midi.ui.addNoteButton(buttonPositionX, buttonPositionY, buttonSize, buttonSize)
    .withDegreeAndMode({rootName: 'C4', degree: 'V', mode: 'mixolydian', chordSize: 4});
  movePositionForNextButton();

  // F sharp major seventh sharp five Chord :-D
  midi.ui.addNoteButton(buttonPositionX, buttonPositionY, buttonSize, buttonSize)
    .withChord(
      {
        name: 'F#maj7s5',
        octave: 4,
        velocity: 100,
        channel: 1  // not affected by pitch bend as it's channel 1
      });

  movePositionToNewlineForNextButton();
  
  // standard major chord progression (in F here)
  ['I', 'V', 'VI', 'IV'].forEach(function (chordDegree) {
    midi.ui.addNoteButton(buttonPositionX, buttonPositionY, buttonSize, buttonSize)
      .withDegreeAndMode({rootName: 'F3', degree: chordDegree, mode: 'major'})
      .withBackgroundColorWhenPlayed(redColor);
    movePositionForNextButton();
  });
  
  movePositionToNewlineForNextButton();
  
  // chord progression of the final part of "hey jude" from the Beetles: mixolydian mode F major scale
  ['VII', 'IV', 'I'].forEach(function (chordDegree) {
    midi.ui.addNoteButton(buttonPositionX, buttonPositionY, buttonSize, buttonSize)
      .withDegreeAndMode({rootName: 'F4', degree: chordDegree, mode: 'mixolydian'})
      .withBackgroundColorWhenPlayed(redColor);
    movePositionForNextButton();
  });

  movePositionToNewlineForNextButton();
  
  // "where is my mind" from the Pixies chord progression
  midi.ui.addNoteButton(buttonPositionX, buttonPositionY, buttonSize, buttonSize)
    .withChord({name: 'E', octave: 4});

  movePositionForNextButton();
  
  midi.ui.addNoteButton(buttonPositionX, buttonPositionY, buttonSize, buttonSize)
    .withChord({name: 'C#m', octave: 4});
    
  movePositionForNextButton();
  
  midi.ui.addNoteButton(buttonPositionX, buttonPositionY, buttonSize, buttonSize)
    .withChord({name: 'G#', octave: 4});
    
  movePositionForNextButton();
  
  midi.ui.addNoteButton(buttonPositionX, buttonPositionY, buttonSize, buttonSize)
    .withChord({name: 'A', octave: 4});

  movePositionToNewlineForNextButton();

  // Controls
  midi.ui.addModulationWheelSlider(buttonPositionX, buttonPositionY, buttonSize, buttonSize);
  movePositionForNextButton();
  
  midi.ui.addCcSlider(buttonPositionX, buttonPositionY, buttonSize, buttonSize, 74)
    .withChannel(3)
    .withText('filter');
  movePositionForNextButton();
    
  midi.ui.addCcSlider(buttonPositionX, buttonPositionY, buttonSize, buttonSize, 71)
    .withText('timbre');
  movePositionForNextButton();

  midi.ui.addPitchBendSlider(buttonPositionX, buttonPositionY, buttonSize, buttonSize);
  movePositionForNextButton();
  
  midi.ui.addPitchBendSlider(buttonPositionX, buttonPositionY, buttonSize, buttonSize)
    .withText('bend ch1')
    .withChannel(1);
  movePositionForNextButton();
}

function movePositionForNextButton() {
  // Buttons are displayed from left to right and go to newline when overflowing
  if (buttonPositionX + buttonSize + buttonSpacing >= 1) {
    buttonPositionX = 0;
    buttonPositionY += buttonSize + buttonSpacing;
  } else {
    buttonPositionX += buttonSize + buttonSpacing;    
  }
}

function movePositionToNewlineForNextButton() {
  buttonPositionX = 2;  // force overflow :D
  movePositionForNextButton();
}

It's quite verbose and many method calls are not required but they are here to show what you can do.
I agree with your idea of a helpers folder, it could be great! I don't know how hard it would be to implement though. I'll have a look!
Not really related, but the documentation I added is not displayed in the web editor. Any idea of what I should do?

[victordiaz]

@polhomarkho Thanks for this, I will review it soon.
Sorry if the "cleanup" I did in the repo made things a bit more difficult. Nevertheless I think it was good to re-arrange things :)

So I was thinking that this cool example that you made, could fit a new category called Demos.
In this category we could include more elaborate projects that go beyond explaining how the API works.

I'm thinking about this, because I have lots of projects that I've done in the past years that are getting lost in my hard drive.
It might be nice to include some of them, so maybe somebody could find them useful.
Having a Demos section will make a clear separation between what is a quick example that shows you how to use the API and a project that you could launch, use it and tweak it for your own needs.

I would love to know what you think about it!

[polhomarkho]

Hi @victordiaz, that's a good idea. The midi example in this PR shows mainly the midi ui usage and it starts to be a bit big and hard to understand so it would probably fit in a demo section.
Did you try the example? The funniest part is probably the ccslider because you can control many things like the filter opening but you need to map it in vital (it's really easy).
If you want to try it but don't know how to, I can do a quick video to show how to use all these midi things!

[polhomarkho]

Hi Victor!
I took some time to try to see if I can move the ui part in an helper script.
I first tried to create a new file library.js next to the main.js file, but I cannot import/require it :

if I either do require('library');, require('library.js');, require('./library');, require('./library.js'); in the main.js, I get an error.

I had some success adding the library.js file in the /storage/emulated/0/phonk_io/libraries folder and adding this code to allow require to work on the files in the libraries folder and it works (link to the solution I found) :

I can share the code in a branch if you want to see the working example.

About the first issue I had (local project library.js file), maybe we could add the current project path to the require paths so that we could require local project files. It could be really great to split big projects in multiple js files!

To come back to your suggestion, after reading again the classes in PHONK-android/phonk_apprunner/src/main/java/io/phonk/runner/apprunner/api/widgets/midi, I think they should stay as java files because they use many constants and methods from other "backend" classes (ChordBuilder, PMidiController constants...). We would need to export all these classes and constants to change them as javascript helper functions.
It's tricky because they are between ui and business concerns. What do you think about this?

Thanks!

[victordiaz]

Hi @polhomarkho,
Thanks for your comment. I did not have much time to have a deeper look to the code, I'm still trying to catch up with other things :D
So regarding loading other files, there is a app.load('file.js') method.
There is an example here https://github.com/victordiaz/PHONK/tree/develop/PHONK-examples/Advanced/LoadJsFile
Perhaps not the most intuitive place to put it :/ and the way it loads is just too simplistic.

Adding a require method could become handy also!
As you probably saw, the Mozilla Rhino is not up-to-date with modern JS, so things like import cannot be done without dirty hacks.

Regarding the UI methods that you added, I'm not superconvinced. I think it might be a bit confusing to have UI methods for very specific needs. In PHONK "I try" to keep things a bit simple. Having specific .js files within the project with custom UI would be a better approach to keep more canonical the rest of the API.

So for example if you want to reference the constants in those .js files, you could always move the constants inside the class midi, so they could be accessed like this in your .js file
addSlider(x, y, w, h).range(midi.MIN_CC_VALUE, midi.MAX_CC_VALUE)

I will try to play with this today though :)

[victordiaz]

hi @polhomarkho
I think I will leave this pull request out for the next version and have a better review later of this PR. I think is very valuable so I dont want to lose what you did :)

All the best!