Branching and looping for p5.strands

[davepagurek]

Resolves #7868

Changes

• Finished implementation of if/else GLSL generation
• Added transpilation of JS branching into GLSL branching
• Added tests for ifs and nested ifs
• Added tests for swizzle assignments in blocks
• Looping support
• Transpilation of JS loops into strands loops
• looping tests
• Bug fix: handle self-update statements like i++ again
• Bug fix: handle >= operator properly
• Added a contributor doc explaining the key parts of the p5.strands architecture

I refactored a bit in order to handle nested blocks more easily: there's now a SCOPE_START and SCOPE_END block for { and } respectively, and I added an ASSIGNMENT statement that is currently only used to assign to phi variables in each branch.

Details

The main challenge here is that we have to fully replace control flow (if/for) in users' code into function calls so that these structures can generate nodes in our program graph. If we don't, they run in javascript, and are invisible to GLSL generation. For example, if you had a loop that runs 10 times that adds 1 each time, it would output the add 1 line 10 times rather than outputting a for loop.

However, once we have a function call instead of real control flow, we also need a way to make sure that when the users' javascript subsequently references nodes that were updated in the control flow, they properly reference the modified value after the if or for and not the original value. For that, we make the function calls return updated values, and we generate JS code that assigns these updated values back to the original JS variables.

If statements

You write an if statement like this:

const testShader = baseMaterialShader().modify(() => {
  const condition = uniformFloat(() => 1.0);
  getPixelInputs(inputs => {
    let color = 0.5; // initial gray
    if (condition > 0.5) {
      color = 1.0; // set to white in if branch
    }
    inputs.color = [color, color, color, 1.0];
    return inputs;
  });
});

It gets transpiled into this function call. Each branch callback gets a copy of the node so that it can modify it without affecting the original, and each branch callback then returns an object with modified versions of those variables. The whole if/else structure returns an object with nodes representing the output of the branch, and we then assign back values from that to the original variables.

() => {
  const condition = uniformFloat('condition', () => 1);
  getPixelInputs(inputs => {
    let color = float(0.5);
    {
      color = __p5.strandsNode(color);
      const __block_2 = __p5.strandsIf(
        // Condition
        __p5.strandsNode(condition).greaterThan(0.5),
        // If branch
        () => {
          let __copy_color_0 = color.copy();
          __copy_color_0 = myp5.float(1);
          return { color: __copy_color_0 };
        }
      // Else branch
      ).Else(() => {
        let __copy_color_1 = color.copy();
        return { color: __copy_color_1 };
      });
      color = __block_2.color;
    }
    inputs.color = __p5.strandsNode([
      color,
      color,
      color,
      1
    ]);
    return inputs;
  });
};

This then gets compiled to the following GLSL:

(Inputs inputs) {
  float T0 = float(0.5000);
  float T1;
  if (condition > float(0.5000))
  {
    float T2 = float(1.0000);
    T1 = T2;
  }
  else
  {
    T1 = T0;
  }
  inputs.normal = inputs.normal;
  inputs.texCoord = inputs.texCoord;
  inputs.ambientLight = inputs.ambientLight;
  inputs.ambientMaterial = inputs.ambientMaterial;
  inputs.specularMaterial = inputs.specularMaterial;
  inputs.emissiveMaterial = inputs.emissiveMaterial;
  inputs.color = vec4(T1, T1, T1, 1.0000);
  inputs.shininess = inputs.shininess;
  inputs.metalness = inputs.metalness;
  return inputs;
}

For loops

You write a loop like this:

const testShader = baseMaterialShader().modify(() => {
  getPixelInputs(inputs => {
    let color = float(0.0);

    for (let i = 0; i < 3; i++) {
      color = color + 0.1;
    }

    inputs.color = [color, color, color, 1.0];
    return inputs;
  });
});

This gets transpiled into this format, where we use a strands function call, and have a callback function for each part of the for loop. The loop body is structured more like a reduce, taking in current state + loop iteration and returning next state. At the end of the for loop, the properties of the final state are assigned back to their original variables.

() => {
 getPixelInputs(inputs => {
    let color = float(0);
    {
      const __block_1 = __p5.strandsFor(
        // Initial iterator value
        () => { return 0; },
        // Loop condition
        loopVar => (__p5.strandsNode(loopVar).lessThan(3)),
        // Loop update
        loopVar => {
          return loopVar = __p5.strandsNode(loopVar).add(1);
        },

        // Loop body, taking in current state values and returning updated state
        (loopVar, vars) => {
          let __copy_color_0 = vars.color.copy();
          __copy_color_0 = __p5.strandsNode(__copy_color_0).add(0.1);
          return { color: __copy_color_0 };
        },
        
        // Initial state
        { color: __p5.strandsNode(color) }
      );
      color = __block_1.color;
    }
    inputs.color = __p5.strandsNode([
        color,
        color,
        color,
        1
    ]);
    return inputs;
  });
};

This then gets turned into the following GLSL:

(Inputs inputs) {
  float T0 = float(0.0000);
  float T1;
  float T2 = float(0.0000);
  for (
  T1 = T2;
  (T1 < float(3.0000));
  T1 = (T1 + float(1.0000))
  )
  {
    T0 = (T0 + float(0.1000));
  }
  inputs.normal = inputs.normal;
  inputs.texCoord = inputs.texCoord;
  inputs.ambientLight = inputs.ambientLight;
  inputs.ambientMaterial = inputs.ambientMaterial;
  inputs.specularMaterial = inputs.specularMaterial;
  inputs.emissiveMaterial = inputs.emissiveMaterial;
  inputs.color = vec4(T0, T0, T0, 1.0000);
  inputs.shininess = inputs.shininess;
  inputs.metalness = inputs.metalness;
  return inputs;
}

PR Checklist

• npm run lint passes
• Inline reference is included / updated
• Unit tests are included / updated

[perminder-17]

Just wondering that we don't need to have !=, !== cases right? Like if user uses any of the other operations maybe (**) which are not in the case, can we throw an error message saying Unsupported operator?

[davepagurek]

good catch, that's probably what will happen!

[davepagurek]

** might have to be handled differently, because unlike the rest, which get turned into a method in js but then back into an operator in GLSL, ** becomes pow and needs to stay pow in GLSL, which will need some special casing in the code. I'll just leave a TODO for that one for the future.

[lukeplowden]

dynamicNode has been changed to strandsNode since the refactor

[lukeplowden]

I can't get the black to show. Maybe it has something to do with the codegen, rather than the graph though? Take a look at the outputted code. I changed the colours from this test example to make it clearer for me. I guesss that the nested if statement doesn't know that it should be updating T0 and creates its own output variable instead?

let testShader;

async function setup() {
  createCanvas(windowWidth, windowHeight, WEBGL);
  testShader = baseMaterialShader().modify(() => {
    const value = uniformFloat(() => 0.5); // middle value
    getPixelInputs(inputs => {
      let color = vec3(0.0);
      if (value > 0.8) {
        color = vec3(1.0, 0, 0); // white for high values
      } else if (value > 0.3) {
        color = vec3(0, 1, 0); // gray for medium values
      } else {
        color = vec3(0, 0, 1); // black for low values
      }
      inputs.color = [color, 1.0];
      return inputs;
    });
  });
}

function draw() {
  background(0);
  shader(testShader);
  testShader.setUniform('value', mouseX/width)
  sphere(100)
}

(Inputs inputs) {
  vec3 T0;
  if (value > float(0.8000))
  {
    vec3 T1 = vec3(1.0000, 0.0000, 0.0000);
    T0 = T1;
  }
  else
  {
    T0 = vec3(0.0000, 1.0000, 0.0000);
    vec3 T2;
    if (value > float(0.3000))
    {
      vec3 T3 = vec3(0.0000, 1.0000, 0.0000);
      T2 = T3;
    }
    else
    {
      vec3 T4 = vec3(0.0000, 0.0000, 1.0000);
      T2 = T4;
    }
  }
  inputs.normal = inputs.normal;
  inputs.texCoord = inputs.texCoord;
  inputs.ambientLight = inputs.ambientLight;
  inputs.ambientMaterial = inputs.ambientMaterial;
  inputs.specularMaterial = inputs.specularMaterial;
  inputs.emissiveMaterial = inputs.emissiveMaterial;
  inputs.color = vec4(T0, 1.0000);
  inputs.shininess = inputs.shininess;
  inputs.metalness = inputs.metalness;
  return inputs;
}

[lukeplowden]

Is it possible that the assignments arent being wrapped in strandsNode calls because of the pass order? It's causing an error if you do

  if (condition > 0.5) {
    col = 1.0;
  }

Instead of col = float(1) on the second line above.

[davepagurek]

good catch! just added a test for this and it failed, so I pushed an update to wrap all assignments in branches in strandsNode. then it works!