Zack Elliott

README.md

@@ -1,5 +1,38 @@
 # [Project 1: Noise](https://github.com/CIS700-Procedural-Graphics/Project1-Noise)
 
+## Description
+
+### Visual output
+
+#### Song mode:
+
+The circle jitters to the frequency wave of the song playing.  It's not perfect, but at least it's somewhat functional.
+
+
+#### Normal mode:
+
+I tried to make my circle look more like a smoothly distorting blob, rather than a rapidly-spiking explosion, because I thought it looked cooler and was more relaxing to work with as I was debugging my code.  The blob is colored with its surface normals interpolated with a constant, so that they slowly morph over time.
+
+### Perlin noise
+
+I implemented this project in the following manner.  First, my noise function takes as input a 3D coordinate, vec3(x, y, z).  I then determine the 8 boundary lattice points encompassing the input coordinate, and sample random values for them.  Instead of using a pseudo-random hash function, I hard-coded in a permutation of numbers between [0, 255] (as Perlin did in his implementation), mainly for performance reasons as my computer can barely run WebGL stuff without its fan kicking in overtime.
+
+Then, I interpolate between the 8 lattice points to determine a final random noise value for the original input coordinate.  When interpolating, I use the "fade" function mentioned from the slide deck in class to obtain smoother interpolation.
+
+In addition to returning simple noise, I also implemented a function to return noise of multiple octaves combined together.  This function, octaveNoise, uses persistence, frequency, and amplitude to create an improved noise function.
+
+### GUI
+
+All aspects of the octave noise function can be changed via the dat.gui module.  The song can also be turned on and off.
+
+### Music (implemented, but not perfect...)
+
+I used Web Audio API to set up a mini-pipeline to play a song from "La La Land" (great movie!).  I used a component in the pipeline called an Analyser to spit out song-waveform data every animation frame, which I then passed to the shader.  However, I ran into a couple of problems / creative-roadblocks:
+
+ - Every frame passed a waveform data array of length 1024, which was too large to send to my vertex shader via a uniform variable (the compiler spit out some "too many uniforms error").  This meant that I was unable to work with the waveform data unless I (1) found a way to compress it or (2) tried attaching the data onto vertices via vertex attributes outside of the shader.  I ended up smoothing the original waveform via a moving average, and then passing it to the shader as a smaller array.
+
+- Additionally, I struggled to find a good way to incorporate the data into the animation.  Perhaps the song I chose was too loud / had too many simultaneous instruments, because whenever I attempted to add in song data to my shader, the animation became very jittery, which was no good.  As of now, the circle jitters somewhat in tune with the beat and volume of the song.
+
 ## Objective
 
 Get comfortable with using three.js and its shader support and generate an interesting 3D, continuous surface using a multi-octave noise algorithm.
@@ -37,7 +70,7 @@ You can skip this part if you really want, but I highly suggest you read it.
 
 This is the important file that `npm` looks at. In it, you can see the commands it's using for the `start`, `build`, and `deploy` scripts mentioned above. You can also see all of the dependencies the project requires. I will briefly go through what each of these is.
  - dat-gui: Gives us a nice and simple GUI for modifying variables in our program
- 
+
  - gl-matrix: Useful library for linear algebra, much like glm
 
  - stats-js: Gives us a nice graph for timing things. We use it to report how long it takes to render each frame
@@ -72,7 +105,7 @@ Note that three.js automatically injects several uniform and attribute variables
 
 ## Noise Generation
 
-In the shader, write a 3D multi-octave lattice-value noise function that takes three input parameters and generates output in a controlled range, say [0,1] or [-1, 1]. This will require the following steps. 
+In the shader, write a 3D multi-octave lattice-value noise function that takes three input parameters and generates output in a controlled range, say [0,1] or [-1, 1]. This will require the following steps.
 
 1. Write several (for however many octaves of noise you want) basic pseudo-random 3D noise functions (the hash-like functions we discussed in class). It's fine to reference one from the slides or elsewhere on the Internet. Again, this should just be a set of math operations, often using large prime numbers to random-looking output from three input parameters.
 
@@ -106,13 +139,3 @@ Using dat.GUI and the examples provided in the reference code, make some aspect
 - Mouse interactivity (medium): Find out how to get the current mouse position in your scene and use it to deform your cloud, such that users can deform the cloud with their cursor.
 
 - Music (hard): Figure out a way to use music to drive your noise animation in some way, such that your noise cloud appears to dance.
-
-## Submission
-
-- Update README.md to contain a solid description of your project
-
-- Publish your project to gh-pages. `npm run deploy`. It should now be visible at http://username.github.io/repo-name
-
-- Create a [pull request](https://help.github.com/articles/creating-a-pull-request/) to this repository, and in the comment, include a link to your published project.
-
-- Submit the link to your pull request on Canvas.

src/framework.js

@@ -1,4 +1,3 @@
-
 const THREE = require('three');
 const OrbitControls = require('three-orbit-controls')(THREE)
 import Stats from 'stats-js'
@@ -25,7 +24,7 @@ function init(callback, update) {
   window.addEventListener('load', function() {
 
     var scene = new THREE.Scene();
-    var camera = new THREE.PerspectiveCamera( 75, window.innerWidth/window.innerHeight, 0.1, 1000 );
+    var camera = new THREE.PerspectiveCamera( 100, window.innerWidth/window.innerHeight, 0.1, 1000 );
     var renderer = new THREE.WebGLRenderer( { antialias: true } );
     renderer.setPixelRatio(window.devicePixelRatio);
     renderer.setSize(window.innerWidth, window.innerHeight);

src/main.js

@@ -1,60 +1,135 @@
 
-const THREE = require('three'); // older modules are imported like this. You shouldn't have to worry about this much
-import Framework from './framework'
-import Noise from './noise'
-import {other} from './noise'
+const THREE = require('three');
+import Framework from './framework';
+
+// Initialize Web Audio API stuff
+var ctx = new (window.AudioContext || window.webkitAudioContext)();
+var src = ctx.createBufferSource();
+var analyser = ctx.createAnalyser();
+
+// Connect/disconnect GUI toggle for audio
+var isPlaying = true;
+var audioGUI = {
+  toggle: function () {
+    if (isPlaying) {
+      src.disconnect();
+      isPlaying = false;
+      uniforms.isAudioPlaying.value = 0;
+    } else {
+      src.connect(analyser);
+      isPlaying = true;
+      uniforms.isAudioPlaying.value = 1;
+    }
+  }
+}
+
+// Make GET request for song
+var req = new XMLHttpRequest();
+var url = 'http://zelliottm.com/assets/song.mp3';
+
+req.open('GET', url, true)
+req.responseType = 'arraybuffer';
+req.onload = function() {
+
+    // Decode data and link every node up
+    ctx.decodeAudioData(req.response, function(buffer) {
+      src = ctx.createBufferSource();
+      src.buffer = buffer;
+      src.connect(analyser);
+
+      analyser.connect(ctx.destination);
+
+      // Toggle song
+      src.start();
+    });
+};
+req.send();
+
+// This array will hold the audio data used
+// to animate our blob
+var audioData = new Float32Array(analyser.frequencyBinCount);
+
+// Smoothed data generated from the frequency
+// array above.
+var smoothedAudioData = [];
+
+// Declare uniforms
+const perm = [
+  151,160,137,91,90,15,131,13,201,95,96,53,194,233,7,225,140,36,103,30,69,142,8,99,37,240,21,
+  10,23,190,6,148,247,120,234,75,0,26,197,62,94,252,219,203,117,35,11,32,57,177,33,88,237,149,
+  56,87,174,20,125,136,171,168,68,175,74,165,71,134,139,48,27,166,77,146,158,231,83,111,229,
+  122,60,211,133,230,220,105,92,41,55,46,245,40,244,102,143,54,65,25,63,161,1,216,80,73,209,
+  76,132,187,208,89,18,169,200,196,135,130,116,188,159,86,164,100,109,198,173,186,3,64,52,217,
+  226,250,124,123,5,202,38,147,118,126,255,82,85,212,207,206,59,227,47,16,58,17,182,189,28,
+  42,223,183,170,213,119,248,152,2,44,154,163,70,221,153,101,155,167,43,172,9,129,22,39,253,
+  19,98,108,110,79,113,224,232,178,185,112,104,218,246,97,228,251,34,242,193,238,210,144,
+  12,191,179,162,241, 81,51,145,235,249,14,239,107,49,192,214,31,181,199,106,157,184,84,204,
+  176,115,121,50,45,127,4,150,254,138,236,205,93,222,114,67,29,24,72,243,141,128,195,78,66,
+  215,61,156,180
+];
+var start = Date.now();
+var uniforms = {
+  speed: { value: 2.0 },
+  time: { value: 0.0 },
+  freq: { value: 0.6 },           // frequency
+  pers: { value: 0.20 },           // persistence
+  amp: { value: 1.0 },            // amplitude
+  octaves: { value: 6 },          // number of octaves
+  p: { value: perm },             // perm array,
+  smoothedAudioData: { value: smoothedAudioData },
+  isAudioPlaying: { value: 1 }
+};
 
 // called after the scene loads
 function onLoad(framework) {
-  var scene = framework.scene;
-  var camera = framework.camera;
-  var renderer = framework.renderer;
-  var gui = framework.gui;
-  var stats = framework.stats;
-
-  // LOOK: the line below is synyatic sugar for the code above. Optional, but I sort of recommend it.
-  // var {scene, camera, renderer, gui, stats} = framework; 
-
-  // initialize a simple box and material
-  var box = new THREE.BoxGeometry(1, 1, 1);
-
-  var adamMaterial = new THREE.ShaderMaterial({
-    uniforms: {
-      image: { // Check the Three.JS documentation for the different allowed types and values
-        type: "t", 
-        value: THREE.ImageUtils.loadTexture('./adam.jpg')
-      }
-    },
-    vertexShader: require('./shaders/adam-vert.glsl'),
-    fragmentShader: require('./shaders/adam-frag.glsl')
+  var {scene, camera, renderer, gui, stats} = framework;
+
+  // Load geometry & mesh
+  var geom = new THREE.IcosahedronBufferGeometry(1, 4);
+  var material = new THREE.ShaderMaterial({
+    uniforms: uniforms,
+    vertexShader: require('./shaders/icos-vert.glsl'),
+    fragmentShader: require('./shaders/icos-frag.glsl')
   });
-  var adamCube = new THREE.Mesh(box, adamMaterial);
+  var mesh = new THREE.Mesh(geom, material);
 
-  // set camera position
+  // Set camera position
   camera.position.set(1, 1, 2);
   camera.lookAt(new THREE.Vector3(0,0,0));
 
-  scene.add(adamCube);
+  scene.add(mesh);
 
-  // edit params and listen to changes like this
-  // more information here: https://workshop.chromeexperiments.com/examples/gui/#1--Basic-Usage
+  // More info here: https://workshop.chromeexperiments.com/examples/gui/#1--Basic-Usage
   gui.add(camera, 'fov', 0, 180).onChange(function(newVal) {
     camera.updateProjectionMatrix();
   });
-}
 
-// called on frame updates
-function onUpdate(framework) {
-  // console.log(`the time is ${new Date()}`);
+  gui.add(uniforms.speed, 'value', 1, 10).name('speed');
+  gui.add(uniforms.freq, 'value', 0, 4).name('frequency');
+  gui.add(uniforms.pers, 'value', 0, 2).name('persistence');
+  gui.add(uniforms.amp, 'value', 0, 2).name('amplitude');
+  gui.add(uniforms.octaves, 'value', 0, 10).name('octaves').step(1.0);
+  gui.add(audioGUI, 'toggle').name('toggle song');
 }
 
-// when the scene is done initializing, it will call onLoad, then on frame updates, call onUpdate
-Framework.init(onLoad, onUpdate);
+// Called on frame updates
+function onUpdate(framework) {
+  analyser.getFloatTimeDomainData(audioData);
 
-// console.log('hello world');
+  // Smooth audio data via a moving average
+  var smoothStep = 16;
+  for (var i = 0; i < audioData.length / smoothStep; i++) {
+    var total = 0;
+    var index = i * smoothStep;
+    for (var j = 0; j < smoothStep; j++) {
+      total += audioData[index + j];
+    }
 
-// console.log(Noise.generateNoise());
+    smoothedAudioData[i] = total / smoothStep;
+  }
 
-// Noise.whatever()
+  uniforms.smoothedAudioData.value = smoothedAudioData;
+  uniforms.time.value = Date.now() - start;
+}
 
-// console.log(other())
+Framework.init(onLoad, onUpdate);

src/shaders/icos-frag.glsl

@@ -0,0 +1,24 @@
+varying float randTime;
+varying float randAudio;
+varying vec3 vNormal;
+
+uniform float time;
+uniform vec3 color;
+
+vec3 rgb(vec3 v) {
+  return v / 255.0;
+}
+
+vec3 lerp(vec3 a, vec3 b, float t) {
+    return (a * (1.0 - t)) + (b * t);
+}
+
+vec3 cosinterp(vec3 a, vec3 b, float t) {
+    const float PI = 3.14159265358979323;
+    float tCos = (1.0 - cos(t * PI)) * 0.5;
+    return lerp(a, b, tCos);
+}
+
+void main() {
+  gl_FragColor = vec4(cosinterp(vNormal, vec3(0.9, 0.9, 0.9), randTime), 1.0);
+}