diff --git a/.vscode/.browse.VC.db b/.vscode/.browse.VC.db new file mode 100644 index 0000000..2547b86 Binary files /dev/null and b/.vscode/.browse.VC.db differ diff --git a/.vscode/.browse.VC.db-wal b/.vscode/.browse.VC.db-wal new file mode 100644 index 0000000..b6a863b Binary files /dev/null and b/.vscode/.browse.VC.db-wal differ diff --git a/README.md b/README.md index d4ef264..b66e515 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,38 @@ # [Project 1: Noise](https://github.com/CIS700-Procedural-Graphics/Project1-Noise) +## Description + +### Visual output + +#### Song mode: + +The circle jitters to the frequency wave of the song playing. It's not perfect, but at least it's somewhat functional. + + +#### Normal mode: + +I tried to make my circle look more like a smoothly distorting blob, rather than a rapidly-spiking explosion, because I thought it looked cooler and was more relaxing to work with as I was debugging my code. The blob is colored with its surface normals interpolated with a constant, so that they slowly morph over time. + +### Perlin noise + +I implemented this project in the following manner. First, my noise function takes as input a 3D coordinate, vec3(x, y, z). I then determine the 8 boundary lattice points encompassing the input coordinate, and sample random values for them. Instead of using a pseudo-random hash function, I hard-coded in a permutation of numbers between [0, 255] (as Perlin did in his implementation), mainly for performance reasons as my computer can barely run WebGL stuff without its fan kicking in overtime. + +Then, I interpolate between the 8 lattice points to determine a final random noise value for the original input coordinate. When interpolating, I use the "fade" function mentioned from the slide deck in class to obtain smoother interpolation. + +In addition to returning simple noise, I also implemented a function to return noise of multiple octaves combined together. This function, octaveNoise, uses persistence, frequency, and amplitude to create an improved noise function. + +### GUI + +All aspects of the octave noise function can be changed via the dat.gui module. The song can also be turned on and off. + +### Music (implemented, but not perfect...) + +I used Web Audio API to set up a mini-pipeline to play a song from "La La Land" (great movie!). I used a component in the pipeline called an Analyser to spit out song-waveform data every animation frame, which I then passed to the shader. However, I ran into a couple of problems / creative-roadblocks: + + - Every frame passed a waveform data array of length 1024, which was too large to send to my vertex shader via a uniform variable (the compiler spit out some "too many uniforms error"). This meant that I was unable to work with the waveform data unless I (1) found a way to compress it or (2) tried attaching the data onto vertices via vertex attributes outside of the shader. I ended up smoothing the original waveform via a moving average, and then passing it to the shader as a smaller array. + +- Additionally, I struggled to find a good way to incorporate the data into the animation. Perhaps the song I chose was too loud / had too many simultaneous instruments, because whenever I attempted to add in song data to my shader, the animation became very jittery, which was no good. As of now, the circle jitters somewhat in tune with the beat and volume of the song. + ## Objective Get comfortable with using three.js and its shader support and generate an interesting 3D, continuous surface using a multi-octave noise algorithm. @@ -37,7 +70,7 @@ You can skip this part if you really want, but I highly suggest you read it. This is the important file that `npm` looks at. In it, you can see the commands it's using for the `start`, `build`, and `deploy` scripts mentioned above. You can also see all of the dependencies the project requires. I will briefly go through what each of these is. - dat-gui: Gives us a nice and simple GUI for modifying variables in our program - + - gl-matrix: Useful library for linear algebra, much like glm - stats-js: Gives us a nice graph for timing things. We use it to report how long it takes to render each frame @@ -72,7 +105,7 @@ Note that three.js automatically injects several uniform and attribute variables ## Noise Generation -In the shader, write a 3D multi-octave lattice-value noise function that takes three input parameters and generates output in a controlled range, say [0,1] or [-1, 1]. This will require the following steps. +In the shader, write a 3D multi-octave lattice-value noise function that takes three input parameters and generates output in a controlled range, say [0,1] or [-1, 1]. This will require the following steps. 1. Write several (for however many octaves of noise you want) basic pseudo-random 3D noise functions (the hash-like functions we discussed in class). It's fine to reference one from the slides or elsewhere on the Internet. Again, this should just be a set of math operations, often using large prime numbers to random-looking output from three input parameters. @@ -106,13 +139,3 @@ Using dat.GUI and the examples provided in the reference code, make some aspect - Mouse interactivity (medium): Find out how to get the current mouse position in your scene and use it to deform your cloud, such that users can deform the cloud with their cursor. - Music (hard): Figure out a way to use music to drive your noise animation in some way, such that your noise cloud appears to dance. - -## Submission - -- Update README.md to contain a solid description of your project - -- Publish your project to gh-pages. `npm run deploy`. It should now be visible at http://username.github.io/repo-name - -- Create a [pull request](https://help.github.com/articles/creating-a-pull-request/) to this repository, and in the comment, include a link to your published project. - -- Submit the link to your pull request on Canvas. \ No newline at end of file diff --git a/adam.jpg b/adam.jpg deleted file mode 100644 index a762190..0000000 Binary files a/adam.jpg and /dev/null differ diff --git a/src/framework.js b/src/framework.js index 9cfcd1b..59daf4a 100644 --- a/src/framework.js +++ b/src/framework.js @@ -1,4 +1,3 @@ - const THREE = require('three'); const OrbitControls = require('three-orbit-controls')(THREE) import Stats from 'stats-js' @@ -25,7 +24,7 @@ function init(callback, update) { window.addEventListener('load', function() { var scene = new THREE.Scene(); - var camera = new THREE.PerspectiveCamera( 75, window.innerWidth/window.innerHeight, 0.1, 1000 ); + var camera = new THREE.PerspectiveCamera( 100, window.innerWidth/window.innerHeight, 0.1, 1000 ); var renderer = new THREE.WebGLRenderer( { antialias: true } ); renderer.setPixelRatio(window.devicePixelRatio); renderer.setSize(window.innerWidth, window.innerHeight); diff --git a/src/main.js b/src/main.js index 92b19a4..bbfc192 100644 --- a/src/main.js +++ b/src/main.js @@ -1,60 +1,135 @@ -const THREE = require('three'); // older modules are imported like this. You shouldn't have to worry about this much -import Framework from './framework' -import Noise from './noise' -import {other} from './noise' +const THREE = require('three'); +import Framework from './framework'; + +// Initialize Web Audio API stuff +var ctx = new (window.AudioContext || window.webkitAudioContext)(); +var src = ctx.createBufferSource(); +var analyser = ctx.createAnalyser(); + +// Connect/disconnect GUI toggle for audio +var isPlaying = true; +var audioGUI = { + toggle: function () { + if (isPlaying) { + src.disconnect(); + isPlaying = false; + uniforms.isAudioPlaying.value = 0; + } else { + src.connect(analyser); + isPlaying = true; + uniforms.isAudioPlaying.value = 1; + } + } +} + +// Make GET request for song +var req = new XMLHttpRequest(); +var url = 'http://zelliottm.com/assets/song.mp3'; + +req.open('GET', url, true) +req.responseType = 'arraybuffer'; +req.onload = function() { + + // Decode data and link every node up + ctx.decodeAudioData(req.response, function(buffer) { + src = ctx.createBufferSource(); + src.buffer = buffer; + src.connect(analyser); + + analyser.connect(ctx.destination); + + // Toggle song + src.start(); + }); +}; +req.send(); + +// This array will hold the audio data used +// to animate our blob +var audioData = new Float32Array(analyser.frequencyBinCount); + +// Smoothed data generated from the frequency +// array above. +var smoothedAudioData = []; + +// Declare uniforms +const perm = [ + 151,160,137,91,90,15,131,13,201,95,96,53,194,233,7,225,140,36,103,30,69,142,8,99,37,240,21, + 10,23,190,6,148,247,120,234,75,0,26,197,62,94,252,219,203,117,35,11,32,57,177,33,88,237,149, + 56,87,174,20,125,136,171,168,68,175,74,165,71,134,139,48,27,166,77,146,158,231,83,111,229, + 122,60,211,133,230,220,105,92,41,55,46,245,40,244,102,143,54,65,25,63,161,1,216,80,73,209, + 76,132,187,208,89,18,169,200,196,135,130,116,188,159,86,164,100,109,198,173,186,3,64,52,217, + 226,250,124,123,5,202,38,147,118,126,255,82,85,212,207,206,59,227,47,16,58,17,182,189,28, + 42,223,183,170,213,119,248,152,2,44,154,163,70,221,153,101,155,167,43,172,9,129,22,39,253, + 19,98,108,110,79,113,224,232,178,185,112,104,218,246,97,228,251,34,242,193,238,210,144, + 12,191,179,162,241, 81,51,145,235,249,14,239,107,49,192,214,31,181,199,106,157,184,84,204, + 176,115,121,50,45,127,4,150,254,138,236,205,93,222,114,67,29,24,72,243,141,128,195,78,66, + 215,61,156,180 +]; +var start = Date.now(); +var uniforms = { + speed: { value: 2.0 }, + time: { value: 0.0 }, + freq: { value: 0.6 }, // frequency + pers: { value: 0.20 }, // persistence + amp: { value: 1.0 }, // amplitude + octaves: { value: 6 }, // number of octaves + p: { value: perm }, // perm array, + smoothedAudioData: { value: smoothedAudioData }, + isAudioPlaying: { value: 1 } +}; // called after the scene loads function onLoad(framework) { - var scene = framework.scene; - var camera = framework.camera; - var renderer = framework.renderer; - var gui = framework.gui; - var stats = framework.stats; - - // LOOK: the line below is synyatic sugar for the code above. Optional, but I sort of recommend it. - // var {scene, camera, renderer, gui, stats} = framework; - - // initialize a simple box and material - var box = new THREE.BoxGeometry(1, 1, 1); - - var adamMaterial = new THREE.ShaderMaterial({ - uniforms: { - image: { // Check the Three.JS documentation for the different allowed types and values - type: "t", - value: THREE.ImageUtils.loadTexture('./adam.jpg') - } - }, - vertexShader: require('./shaders/adam-vert.glsl'), - fragmentShader: require('./shaders/adam-frag.glsl') + var {scene, camera, renderer, gui, stats} = framework; + + // Load geometry & mesh + var geom = new THREE.IcosahedronBufferGeometry(1, 4); + var material = new THREE.ShaderMaterial({ + uniforms: uniforms, + vertexShader: require('./shaders/icos-vert.glsl'), + fragmentShader: require('./shaders/icos-frag.glsl') }); - var adamCube = new THREE.Mesh(box, adamMaterial); + var mesh = new THREE.Mesh(geom, material); - // set camera position + // Set camera position camera.position.set(1, 1, 2); camera.lookAt(new THREE.Vector3(0,0,0)); - scene.add(adamCube); + scene.add(mesh); - // edit params and listen to changes like this - // more information here: https://workshop.chromeexperiments.com/examples/gui/#1--Basic-Usage + // More info here: https://workshop.chromeexperiments.com/examples/gui/#1--Basic-Usage gui.add(camera, 'fov', 0, 180).onChange(function(newVal) { camera.updateProjectionMatrix(); }); -} -// called on frame updates -function onUpdate(framework) { - // console.log(`the time is ${new Date()}`); + gui.add(uniforms.speed, 'value', 1, 10).name('speed'); + gui.add(uniforms.freq, 'value', 0, 4).name('frequency'); + gui.add(uniforms.pers, 'value', 0, 2).name('persistence'); + gui.add(uniforms.amp, 'value', 0, 2).name('amplitude'); + gui.add(uniforms.octaves, 'value', 0, 10).name('octaves').step(1.0); + gui.add(audioGUI, 'toggle').name('toggle song'); } -// when the scene is done initializing, it will call onLoad, then on frame updates, call onUpdate -Framework.init(onLoad, onUpdate); +// Called on frame updates +function onUpdate(framework) { + analyser.getFloatTimeDomainData(audioData); -// console.log('hello world'); + // Smooth audio data via a moving average + var smoothStep = 16; + for (var i = 0; i < audioData.length / smoothStep; i++) { + var total = 0; + var index = i * smoothStep; + for (var j = 0; j < smoothStep; j++) { + total += audioData[index + j]; + } -// console.log(Noise.generateNoise()); + smoothedAudioData[i] = total / smoothStep; + } -// Noise.whatever() + uniforms.smoothedAudioData.value = smoothedAudioData; + uniforms.time.value = Date.now() - start; +} -// console.log(other()) \ No newline at end of file +Framework.init(onLoad, onUpdate); \ No newline at end of file diff --git a/src/shaders/adam-frag.glsl b/src/shaders/adam-frag.glsl deleted file mode 100644 index 5dfa18c..0000000 --- a/src/shaders/adam-frag.glsl +++ /dev/null @@ -1,13 +0,0 @@ -varying vec2 vUv; -varying float noise; -uniform sampler2D image; - - -void main() { - - vec2 uv = vec2(1,1) - vUv; - vec4 color = texture2D( image, uv ); - - gl_FragColor = vec4( color.rgb, 1.0 ); - -} \ No newline at end of file diff --git a/src/shaders/adam-vert.glsl b/src/shaders/adam-vert.glsl deleted file mode 100644 index e4b8cc0..0000000 --- a/src/shaders/adam-vert.glsl +++ /dev/null @@ -1,6 +0,0 @@ - -varying vec2 vUv; -void main() { - vUv = uv; - gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 ); -} \ No newline at end of file diff --git a/src/shaders/icos-frag.glsl b/src/shaders/icos-frag.glsl new file mode 100644 index 0000000..bf8fc0e --- /dev/null +++ b/src/shaders/icos-frag.glsl @@ -0,0 +1,24 @@ +varying float randTime; +varying float randAudio; +varying vec3 vNormal; + +uniform float time; +uniform vec3 color; + +vec3 rgb(vec3 v) { + return v / 255.0; +} + +vec3 lerp(vec3 a, vec3 b, float t) { + return (a * (1.0 - t)) + (b * t); +} + +vec3 cosinterp(vec3 a, vec3 b, float t) { + const float PI = 3.14159265358979323; + float tCos = (1.0 - cos(t * PI)) * 0.5; + return lerp(a, b, tCos); +} + +void main() { + gl_FragColor = vec4(cosinterp(vNormal, vec3(0.9, 0.9, 0.9), randTime), 1.0); +} \ No newline at end of file diff --git a/src/shaders/icos-vert.glsl b/src/shaders/icos-vert.glsl new file mode 100644 index 0000000..ad3bbd3 --- /dev/null +++ b/src/shaders/icos-vert.glsl @@ -0,0 +1,139 @@ +// Implementation referenced: +// Perlin's Improved Noise: http://mrl.nyu.edu/~perlin/noise/ +// explanations of Improved Noise: http://flafla2.github.io/2014/08/09/perlinnoise.html#the-hash-function + +varying float randTime; +varying float randAudio; +varying vec3 vNormal; + +uniform float speed; +uniform float time; +uniform float freq; +uniform float pers; +uniform float amp; +uniform int octaves; +uniform float p[256]; +uniform float smoothedAudioData[64]; // 1024 / 16 = 64 (frequency.binCount / smoothStep) +uniform int isAudioPlaying; + +float lerp(float a, float b, float t) { + return (a * (1.0 - t)) + (b * t); +} + +float cosinterp(float a, float b, float t) { + const float PI = 3.14159265358979323; + float tCos = (1.0 - cos(t * PI)) * 0.5; + return lerp(a, b, tCos); +} + +float fade(float t) { + return t * t * t * (t * (t * 6.0 - 15.0) + 10.0); +} + +float grad(float hash, float x, float y, float z) { + float h = mod(hash, 256.0); + float u = (h < 8.0) ? x : y; + float v = (h < 4.0) ? y : ((h == 12.0 || h == 14.0) ? x : z); + return ((mod(h, 2.0) == 0.0) ? u : -u) + (((h == 0.0) || (h == 1.0)) ? v : -v); +} + +float perm(float t) { + if (t > 256.0) t = mod(t, 256.0); + return p[int(t)]; +} + +float nestedPerm(float a, float b, float c) { + return perm(perm(perm(a) + b) + c); +} + +float noise(vec3 v) { + + // Determine the unit cube that contains the point (x, y, z) + float xi = mod(floor(v.x), 256.0); + float yi = mod(floor(v.y), 256.0); + float zi = mod(floor(v.z), 256.0); + + // Incremented copies of xi, yi, zi + float xj = xi + 1.0; + float yj = yi + 1.0; + float zj = zi + 1.0; + + // Relative coordinate of point inside cube + float xf = v.x - floor(v.x); + float yf = v.y - floor(v.y); + float zf = v.z - floor(v.z); + + // Decremented copies of xf, yf, zf + float xg = xf - 1.0; + float yg = yf - 1.0; + float zg = zf - 1.0; + + // Compute fade curves for point + float a = fade(xf); + float b = fade(yf); + float c = fade(zf); + + // Hash values for each of the 8 cube corners + float aaa = nestedPerm(xi, yi, zi); + float baa = nestedPerm(xj, yi, zi); + float aba = nestedPerm(xi, yj, zi); + float aab = nestedPerm(xi, yi, zj); + float bba = nestedPerm(xj, yj, zi); + float bab = nestedPerm(xj, yi, zj); + float abb = nestedPerm(xi, yj, zj); + float bbb = nestedPerm(xj, yj, zj); + + // Interpolate surrounding 8 lattice values + float laa = cosinterp(grad(aaa, xf, yf, zf), grad(baa, xg, yf, zf), a); + float lab = cosinterp(grad(aba, xf, yg, zf), grad(bba, xg, yg, zf), a); + float lba = cosinterp(grad(aab, xf, yf, zg), grad(bab, xg, yf, zg), a); + float lbb = cosinterp(grad(abb, xf, yg, zg), grad(bbb, xg, yg, zg), a); + float la = cosinterp(laa, lab, b); + float lb = cosinterp(lba, lbb, b); + float l = cosinterp(la, lb, c); + + // Change range from [-1, 1] to [0, 1] + return (l + 1.0) / 2.0; +} + +float octaveNoise(vec3 v) { + float total = 0.0; + float maxNoise = 0.0; + float ampV = amp; + float freqV = freq; + + // GLSL doesn't compile unless the for loop is + // guaranteed to finish. + const int maxOctaves = 100; + + for (int i = 0; i < maxOctaves; i++) { + if (i > octaves) break; + + v *= freqV; + total += noise(v) * ampV; + maxNoise += ampV; + + ampV *= pers; + freqV *= 2.0; + } + + // Again, normalize between [0, 1] + return total / maxNoise; +} + +float audioNoise(vec3 v) { + vec3 vec = normalize(vec3(1.0, 1.0, 2.0)); + int dotT = int((1.0 - ((dot(vec, normalize(v)) + 1.0) / 2.0)) * 64.0); + return smoothedAudioData[dotT]; +} + +void main() { + vNormal = normal; + randTime = octaveNoise(position + (time * speed / 2000.0)); + randAudio = audioNoise(position); + + float displacement = (isAudioPlaying == 1) ? randAudio : randTime; + vec3 noisePosition = position + (normal * displacement / 2.0); + + gl_Position = projectionMatrix * modelViewMatrix * vec4(noisePosition, 1.0); +} \ No newline at end of file