Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 33 additions & 56 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,72 +1,49 @@
Assignment 4 - Creative Coding: Interactive Multimedia Experiences
===

Due: September 27th, by 11:59 PM.

For this assignment we will focus on client-side development using popular audio/graphics/visualization technologies; the server requirements are minimal. The goal of this assignment is to refine our JavaScript knowledge while exploring the multimedia capabilities of the browser.

Baseline Requirements
---

Your application is required to implement the following functionalities:

- A server created using Express (you can also use an alternative server framework such as Koa) for basic file delivery and middleware. Your middleware stack should include the `compression` and `helmet` [middlewares]((https://expressjs.com/en/resources/middleware.html)) by default. You are not required to use Glitch for this assignment (but using Glitch is fine!); [Heroku](https://www.heroku.com) is another excellent option to explore. The course staff can't be resposible for helping with all other hosting options outside of Glitch, but some of us do have experience with other systems. It also never hurts to ask on Slack, as there's 99 other classmates who might have the experience you're looking for!
- A client-side interactive experience using at least one of the web technologies frameworks we discussed in class over the past week.
- [Three.js](https://threejs.org/): A library for 3D graphics / VR experiences
- [D3.js](https://d3js.org): A library that is primarily used for interactive data visualizations
- [Canvas](https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API): A 2D raster drawing API included in all modern browsers
- [SVG](https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API): A 2D vector drawing framework that enables shapes to be defined via XML.
- [Web Audio API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API): An API for audio synthesis, analysis, processing, and file playback.
- A user interface for interaction with your project, which must expose at least six parameters for user control. [dat.gui](https://workshop.chromeexperiments.com/examples/gui/#1--Basic-Usage) is highly recommended for this. You might also explore interaction by tracking mouse movement via the `window.onmousemove` event handler in tandem with the `event.clientX` and `event.clientY` properties. Consider using the [Pointer Events API](https://developer.mozilla.org/en-US/docs/Web/API/Pointer_events) to ensure that that mouse and touch events will both be supported in your app.
- Your application should display basic documentation for the user interface when the application first loads. This documentation should be dismissable, however, users should be able to redisplay it via either a help buton (this could, for example, be inside a dat.gui interface) or via a keyboard shortcut (commonly the question mark).
- Your application should feature at least two different ES6 modules that you write ([read about ES6 modules](https://www.sitepoint.com/understanding-es6-modules/)) and include into a main JavaScript file. This means that you will need to author *at least three JavaScript files* (a `app.js` or `main.js` file and two modules). We'll discuss modules in class on Monday 9/23; for this assignment modules should contain at least two functions.
- You are required to use a linter for your JavaScript. There are plugins for most IDEs, however it will be difficult to run the linter directly in Glitch. If you haven't moved to developing on your personal laptop and then uploading to Glitch when your project is completed, this is the assignment to do so!
- Your HTML and CSS should validate. There are options/plugins for most IDEs to check validation.

The interactive experience should possess a reasonable level of complexity. Some examples:
### Three.js
- A generative algorithm creates simple agents that move through a virtual world. Your interface controls the behavior / appearance of these agents.
- A simple 3D game
- An 3D audio visualization of a song of your choosing. User interaction should control aspects of the visualization.
### Canvas
- Implement a generative algorithm such as [Conway's Game of Life](https://bitstorm.org/gameoflife/) (or 1D cellular automata) and provide interactive controls. Note that the Game of Life has been created by 100s of people using <canvas>; we'll be checking to ensure that your implementation is not a copy of these.
- Design a 2D audio visualizer of a song of your choosing. User interaction should control visual aspects of the experience.
### Web Audio API
- Create a screen-based musical instrument using the Web Audio API. You can use projects such as [Interface.js](http://charlie-roberts.com/interface/) or [Nexus UI](https://nexus-js.github.io/ui/api/#Piano) to provide common musical interface elements, or use dat.GUI in combination with mouse/touch events (use the Pointer Events API). Your GUI should enable users to control aspects of sound synthesis.
### D3.js
- Create visualizations using the datasets found at [Awesome JSON Datasets](https://github.com/jdorfman/Awesome-JSON-Datasets). Experiment with providing different visualizations of the same data set, and providing users interactive control over visualization parameters and/or data filtering. Alternatively, create a single visualization with using one of the more complicated techniques shown at [d3js.org](d3js.org) and provide meaningful points of interaction for users.
## A Digital Visualization Portfolio
Link: http://a4-petrakumi12.glitch.me

Deliverables
---
Include a very brief summary of your project here. Images are encouraged, along with concise, high-level text. Be sure to include:

Do the following to complete this assignment:
- The goal of this application is to provide a single source for multiple types of data visualization, to have as a personal portfolio.
- This application contains both implementations of code snippets I found online which I then manipulated to my liking
(the main page background and the WebGL Sound visualization), as well as visualizations I made from scratch
(the fractal and canvas sound visualization). In both cases, I made sure to understand every part of the code that I included.
- I included the online snippets because they helped me understand the workings of these libraries in order for me to
more easily implement my own functionality afterwards.
- The biggest challenge I faced was getting started using all the visualization libraries, as I did not have any prior experience with any of them

1. Implement your project with the above requirements.
3. Test your project to make sure that when someone goes to your main page on Glitch/Heroku/etc., it displays correctly.
4. Ensure that your project has the proper naming scheme `a4-yourname` so we can find it.
5. Fork this repository and modify the README to the specifications below. *NOTE: If you don't use Glitch for hosting (where we can see the files) then you must include all project files that you author in your repo for this assignment*.
6. Create and submit a Pull Request to the original repo. Name the pull request using the following template: `a4-gitname-firstname-lastname`.
- I used ESLinter through WebStorm to lint, using the automatic configuration. I used WebStorm's automatic validator for code validation

Sample Readme (delete the above when you're ready to submit, and modify the below so with your links and descriptions)
---
** I added two sample songs to the Songs folder that you can download and try the audio visualizers with.

## Your Web Application Title
## Technical Achievements
- **Tech Achievement 1**: Used canvas, WebGL, D3, and Three.js on my visualizations
- **Tech Achievement 2**: My audiovisualizers use amplitude analysis to drive visualization.
- **Tech Achievement 3**: I experimented with custom FragmentShaders for color generation based on sound amplitude.
- **Tech Achievement 4**: I visualized the dataset of characters in Thor that appear in the same scene together using a
correlation circle with D3, including user interaction by hovering over for more information on the data
- **Tech Achievement 5**: I added start/stop functionality to the fractal generator, which users can press to stop the generation
- **Tech Achievement 6**: I added dat.gui to the fractal generator, allowing the user to change max radius, min radius,
and color of the circles. The color change will apply on top of the current image, while the new radii will cause the
context to clear and the visualization to start from scratch.
- **Tech Achievement 7**: I used dat.gui to control characteristics of the TorusKnotGeometry
- **Tech Achievement 8**: Automatic resize of three js renderer with screen size change on the main page
- **Tech Achievement 9**: Used js modules to generate the D3 graph as well as set the stage for audio analysis given the
inputted file. I then used the latter in both audio visualizers.
- **Tech Achievement 10**: Allowed the user to click anywhere to dismiss the help text, and click back at the Typed
cursor to regenerate the animation
- **Tech Achievement 11**: Generated color gradients to use in the linear sound visualizer, which changes
the color based on the amplitude of the sound

your hosting link e.g. http://a4-charlieroberts.glitch.me

Include a very brief summary of your project here. Images are encouraged, along with concise, high-level text. Be sure to include:
### Design/Evaluation Achievements
- **Design Achievement 1**: Used Typed library for quick help on how to navigate the website on the main page
- **Design Achievement 2**: Used Bootstrap with some modifications for the main page layout, with a parallax-style effect

- the goal of the application
- challenges you faced in realizing the application
- a brief description of the JS linter you used and what rules it follows (we'll be looking at your JS files for consistency)

## Technical Achievements
- **Tech Achievement 1**: I wrote my own custom GLSL shaders to use as a material for my Three.js objects.
- **Tech Achievement 2**: My audiovisualizer uses both FFT and amplitude analysis to drive visualization.
- **Tech Achievement 3**: I optimized the efficiency of my reaction-diffusion algorithm by...
- **Tech Achievement 4**: I visualized the dataset X using three different visualization technqiues provided by D3, andprovided

### Design/Evaluation Achievements
- **Design Achievement 1**: I ensured that my application would run on both desktops / mobile devices by changing X
- **Design Achievement 2**: I followed best practices for accessibility, including providing alt attributes for images and using semantic HTML. There are no `<div>` or `<span>` elements in my document.
- **Design Achievement 3**: We tested the application with n=X users, finding that...
Binary file not shown.
Binary file not shown.
23 changes: 23 additions & 0 deletions app.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
const express = require('express'),
bodyParser = require('body-parser'),
helmet = require('helmet'),
compression = require('compression');


app = express();
app.use(helmet()); // protects server
app.use(compression()); //will compress all responses


app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.static("public"));
app.set('view engine', 'html');

app.get('/', function (req, res) {
res.sendFile(__dirname + '/index.html');
});



app.listen(3000, () => console.log('Listening on port 3000'));
91 changes: 91 additions & 0 deletions index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
<!--ideas to add:
https://codepen.io/NyX/pen/YPmVBb
-->

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<title>Digital Visualization Portfolio</title>

<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css"
integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous">
<link rel="stylesheet" href="public/css/titleStyle.css">
<link rel="stylesheet" href="public/css/homeStyle.css">
<link rel="stylesheet" href="public/css/allStyles.css">
</head>
<body>
<!--title-->
<div id="statsOutput"></div>
<div id="titleCanvas"></div>
<section class="titleSection">
<div class="titleDiv">This is a</div>
<div class="titleDiv">
<span>Digital Visualization Portfolio</span>
</div>
<div id="helpContainer">
<span id="typed"></span>
</div>
</section>
<!--content-->
<section class="homeContentSection">
<h2> Click on a Card to Learn More</h2>
<div class="row cgrid">
<div class="col-sm-6 px-0">
<div class="card">
<div class="card-body" onclick="document.location.href='public/d3Page.html';">
<h3>Data Visualization With D3</h3>
<!-- <h5 class="card-title">Special title treatment</h5>-->
<!-- <p class="card-text">With supporting text below as a natural lead-in to additional content.</p>-->
</div>
</div>
</div>
<div class="col-sm-6 px-0">
<div class="card">
<div class="card-body" onclick="document.location.href='public/2DVizPage.html';">
<h3>Sound Visualization With Canvas</h3>
<!-- <h5 class="card-title">Special title treatment</h5>-->
<!-- <p class="card-text">With supporting text below as a natural lead-in to additional content.</p>-->
</div>
</div>
</div>
</div>

<div class="row cgrid">
<div class="col-sm-6 px-0">
<div class="card">
<div class="card-body" onclick="document.location.href='public/3DVizPage.html';">
<h3>Sound Visualization With WebGL</h3>
</div>
</div>
</div>
<div class="col-sm-6 px-0">
<div class="card">
<div class="card-body" onclick="document.location.href='public/fractalPage.html';">
<h3>Generating Fractals With Canvas</h3>
</div>
</div>
</div>
</div>



</section>
<script src="https://cdn.jsdelivr.net/npm/typed.js@2.0.11"></script>
<script src="https://threejs.org/build/three.js"></script>
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/three.js/r68/three.min.js"></script>
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/dat-gui/0.5/dat.gui.min.js"></script>
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/stats.js/r11/Stats.min.js"></script>
<script src="https://code.jquery.com/jquery-3.3.1.slim.min.js"
integrity="sha384-q8i/X+965DzO0rT7abK41JStQIAqVgRVzpbzo5smXKp4YfRvH+8abtTE1Pi6jizo"
crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.7/umd/popper.min.js"
integrity="sha384-UO2eT0CpHqdSJQ6hJty5KVphtPhzWj9WO1clHTMGa3JDZwrnQq4sF86dIHNDz0W1"
crossorigin="anonymous"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/js/bootstrap.min.js"
integrity="sha384-JjSmVgyd0p3pXB1rRibZUAYoIIy6OrQ6VrjIEaFf/nJGzIxFDsf4x0xIM+B07jRM"
crossorigin="anonymous"></script>
<script type="text/javascript" src="public/js/newHomePage.js"></script>
</body>
</html>
26 changes: 26 additions & 0 deletions public/2DVizPage.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>MusicViz</title>
<link rel="stylesheet" href="css/allStyles.css">
<link rel="stylesheet" href="css/3DVizStyle.css">

</head>
<body>
<div id="content">
<audio id="audio" controls></audio>
<div id="out">
<label for="thefile" class="file"> Choose an audio file
<input type="file" id="thefile" accept="audio/*" />
</label>
</div>
<canvas id="oscillatingElement"></canvas>
</div>

<script src="http://threejs.org/build/three.min.js"></script>
<script type="module" src="js/singleLineVizScript.js"></script>
<!--<script type="text/javascript" src="js/musicVizScript.js"></script>-->

</body>
</html>
59 changes: 59 additions & 0 deletions public/3DVizPage.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>3D Music Visualization</title>
<link rel="stylesheet" href="css/allStyles.css">
<link rel="stylesheet" href="css/3DVizStyle.css">

<!-- vertex shader that passes through vertex position without modifying it-->
<script id="vertexShader" type="x-shader/x-vertex">
attribute vec2 position;
void main(void) {
gl_Position = vec4(position, 0, 1);
}
</script>

<!-- fragment shader that multiplies the output color wiht the spectrum intensity
to create some cool visuals. Also, 1 is scaled by 0.2 since most audio is in the first 20% of the spectrum texture-->
<!-- this shader is an adaptation of this one: https://www.shadertoy.com/view/XsXXDn-->
<script id="fragmentShader" type="x-shader/x-fragment">
precision mediump float;

uniform float time;
uniform vec2 resolution;
uniform sampler2D spectrum;

void main(void) {
vec3 c;
float z = 0.1 * time;
vec2 uv = gl_FragCoord.xy / resolution;
vec2 p = uv - 0.5;
p.x *= resolution.x / resolution.y;
float l = 0.5 * length(p);
for (int i = 0; i < 3; i++) {
z += 0.1;
uv += p / l * (sin(z) + 1.0) * abs(sin(l * 9.0 - z * 2.0));
c[i] = 0.01 / length(abs(mod(uv, 1.0) - 0.5));
}
float intensity = texture2D(spectrum, vec2(l, 0.5)).x;
gl_FragColor = vec4(c / l * intensity, time);
}
</script>

</head>
<body>
<div id="content">
<audio id="audio" controls></audio>
<div id="out">
<label for="thefile" class="file"> Choose an audio file
<input type="file" id="thefile" accept="audio/*" />
</label>
</div>
<canvas id="oscillatingElement"></canvas>
</div>

<script src="http://threejs.org/build/three.min.js"></script>
<script type="module" src="js/webGLShaders.js"></script>
</body>
</html>
9 changes: 9 additions & 0 deletions public/css/3DVizStyle.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
canvas {
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 100%;
background-color: black;
z-index: -1;
}
5 changes: 5 additions & 0 deletions public/css/allStyles.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
@import url('https://fonts.googleapis.com/css?family=Roboto:300');
body {
padding: 100px;
font-family: 'Roboto', monospace !important;
}
57 changes: 57 additions & 0 deletions public/css/d3Style.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
/*div {*/
/* width: 100px;*/
/* height: 100px;*/
/* !*position: absolute;*!*/
/* !*overflow: visible;*!*/
/*}*/

/*svg {*/
/* width: 50vw;*/
/* height: 20vh;*/
/*}*/

h1 {
text-align: center;
font-style: oblique;
padding-bottom: 70px;
}

div {
justify-content: center;
align-content: center;
}

td {
height: 600px;
}

svg {
height: 600px;
}


.vizTable {
width: 100%;
text-align: center;
vertical-align: center;
}

.visTitle {
padding-left: 10px;
padding-right: 10px;
max-width: 30vw;
background-color: gold;
}

.blue {
background-color: #5fbdf6;
}


#my_dataviz {
justify-self: right;
align-self: center;
text-align: right;
vertical-align: center;
min-width: 60vw;
}
Loading