HueNote

Dylan Freedman > Stanford Music 256-a > Project 2

HueNote is an audiovisual narrative that visually depicts a user's microphone input.

The program displays a raw signal as faded white rectangles and the FFT of the signal as a circle with peaks corresponding to the magnitude of that specific frequency after all frequencies have been collapsed into a single octave. The circles propogate outwards. The background color changes based on the predominant note.

The result is an immersive programmatic story that reacts fluidly to music, whistling, and experimentative noises.

Download

Mac OS X App (tested on Sierra): HueNote (2 MB)

If opening the app gives you an "Unidentified Developer

Building from source: Follow this link to download the huenote project.

Build instructions

From the root directory of the extracted huenote.zip file, run

make osx && ./HueNote

to build for Mac OS X. To build for other systems, try substituting "make osx" with "make linux-alsa", "make linux-jack", "make linux-pulse", "make cygwin", or "make win32" — depending on the desired target.

Motivation

In building this audiovisual experience, the notion of "note," or "pitch class," was important to me. Notes represent a fundamental building block in music theory but are not intuitive for computer programs to understand from raw audio.

To capture a simplified essence of what a note is computationally, I collapsed the frequency spectrum into a simple octave in logarithmic space, the same space in which humans interalize pitch.

It was difficult to visualize a line plot of these stacked pitches, as they changed rapidly and did not wrap around nicely. To fix this, I turned the line into a circle and radiated its history outwards so drastic changes could be visualized in real time.

To give the visualization a smooth and fluid feel, all changes happen with Zeno interpolation, that is a gradual percentage change increase towards a desired new state for every visual component every frame.

To improve pitch perception, I used a Hamming window with a large buffer size for the Fast Fourier Transform, which maps raw audio to frequencies and amplitudes and has sharper accuracy if more data is passed in. The raw signal is displayed with a shorter buffer size to show quick changes.

The jagged spiky arrows that soar across the visualization when music gets dramatic were unintentional, but it is the author's opinion that they look cool and add to the visual experience. They are the result of extremely high values in a particular frequency bin, which causes the jag in the circle to be so sharp that the circle no longer appears entirely connected.

Usage

Keyboard Usage: q: Quit a: Toggle display of raw signal in background s: Toggle display of circles z: Toggle fading inner circles c: Play a messed up Shephard tone x: Toggle collapsing fft to one octave b: Toggle showing colored background ?: Print this usage message Mouse controls: Left : Increase the spacing between circles Right: Decrease the spacing between circles Special control (only works on Mac OS X with Chrome installed): Cmd+Space c h r o m e Enter Cmd+T m a i l t o : f r e e d m a n d @ g m a i l . c o m: Emails author