KrazyDad » Blog Archive » Visual Harmony

« Food Stuffs

A φ Fractal »

Visual Harmony

This weekend I’ve been playing, once again, with the ideas of experimental film pioneer John Whitney, using both graphics and audio. While Whitney was interested in turning musical ideas into motion graphics, I’m doing the inverse — turning one of his key animation ideas back into music.

Whitney made a number of films based around the simple idea of harmonic relationships. Above is a visual example of one his ideas that I implemented in Flash.

Here’s Whitney talking about this idea:

An early intuition about how to control total dynamics led me to activate all graphic elements through a motion function that advances each element differentially. For example, if one element were set to move at a given rate, the next element might be moved at two times that rate. Then the third would move at three times that rate and so on. Each element would move at a different rate and in a different direction within the field of action. So long as all elements obey a rule of direction and rate, and none drifts aimlessly or randomly, then pattern configurations form and reform. This is harmonic resonance, and it echoes musical harmony, stated in explicit terms. I tried this procedure in several films, and was gratified by the consistency of the confirmation it demonstrated.
– John Whitney, “Digital Harmony”, pp. 38

In this movie, each of the 48 dots is moving in a circle. Each of the dots is on a 3 minute cycle. At the end of 3 minutes, the outermost dot will have moved around the circle once (this dot represents the first harmonic or fundamental). The next dot will have moved around the circle twice (representing the second harmonic). The next dot three times, and so on. The innermost dot moves around the circle 48 times.

Now, imagine these dots are raised bumps on a disc which is controlling a music box, with each bump triggering a note when it passes the zero degree line (a line extending from the center to the east). The result would look and sound something like this three minute looping movie, which I built in Flash:

Whitney Music Box (chromatic version)

In this particular implementation, the music box has the bars for a 48-note chromatic scale. The pattern causes chords, ascending and descending runs and melodies to be heard.

You may notice some interesting links between the visuals and the audio, especially if you are a musician. For example, when the pattern forms a 3-arm starfish, the chords you are hearing are diminished chords, which consist of minor thirds, an interval in which the notes are 3 chromatic steps apart. The chords you hear always bear this type of relationship to the pattern you are seeing, consisting of intervals which match the arrangement of arms.

I generated the audio using my sound synthesis program, Syd, which provides a very elegant way to accomplish this kind of thing. Unfortunately, Syd is horribly supported these days (I haven’t updated the Macintosh version since before OS X). Here’s a sample Syd patch which was used to generate the audio (I actually used 3 similar patches and mixed the results together).

Update:

Here’s are some other variations. The first few assign pitches based on the harmonic series, rather than the chromatic scale (thus the dot that represents the 10th harmonic actually sounds like the 10th harmonic). In these variations, the “chords” sound like single notes with a buzzier timbre, because when you combine harmonics, you are performing a kind of additive synthesis.

Whitney Music Box (harmonic version - fundamental = 55hz)

Whitney Music Box (harmonic version - fundamental = 20hz)

The above variation uses a lower fundamental which is close to the limit of human pitch perception.

Whitney Music Box (harmonic version - fundamental = 48hz, reversed)

The above variation reverses the assignment of the pitches, so that the lower fundamentals are heard more frequently.

Whitney Music Box (harmonic version - Palindrome)

The above variation uses both forward and reverse pitch mappings at the same time, to create a palindrome effect. The oscillators with the reverse pitch mapping are 180 degrees out of phase, which causes some nodes to cancel each other out, and creates some mildly interesting filtering effects.

Whitney Music Box (chromatic version - 88 tines)

The above variation uses 88 tines in a chromatic arrangement with alternating ascending and descending notes. Because there are more bumps on the wheel, there are more notes in the sequence.

Update 4/25/06:

I’ve generated a few more variations, which you can see here. I’ve begun using STK (the synthesis toolkit) instead of Syd to generate the audio, since this is more commonly available software that will allow other folks to experiment with this type of system. I generate the audio score using a Perl script which outputs note events in the SKINI format. I use the command-line utility calico to render the audio.

Update 4/26/06:

Nyquist, the Lisp-based music language, also provides an elegant way to generate this type of music. Here’s a sample nyquist score that I used to generate variation 11.

Update 5/2/06:

www.krazydad.com/blog/2006/05/02/a-fractal/In this new post, I describe the nature of the star-fish patterns in more detail.

This entry was posted on Sunday, April 23rd, 2006 at 4:25 pm and is filed under Jim's Projects. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

45 Responses to “Visual Harmony”

chadm Says:
April 25th, 2006 at 7:01 pm
I think you may enjoy the interactive musical art/toy by Toshio Iwai called Electroplankton. You will need a Nintendo DS for it.
jbum Says:
April 25th, 2006 at 7:39 pm
Thanks Chad. I’ve heard good things about Electroplankton, but haven’t had a chance to play with it yet (due to lack of said Nintendo DS…).

Regards,

- Jim
jabarton Says:
April 30th, 2006 at 7:29 pm
amaaaaazing. i need to read this book perhaps. can you do a subharmonic version?
KrazyDad » Blog Archive » A φ Fractal Says:
May 2nd, 2006 at 10:58 am
[…] « Visual Harmony […]
loreto martin » Blog Archive » Whitney Music Box Says:
May 14th, 2006 at 8:08 am
[…] John Whitney, Arabesque, 1975, […]
KrazyDad » Blog Archive » I’ve been dugg! Says:
May 15th, 2006 at 4:49 am
[…] About the Whitney Music Box […]
oxenrig Says:
May 15th, 2006 at 9:52 pm
Dude…. this thing’s driving me insane…. like it makes perfect sense, but at the same time…. it doesn’t. It’s like an acid trip…. i’m seeing sounds & hearing colors…. is there something wrong with me?
Peripatetic Circumambulant » Whitney music box Says:
May 15th, 2006 at 10:31 pm
[…] Whitney music box is amazing. I found it via this post on Digg. This post by its maker explains it. I have not bought into music of the spheres, but many of them are strangely satisfying. […]
tahome Says:
May 16th, 2006 at 8:18 am
Nice! However, animation and sound go out of sync after one period (~2min.) and the colored balls stick to a straight line. I was trying var.9 in Mozialla Firefox on MacOS X (but I doubt it works right on Windows, can’t check from here now, sorry)

HTH,
–th
jbum Says:
May 16th, 2006 at 10:01 am
tahome: Yeah, I know. The variations with more than 48 notes are particularly problematic with the sync.

There are two problems:

1) As time passes, audio is increasingly out of sync with animation - due to differences in playback speed, and imprecise loop point.

2) After 1 revolution, ball motion starts to acumulate errors, starting in the center and expanding outwards. I still haven’t figured out why the balls get out of alignment after 1 revolution, but I assume it’s some kind of cumulative floating point error. I haven’t spotted how this might occur in the code yet.

The interactive versions I have been working on (which generate MIDI events in real-time) don’t suffer from either of these problems.
jbum Says:
May 17th, 2006 at 11:53 am
Okay, error #2 is fixed in all but variation 7. That one’ll get fixed later today.
ianmcarey Says:
May 17th, 2006 at 1:48 pm
Hi, I’m really enjoying all the variations. As a musician, a couple of really intesting things jump out at me when listening to them–first, they all make musical sense intuitively, probably because the ear detects the step-based patterns (whether chromatic or harmonic) even when there are mulitple iterations going simultaneously at different speeds. Second, even though the function is mathematical, they all come across as having a really great sense of pacing–that is, a balance between easily noticeable “runs” giving way to random-sounding arpeggios broken up by those giant chords, giving way gradually to groups of runs. I think serial composers would kill for these kinds of effects.

I’d be curious to see what a version would sound like with a more consonant scale, such as diatonic or pentatonic–or even using a recognizable melody (so that not everything goes by steps).
jbum Says:
May 17th, 2006 at 2:19 pm
ianmcary:

Cool! A comment of substance. I don’t see too many of those here in the netherreaches of the blogosphere.

I’m going to take you up on your suggestion and generate a few more variations. I was thinking the same thing (about using a scale) when I did the prime/non-prime variations.

How about one which uses the regular 48-note orbits, but uses a blues scale? I’ll do that next.

There’s lots of ways to do a blues scale, how about

C Eb E F F# G Bb C

Maybe I should throw in the D?

If you have any other suggestions, let me know, and I’ll throw ‘em in. I kind of like Lydian and Phygian modes. I haven’t done pentatonic, cause I thought it would reduce harmonic interest..

As far as a recognizeable melody, it would be good to use something that has a fairly regular rhythm, and somewhow fits with the theme. Mebbe “Across the Universe?” Naw… too uniform…
jbum Says:
May 17th, 2006 at 2:54 pm
Okay, ask and ye shall receive.

I’ve added this directory:

www.coverpop.com/whitney/midi/

Which contains some midi experiments using scales (the blues ones sound pretty nifty. okay, they all sound nifty) and the perl script used to generate them.

Enjoy!

These are just music only, but have the same pattern as the 48 and 88 note sequences.
The Music Gush » Blog Archive » The Whitney Music Box Says:
May 17th, 2006 at 3:24 pm
[…] There’s a blog entry with further explanation here, but my understanding of it is still quite limited. I thought it might appeal to the musicians (and fans of Electroplankton, I agree with the commenter). It’s somewhat hypnotic . . . […]
ianmcarey Says:
May 17th, 2006 at 6:22 pm
Wow, that was fast!

A couple of thoughts: the “Major88″ sounded exactly like I’d hoped it would–the intervals jump out even clearer than in the chromatic versions. Fantastic.

You probably noticed, but any diatonic mode, including lydian or aeolian (natural minor) is going to end up sounding the same when put through that pattern. The harmonic minor one is great, though, and definitely has a totally different feel to it.

The blues scale ones are interesting–the bluesy tonality really comes out in the first version, while I think it’s harder to hear in the “88″ version, since there’s so much happening at once.

Some other possibilities you could try would include the whole tone or diminished (half-step, whole-step, half, whole, etc.) scales; as for melodies, I’d be curious to see what would happen with some Bach fugue subjects. (You can find a nifty guide to all the Well-Tempered Clavier fugue subjects here.)

Another possibility (which would probably just sound like a mess) would be some tone rows (12-note patterns where each note occurs once). Adding a rhythmic element (so not all notes are of equal length) would definitely be interesting, too.

Thanks for indulging me on those!
jbum Says:
May 17th, 2006 at 8:13 pm
Some more midi files added that use the prime and non-prime methods to add rhythm. “primeblues” “nonprimemajor” etc. More stuff to come…
holotone.net Says:
May 18th, 2006 at 3:45 pm
[…] From the developer’s blog: This weekend I’ve been playing, once again, with the ideas of experimental film pioneer John Whitney, using both graphics and audio. While Whitney was interested in turning musical ideas into motion graphics, I’m doing the inverse — turning one of his key animation ideas back into music. […]
jbum Says:
May 18th, 2006 at 3:53 pm
I’m noticing that the “major” version (in the midi collection mentioned above) seems to be going from I to V near the middle of the piece). It would be nice
to find a way to exploit this or accentuate this more.
Oh Internet » Blog Archive » Harmonics Says:
May 20th, 2006 at 9:27 am
[…] The fantastically named Jim Bumgardner made this neat page that takes animation ideas from some of John Whiney’s films on harmonic relationships and uses them to create music. More details about the techniques can be found here. […]
White Man Stew » Music Box Variations Says:
May 20th, 2006 at 10:01 am
[…] The “Whitney Music Box” (flash, sound) is described as “a musical realization of the motion graphics of John Whitney as described in his book Digital Harmony.” […]
pictosonic Says:
May 22nd, 2006 at 10:58 am
This is really interesting. I made an attempt in Java, probably less aesthetic, based on spectral analysis.

Here :
pictosonic.free.fr/blog/index.php?2006/05/18/17-analyse-spectrale-animee-avec-des-fuseaux
More here with Flash videos :
pictosonic.free.fr/blog/index.php?Animations

I have problems determining accuratly notes from frequencies using FFT) extracted from .WAV files.
alwaysBETA » Beta Bites: Issue 13 Says:
May 26th, 2006 at 4:37 pm
[…] Grant - whitney music box - Inspired by experimental film pioneer John Whitney, Jim Bumgardner made this mix of animation, mathematics and animation that helps show how harmonics and chords are related. You might remember this guy from his Flickr Colr Pickr. There’s more information available at this blog post. […]
Articles and Texticles / Whitney Music Box on Coverpop.com Says:
May 29th, 2006 at 7:51 pm
[…] (Quote:) A musical realization of the motion graphics of john whitney as described in his book “digital harmony”. Here’s a description. […]
Bibi's box Says:
May 31st, 2006 at 8:57 am
Whitney Music Box

Whitney Music Box: a musical realization of the motion graphics of John Whitney as described in his book “Digital Harmony”.The thirteen variations of the music box are the result of Whitney’s idea of harmonic relationships, based on audio, graphics…
plastron Says:
June 17th, 2006 at 11:38 am
Very nice work! Having read Whitney’s book, it’s great to be able to see his ideas fleshed out, especially since his films are hard to find and often are not as ’smooth’ looking as this. Do you think you will ever make a standlone, downloadable version of these examples (or is that possible)? I would love to have a “hard copy” so to speak, especially for showing to students of abstract film.
jbum Says:
June 17th, 2006 at 12:19 pm
Plastron: If you right-click the ’swf’ link at the bottom right of the Whitney Music Box page, you can download the individual flash movies.
The Rare/Often » Blog Archive » Whitney Music Box Says:
July 4th, 2006 at 2:10 pm
[…] Whitney Music Box - YESSSSSSS!!!!! This is kick ass too. Read more about this at Krazydad’s blod post here. […]
KrazyDad » Blog Archive » Favorite Posts Says:
July 7th, 2006 at 4:16 pm
[…] Visual Harmony […]
misterwarwick Says:
July 10th, 2006 at 12:11 am
cool stuff!

What is the tuning?

It should be pythagorean. Unfortunately, it sounds like standard even tempered tuning.

But it’s still cool beans.

best,

HW
Multiple Extase » Visuelle Harmonie Says:
July 10th, 2006 at 4:09 pm
[…] Jim Bumgardner morpht eine Hauptszene aus dem Experimentalfilm “Digital Harmony” von John Whitney zurück zur Musik. […]
Whitney Music Box at Junk Musik Says:
July 18th, 2006 at 3:18 pm
[…] Read more […]
Electric Spork Says:
October 8th, 2006 at 12:23 am
Whitney Music Box, quite possibly the coolest use of Flash to date…

Strangely hypnotic, fascinating, and even a little scary all at the same time, the Whitney Music Box is described as:
A musical realization of the motion graphics of John Whitney as described in his book “Digital Harmony”.
It’s a bit …
roclar.net Says:
October 9th, 2006 at 5:34 pm
[…] As jbum explains on his blog KrazyDad: “This weekend I’ve been playing, once again, with the ideas of experimental film pioneer John Whitney, using both graphics and audio. While Whitney was interested in turning musical ideas into motion graphics, I’m doing the inverse — turning one of his key animation ideas back into music.” The result is curiously intriguing. Watching the multi-colored balls going around a center point playing music while creating interesting patterns can be very soothing. […]
ChrisPhoenix Says:
October 11th, 2006 at 1:26 am
This is very cool. Very, very cool.

I believe it’s impossible for any artist in any medium to do a picture of the Mandelbrot set that’s as good as the original. Likewise, I suspect that atonal and other modern “classical” composers cannot do as well as this music. The Minute Waltz (var. 11) reminds me strongly of Mahler after the first few seconds–but I hate Mahler and I like the Minute Waltz. In fact, atonal and modern “classical” music has always sounded to me exactly like an orchestra tuning up, but I like almost all your variations. (I was raised on Bach &c, taken to concerts before I could talk–and stayed silent for hours, listening. I love classical, just not “classical.”)

I find myself wanting to rotate the white line a bit, so that it would catch the dots before or after they line up. Likewise, on the stereo versions, the odd-numbered lineups hit the right-hand line but never the left-hand line; there’s an imbalance there but I don’t know what to do about it.

Have you tried moving the dots at speeds corresponding to irrational numbers, e.g. the square roots of primes? That should produce music that never repeats, and never forms exactly simultaneous chords, and has no sensible meter. You can get approximately-linear multiples from ~2 to ~48 by taking the square roots of the first 48 primes, subtracting 0.8, and multiplying by 3.4. See e.g. primes.utm.edu/lists/small/10000.txt for primes. You can get more-linear multiples by taking primes from the middle of the list and likewise adjusting the range to 2-48.

If you do this, please email me–I’d love to know how it sounds.

Thanks,
Chris
jbum Says:
October 11th, 2006 at 9:46 am
Hi Chris.

I used to have a composition teacher, (Alan Chaplin at CalArts) who disliked
it when his students would fill their pieces with long chromatic runs. He disliked it because the runs are not really composed - there is no choice
involved by the composer - it is just a kind of automatic fill of notes.

So, although I like the music produced by the Whitney Music Box, at some
level, I can hear Alan Chaplin behind my shoulder clicking his tongue in
annoyance, because these pieces are not really composed - they are
fully automatic.

At the same time, the content of the pieces is not as overly familiar to us as a chromatic scale, so perhaps that is some small consolation.

You suggested some interesting modifications. Thus far, I have found that the more I perturb the rules (trying to get more complexity out of the system), the music tends to get less interesting, although visually, the dots get more interesting. I believe this is because our optical circuitry is more equipped to handle new information. “The eye likes surprise, and the ear likes comfort.”

You’ll find some additional mods here:

www.coverpop.com/whitney/MIDI/

In which I’ve simply changed the scales, and here:

www.coverpop.com/whitney/index.php?var=v18

Where’ve I’ve altered the ratios of the speeds of the dots to get a non
repeating pattern. The speed ratios are based on divisions of powers of 2
by 12 - the same ratios that appear between pitches in a chromatic scale.

Alan Chaplin clucks his tongue yet again!

I’ll try some of your other ideas when I get a chance…
whitney music box « Ballinger’s Professional Blog Says:
October 11th, 2006 at 10:01 pm
[…] whitney music box A very neat site demonstrating interesting visual and musical ideas. Fun to play with! From the author: “This weekend I’ve been playing, once again, with the ideas of experimental film pioneer John Whitney, using both graphics and audio. While Whitney was interested in turning musical ideas into motion graphics, I’m doing the inverse — turning one of his key animation ideas back into music.” His blog (www.krazydad.com/blog/2006/04/23/visual-harmony/) has more information and more examples. […]
mathseive Says:
October 17th, 2006 at 7:06 am
This is a completely amazingly little project. I am fascinated. I’d like to have a go at doing something similar myself in Max/MSP (whether i do get round to it is another matter!).
Do you have any links to videos of John Whitney’s works? i’d really like to see some more!
Thanks!
jbum Says:
October 17th, 2006 at 11:05 am
I’ve saw some of his films when I attended CalArts, and I saw some good ones at the “Visual Music” exhibition which ran recently in D.C. and L.A.

Unfortunately, these films are not available on DVD. You can see a few poor quality stills of them online if you google “Whitney” and “Visual Music”.
Fiat Lux! » Archivo del weblog » Whitney Music Box, Musica con colores Says:
October 23rd, 2006 at 3:00 pm
[…] Cada vez que un punto de color pasa por la linea del medio reproduce un tono dependiendo del color que sea el punto. Al mezclarse el movimiento de los puntos que giran en espiral se produce musica. Hay diferentes variaciones y tipos de notas para disfrutar. En la pagina de Krazydad podemos ver como surgio esta idea. Un buen alucin para disfrutarlo en pantalla completa. […]
fake artist » Whitney music box Says:
October 26th, 2006 at 11:42 pm
[…] La petite fascination de la matinée. des petits films en flash inspiré du travail de John Whitney. Explication ici en anglais et cliquez sur ce lien pour voir les animations. post-it : pensez à utiliser des sons dans mon futur projet net-art. […]
Whitney music box at monkeyPi Says:
October 31st, 2006 at 3:39 pm
[…] The Whitney Music Box About the WMB […]
Hola, mundo. » Blog Archive » On ChucK, Terry Riley, Jim Bumgardner and emulating old masters through code Says:
November 4th, 2006 at 12:19 pm
[…] Having a look at the programmer’s website (ah, curiosity) I’ve discovered why the url sounded so familiar: He’s also the man behind whitney music box, a well known series of animations inspired by the work of john whitney, as described in his book Digital Harmony: On the Complementarity of Music and Visual Art. The post explaining the work is worth a look. […]
Tai Mingshen » The Whitney Music Box Says:
December 27th, 2006 at 2:15 pm
[…] Jim Bumgardner has created a musical realization of one of experimental film pioneer John Whitney’s ideas on harmonic resonance, as described in his book Digital Harmony. You’ll find the technical details on Jim’s blog KrazyDad. I picked this up from The J-Walk Blog and see that it is also featured on digg. […]
mlarson.org » Says:
March 20th, 2007 at 9:04 am
[…] I’m pretty much fascinated with the Whitney Music Box, which explores some of the ideas in John Whitney’s 1959 book Digital Harmony. I like the microtonal variation, and the sine wave harmonics are cool because harmonics are inherently cool. Jim Bumgardner wrote more about this project and some of the mathematics of the patterns in his blog. […]

You must be logged in to post a comment.

Visual Harmony

45 Responses to “Visual Harmony”

Leave a Reply