Tuesday, September 23, 2008

Have You Listened to Your Program Today?

Assume you could plug a synthesizer into the internals of your computer's CPU, what would you hear? Noise and cacophony would be an intuitive first guess, and probably correct if you would hear the sound in real time. But if we slow down the CPU to approximately 3500 operations per second, we just about might assume the CPU's computation could contain all essential ingredients of music: rhytm and repetition caused by iteration, melodies caused by various mechanisms that change a value incrementally, and harmonies caused by some mechanism of excluding values, such as memory alignment.

Since I wasn't able to open a live CPU without making it slightly disfunctional in the process, I took Valgrind to the rescue: I wrote a little object code instrumentation tool, "skin" as Valgrind authors call them, which logs into a file all values I wish to follow when executing the software with which I wish to have a Close Encounter of the Third Kind. A little more code to generate sound and movies of the values.

Here's what one would here if the bits one to seven in the result of each addition and subtraction would indicate a very short ping. All those bits being zero corresponds to the barely audible 24.5 Hz, or the sub-contra G. Roughly ten highest notes are unfortunately ultrasound, but this was nevertheless the best representation I could come up with.

video

They say listening daily to your spouse improves your relationship. I wonder whether the same applies to programs.

56 Comments:

Anonymous Anonymous said...

That "sounds" just like a program I wrote

4:28 PM  
Anonymous Anonymous said...

my head hurtz >_<

5:09 PM  
Anonymous Anonymous said...

We used to do something ever so vaguely similar by putting radio next to, for example, a TRS-80 and listened to the "computer music"!

5:12 PM  
Blogger B said...

Fabulous project. I'd love to play with what you've done (any chance of posting your valgrind mods?)

Possible next steps: Take some of the repeating values and map them to percussion for a nice rhythm section. Merge increments smaller than x into single notes, with variants of merge patterns depending on various common scale formats.

6:14 PM  
Anonymous Anonymous said...

Very similar (except the MIDI customization) to the 1-bit souund output of most 80's Home Computers (mine was Sinclair ZX Spectrum 48K) :)

6:42 PM  
Blogger jah said...

Brilliant! I particularly liked the sound of GNU sed (from 5:17).

7:22 PM  
Blogger Wanker said...

Awesome stuff! Can you post your work? I would love to see how my website sounds. This may even make a decent presentation for management, if I could show them what good code vs bad code sounded like (if there is any noticeable difference).

7:51 PM  
Blogger shintaro_miyazaki said...

This is very very nice! I am working theoretically (humanities, PhD) in a similiar direction please have a lot at:
http://www.algorhythmics.net/en/?page_id=2

http://vimeo.com/6649750

Would be nice to hear some comments.
Is this project part of your PhD?

8:18 PM  
Blogger cessu said...

B, the valgrind mods are actually pretty trivial, but they are spread around a much larger piece of code which does very nontrivial things that we hope to capitalize academically first. Publishing the code will happen eventually, but will take some time.

Wanker, it is quite possible to distinguish individual algorithms from the sound. From additions one can easily distinguish loops and buffer pointer advancing, a corresponding aurealization of bitwise xors is excellent for finding e.g. cryptographic routines, and I have used both sound and visual representations to show memory references and cache misses they cause.

Though not included in the clips, the most pleasant sounds I've heard come from adding unique items to a linked list: the additions in record field offset computations cause nice trills which repeat at varying heights as the linked list is traversed, and if a new item is added, the theme grows by that note for the next traversal.

Shintaro, I will have to check your links later because the Fifth Fundamental Force of Nature, that exerted by a pillow to a tired head, is currently very strong.

10:52 PM  
Anonymous Anonymous said...

This mega hertz my ears.

10:58 PM  
Anonymous Anonymous said...

Oh ok. So Aphex Twin, basically.

11:16 PM  
Anonymous Anonymous said...

Nah, pnmtojpeg from 6:06 is the best. What a tragedy you can hear there.

11:23 PM  
Blogger eddieroger said...

You should seriously consider taking some of these and making a CD (or compilation of MP3s). It's actually pretty soothing to listen to, and I imagine it would make great background noise while programming. It'd also be neat to have someone ask what I'm listening to and reply, "diff." :-)

11:54 PM  
Anonymous Pazi said...

Some 20 years ago I had a program for Commodore 64 which played notes with SID based on CPU stack values. It sounded just the same as this when I was executing some Basic code! I think the assembly code for the program was published on some magazine back then. I have been looking for similar software for modern computers ever since...

12:00 AM  
Anonymous Anonymous said...

I think you can use this as a debugger! I am sure one can figure out a discordant note more easily than spotting a pattern in trace.

12:27 AM  
Blogger Harold Fowler said...

Oh wow, that actually makes sense dude. Well done!

RT
www.anon-web.int.tc

12:54 AM  
Blogger Texchanchan said...

I want the CD!

This reminds me of change-ringing.

12:56 AM  
Anonymous Anonymous said...

ever tried : sudo cat /dev/core > /dev/audio ?

12:58 AM  
Anonymous Anonymous said...

Sounds like a pin-ball machine being dragged behind a bus. Also, very interesting.

1:12 AM  
Blogger Luke said...

Any chance of a wav file? I'd love to load this into a sampler. (I could rip the audio from the video with some effort, but I'd like to get the exact waveforms if possible).

1:43 AM  
Blogger Jonathan said...

That is cool. Now you need to figure out how to make it play recognizable tunes like they use to do with the mainframes.

I think it would end up looking something like this http://www.youtube.com/watch?v=ipzR9bhei_o

2:23 AM  
Blogger Monklin said...

Reminds me of those old SEGA games I use to play as a kid..hmm.

2:39 AM  
Blogger James said...

A fun thing to do is slap a sound file header on a swap file and the play it. I like the NeXT header since it is just 28 bytes.

3:24 AM  
Anonymous Anonymous said...

@2, don't you mean, "my head Hertz"? ;)

also typo in the last para b4 the vid, should be "hear" instead of "here"

3:54 AM  
Blogger Plasmdude said...

That was amazing! A few months ago a did something similar where I wrote a Java app to play back the “audio” of log files. It was spooky how similar mine sounded to the post.

I just might dust off my old code for that project and bring it to the next step. I had pondered about being able to parse a body of text like a log file into a midi file. Then having some tool that could allow you to play back the log until you here something interesting. At which point you could read the log to learn more about the anomaly or in my case the stack trace.

4:01 AM  
Blogger Chris said...

In 1968 or so my senior Physics teacher put a radio next to an IBM 1130 on which he was running a handful of programs in sequence. He had written the programs specifically to play music: we listened to Christmas tunes.

4:30 AM  
Anonymous Anonymous said...

i applaud your geekiness, the cadence somehow conjured images of mickey mouse?!?!

5:00 AM  
Blogger evilDoug said...

I love this. Could you post an MP3? Seriously, I would listen to a whole album this"music".

5:29 AM  
Blogger BM said...

It reminds me to NES music :)

6:53 AM  
Anonymous Anonymous said...

Write a program that analyzes/responds to sound, then monitor the program operation as you've done here, feed its own sound back into the program. Will some kind of pattern or convergence form?

9:43 AM  
Blogger cessu said...

"Mega hertz my ears" gave away that this must have appeared on reddit...

9:49 AM  
Anonymous Anonymous said...

You should find some programs that produce nice sounds, and run them back to back, or together, or something, and make an entire album of music like this.

I'm sure if you picked the best ones, people would buy it. Some of that stuff was pretty sweet!

9:58 AM  
Blogger Dracc said...

I can fix your inaudible range issue. set the inaudible ranges to different instruments and render them aligned to middle C. Looks like you could have bass, melody and rhythm all at the same time from the same code. You might have to render each instrument one at a time and composite the tracks into a master, but it should be pretty simple.

10:33 AM  
Blogger Gav said...

Some of the sections sound like the CPU is very happy with what it is doing- like a bird singing; but other sections like GCC Driver sounds like the CPU is bored! Love it, nice work!

11:38 AM  
Blogger Ale said...

Cessu.... My congratulations. I really liked this but know it is just an start point of new kind of music composition. I'm not pretty good at programming (just a mere system administrator) but if you think I can help in anything, just tell me.

PS: B said somthing of using percussion. This could be achived via MIDI banks I think. And maybe associate pointers' progressions to trumpets or something (I'm talking with no idea of the real project)

Really brilliant is the messege. Good work.

11:55 AM  
Blogger Martin said...

The bit around 8:26 sounds like experimental techno jazz :)

Now I'm just wondering what the difference in sound between optimised and non-optimised code/data is. Does a bubble sort sound better than a quicksort despite being more inefficient? Does a bitmap of a drawing sound nicer than the equivalent jpeg?

12:36 PM  
Anonymous Anonymous said...

Sounds like Meshuggah, 1-bit style.

2:35 PM  
Anonymous Anonymous said...

I could listen to this all day long, seriously.

7:56 PM  
Blogger miss sabado said...

This is incredible! As a DJ-- I am very inspired. Can you give me a sample that I can spin this in my sets?
www.misssabado.com

9:19 PM  
Blogger kovan said...

That is just awesome.

11:34 PM  
Blogger cessu said...

Dear all, until a few days ago I had nearly forgotten I had a blog, and now I have lost track of all who have requested more of this... Umm what should I call it? Softrave? Muzhack? MegaHurtz?

Anyway, I just dusted off the few years old code snippets that perform the actual aurealization and during the weekend I should be able to create more than enough of it.

12:07 AM  
Anonymous Anonymous said...

Basically a slowed down cassette tape from a zx spectrum!

1:39 AM  
Anonymous Anonymous said...

It's amazingly reminiscent of 1960s science-fiction 'computer' noises. I owe the original Starship Enterprise an apology for thinking it just sounded dated.

3:06 AM  
Blogger deathvaps said...

excelente

4:06 AM  
Anonymous Anonymous said...

Where is da brain specialist? My brain hurts! Weird. Typ.o

6:20 AM  
Blogger jang said...

Great! I love the sounds you generated. They sound like the SFX in the old Sci-Fi movies around the 70s.

6:08 PM  
Blogger Daniel said...

Great idea, it reminds me of the time when i used a radio receiver to listen the different parts of an old (slow) PC I had. You could actually get a very good sense of what the system was up to if you had the audio in the background as you worked.

4:48 AM  
Blogger cessu said...

Note to those who seek hearing damage or internal enlightenment: here's 190 hours more of it.

1:37 AM  
Anonymous unodetantos said...

You could be a Matrix Operator

9:13 PM  
Anonymous Anonymous said...

This sort of thing goes back to the 1950s: The Music of CSIRAC.

2:36 AM  
Blogger Helena Electra said...

I love it! It's great to hear music EVERYWHERE :D

2:48 PM  
Blogger robermann said...

Very good! It seems your purposes are very similar to mine ones on the CodeSounding project (http://codesounding.org/indexeng.html): make it possible to hear the sound of execution of a software, assigning instruments and notes to programming statements (if, for, etc), so that the flowing of execution could be played as a flow of music.

9:37 PM  
Anonymous Anonymous said...

Great job, thank you

1:00 PM  
Blogger thermann said...

There is the research field "Auditory Display" and Sonification is the scientific approach to turn any data into sound. ICAD (www.icad.org) is the community. Program auralization is the term for sonifying programme code. There is a body of research on that, check for instance the CAITLIN project , or google for auralization.

10:01 PM  
Anonymous Anonymous said...

Great! Is it available the source cose?

1:01 AM  
Blogger Michael Chinen said...

Nice work. I would like to see the technique lower level audio stuff, without the translation to MIDI.
Please post the source code and I will post the result.

I'm also working on hacking open source programs to make them sound.
More info at www.fuckingsound.com

6:49 PM  

Post a Comment

<< Home