Markov Contact

Today, I'm releasing Markov Contact, an experiment in dynamic music visualisation. I wanted to take some of the concepts from my earlier Markov Technology a lot further, particularly in terms of the complexity of the visualisations and the level of interactivity. The rest of this post will be talking about the process of making it, so you may want to go try it out first.

I started with the idea for just doing a successor to Markov Technology, a project where I made an endless randomly arranged version of a somewhat repetitive song using a Markov chain. I'd been listening to First Contact by Keiji Yamagishi of Ninja Gaiden fame while working sometimes, and I found myself often wishing I had an endless version. I figured it would be a lot easier the second time around because most of the lessons I learned the hard way could be avoided, However, as I got into it I found some important differences and opportunities to do something more ambitious.

My process for the music was almost identical: first, listen to the song a few times to get an idea of the overall structure, then go through and mark all the sections (verse 1, chorus, bridge etc), then further split each section into subsections (usually a bar or two in length) to make the Markov nodes. During this process I tend to shuffle the subsections around to get a feeling for good points to link them and to find duplicate sections. Eventually I then render the stems out and move on to the code (I use Reaper but any multitrack audio editor would do).

This was the first hint that the project would have to change with the music, because its structure was profoundly different. The original song, Tricka Technology, was very repetitive and self-similar, whereas First Contact only really had redundancy in the chorus-like sections and their interlinks with the rest of the song. So while I could put a lot of the structure into the Markov chain the first time around, this time I began thinking of other ways to represent it.

So I started thinking about space. The song is called First Contact, after all, and several of the sections felt to me like they could be imagined as parts of a journey through space in search of life. The start slowly increases in intensity, which feels like a launch countdown. Then there's a sudden explosive transition, like a liftoff, and then a kind of casual cruising vibe. Between that are sections that could easily be stops along the journey. Finally, what really tied it together for me was when I realised that the connections between the Markov nodes bore a lot of resemblance to constellations. From that point, the space metaphor was all I could think about.

I also wanted to rework the beat visualisation from the the original. The original had a kind of flickering effect, but I wanted something with a lot more movement to it. Of course, if the nodes are constellations, the obvious choice would be to use a star. I originally intended to do a standard 5-pointed star, but after some experimentation 9 points was my favourite. I collapsed all the inactive nodes into simple dots, which made the constellation pattern very obvious and added to the contrast between active nodes and inactive ones. Finally, I realised that stars look very boring when all they do is change size, so I made them rock back and forward to the beat as well.

My next step was to rearrange the nodes into recognisable shapes that I could treat as constellations. This took ages, because I was trying to achieve several things at once: I wanted the nodes to reflect the structure of the music, to look nice, to represent some space-themed shape, and to be fairly regular. I spent a lot of time drawing shapes on a whiteboard, and even longer rearranging the shapes on the screen. To help with this, I added a debug tool that would let me drag the nodes around and save their positions.

At that point, the visual metaphor was speaking loud and clear, but it felt like there was something missing. Because of the nature of the song, the chorus section with the spaceship was quite interconnected, but the other sections were all linear. During those linear sections, there wasn't much to look at. Additionally, I wanted to add some sense of depth. The obvious choice was an animated starfield, and the more I thought about it the more it was clear that it should animate with the mood of the music, reinforcing the same imagery used in the constellations.

Finally, the constellations themselves needed a bit more fleshing out, so I sketched some constellation-style line art to go underneath them. My drawing ability was not strictly up to this task, but luckily most constellations are simple line art so I think it came out alright. Finally, with all the pieces together, the whole thing looked exactly the way I imagined it. The space journey was very evident, and most importantly the music felt fun and alive.

There was also a fair bit of technical work in redesigning the code to handle all these new features and track the music reliably. If you're interested in that, you can check out the code on GitHub. Otherwise, enjoy, and check out some more of Keiji Yamagishi's work – he's truly the master of chiptunes, and I consider this work to be a tribute to his.