IMMORTAL HAND,
IMMORTAL EYE

Devlog 3: Artist Programming

The Immortal Hand, Immortal Eye demo is has been available for public consumption for a little while now. Some people have even played and enjoyed it, which is crazy. I'm very happy to finally be able to share a substantial part of this giant creative endeavor. This demo is the result of some two years of development; it's hard to pin down exactly where the project started, but the idea of "interactive fiction with a contextual soundtrack" first appears in my notes in July 2023:

built in twine. less of a game, more of an interactive text story. takes the form of various web pages in "levels" representing captures taken on a given date. oh and its also a concept album: background music plays in a queue, songs are added/removed/shifted depending on the order you view pages in

It did not end up being built in Twine, as I've discussed, and will discuss more later in this post. But, in general, this is the idea I started with, and which I refined, adapted, advanced, and *accomplished.* I'm not trying to be a braggart — please do not envy me — I'm just commentating on the process of turning the basic and vague into the actually-existent.

So here's a Tumblr post whose timely appearance on my dashboard I will gladly attribute to cosmic serendipity:

Historically, the high technical entry barriers to publishing one's own work has left indie video games vulnerable to allegations of "programmer art": that is, a tendency for the audiovisuals to be created by people who don't know what they're doing because "real" artists are barred from the medium by lack of technical know-how.

In this sense, it's heartening to see that coding one's own game has become sufficiently accessible in the past decade or so that we're finally seeing the fruits of the natural yet historically largely unattested-to counterpart to programmer art: artist programming.
(Prokopetz)

I will admit, straight up, that I am not "good" at programming. My code is so messy and patched-over, it may constitute cruelty against computers. Everything is a sparse array. Who needs a strongly typed language when you can just shove some parseInt()s or toString()s in there as necessary? I always use double quotes, except for when I use single quotes, except for when I need to use graves. These statements are neither pride in my ignorance nor self-deprecation; I'm saying these things to prove the point that you can do a LOT of things if you're just willing to do them badly.

The truth is that everything in IHIE, as well as everything I've ever made, is "artist art." It's artist writing, and artist music, and artist design, because my goal is to make art. I have not found it helpful to believe there's some standard of technical proficiency I need to meet before I'm allowed to make the stuff I want to make. That mindset does not result in making very much art at all. So I'm perfectly fine working with blunt tools; a bad solution I can actually accomplish is worth infinitely more to me than a good one I can't. One time I couldn't figure out how to open a plastic container of cat litter, so I stabbed a hole in the lid with garden shears. Fuck it. It worked.

This is not to say that slapdashery is inherently virtuous, or something. There's no reason to do things barely-competently if you can do them really well for the same price (of money, time, mental energy,) and you definitely should not take this approach when, like, building bridges, or managing a children's hospital. A badly-made bridge is, in fact, worse than no bridge at all, because it imperils the people who expect it not to collapse and drive over it on that assumption. A children's hospital that doesn't make an effort to keep its internal processes clear and efficient is going to result in mistakes that poison children. But the worst thing a piece of art can do is not exist.

(Okay, nuance: art that acts as justification, explanation, or encouragement for abusing other people, politically or individually, Is Bad, and would indeed be better off not existing in the first place. Those who really want to be the next William Luther Pierce but are afraid they're just not good enough writers should absolutely give up on their dreams and never try to make anything at all.)

If I may wax philosophical — by which I mean go from an already-waxing gibbous to a full moon — the creation of stories is one of humanity's neatest tricks. A story is an idea, generally imprinted in some physically-accessible medium, but which fundamentally can be infinitely reproduced in the minds of those who experience it. It's a thing that can be made to exist through the power of the mind alone, and the mind is its native habitat, where it "truly" exists. Art is a way of transmitting ideas between people through intermediary senses; reading a book, listening to a song, playing a game, are all ways of programming a story into your mind. You read some words, and an idea comes into existence, and it remains even if you forget the exact words (the exact notes, the exact sequence of keys pressed.) Those words may be arranged in aesthetically pleasing or displeasing ways, they may transmit the idea with more or less precision, and the medium through which a story is told does affect its meaning — (the same molecules of water seem different in a cup versus a vase versus a fishtank) — but they are, ultimately, subservient to the intangible. They're just carriers.

Got it? Great and/or that's fine. We're going to talk about the music now.

IHIE has a conceptually simple adaptive music system, and it works pretty much exactly as I envisioned it in those halcyon days of predev. Which is not to say that it was simple to build! Figuring out how to make a high-level idea like "music plays depending on where you are in the story" work in code is a series of puzzles, the first of which is semantic: what do I actually want to happen? Starting from absolutely nothing, what do I need to construct?

  1. A way for music to play at all. This is an <audio> element whose source points to an .mp3 in the project's file web.
  2. A way for the music to keep playing without being constantly interrupted by pages reloading. I put the <audio> element in the container around the actual page content, in the same place as the menu.
  3. Music that continues automatically. I give the <audio> an onended property with a function, musicProgress(), that activates every time a track ends to immediately start playing the next one.
  4. An ordered playlist of tracks. I make an array called playlist and go through its contents in order, making sure the delete whatever was just played, so the next track is always at the first position. For simplicity, all tracks are referred to with ID numbers, and stored like that (ie "26.mp3") in the files. This means that I can set the source of the <audio> element to "/music/"+ID+".mp3" to get the right track, instead of having to make another array to correlate ID numbers and file names.
  5. A way to update the playlist dynamically. Visiting a new page triggers a function to queue a track by pushing it to the playlist array. I can set a priority argument that determines if this new track should go to the front or back of the line, or fade out the current track and start playing the new one immediately.
  6. Music that keeps going even if it runs out of specifically ordered tracks. For this, there's a second array that I call "explore," for reasons I don't remember. Explore contains tracks that are suitable for playing in the lull between tracks queued up by story pages, and songs can be added to or removed from it by a similar mechanism. If playlist runs empty, the music progression function instead picks randomly from explore ...
  7. A way to avoid playing the same song over and over. ... except not totally randomly, because there's a third array that stores the time since each track has been played. Whenever a song finishes playing, its sincePlayed time is set to 0, and all other track IDs' is increased by the length of the song that just played. When musicProgress() draws from explore, it first filters out all tracks that have a sincePlayed time of less than ten minutes, and only chooses from the ones remaining. This ensures that there will always be at least two or three more songs before any particular one is played again.
  8. A page that displays current track information for the reader. Every song in IHIE has a name and track art, which I want the reader to be able to see without it being intrusive or distracting. The music tab in the top menu accomplishes this with a simple display of a couple HTML elements. The name of the current track is pulled from an array of correlated titles, (just like I didn't do in step 4,) and the track art is stored as "/sigil/"+ID+".png" so it can be referenced as easily as the tracks themselves. Something I plan to add in the future is a submenu tab to see a list of all the tracks the reader has encountered so far, in case they didn't check the menu while one was playing.

These seven steps achieve all the things necessary for my vision of the music system. You could copy them and build your own, if you wanted! I'm not going to say any of that shit about "creative theft." Please make a hypernovel. I'm begging you. Please make a janky fucking javascript app to tell a multimedia narrative to your exact specifications. It's good for you. It is necessary for your health. I'm a doctor. note 1

By the way, there's a fun side effect to the way this engine works. If I mess up in the code, it's possible for the source-setting in step 4 to try to load something that isn't a real track ID. If the <audio> element can't find the file it's been told to get, it just doesn't play anything, and the music stops forever. To deal with this in testing, I made some error handler tracks to cover for invalid track IDs, by storing them in the game files as things like "undefined.mp3 ." The published version of the demo does not, as far as I know, have any remaining bugs that would cause these tracks to be played. Maybe I'll eventually release them some other way, or maybe they will be ultimate hidden forever... at least until I try to change something and accidentally break the music system again? It's not unlikely.

That's all I have to say about actual development, and the philosophical babble was frontloaded this time. So, if you haven't already played through the demo (or mentally committed to doing so later) but are still reading the devlog for some reason, I guess I'll entreat you once more to check it out. It is free, though anyone who's willing to pay for an unfinished product is welcome to do so and much appreciated. This is technically my job now, and it's up to you how much I get paid for it.

back to IHIE lander | back to main site

note 1: this is not true.