I've been thinking a bit about software design, decisions and, for certain reasons, puzzles. One thing that's difficult to explain is how to make good decisions when designing software, or how to choose between one implementation strategy and another. Like many decision-making processes, it's full of heuristics that are hard to explain, but thanks to the indomitable efforts of AI researchers, some of them have already been described for us.
Constraint Satisfaction Problems, where you simultaneously try to satisfy a bunch of different requirements by changing values around, are a particularly relevant field of research. They mostly involve situations that are too complex to have closed-form mathematical solutions, and too large in scale to simply brute force your way through. Instead, you have to get a bit clever with how you choose which values to try first. In other words, you use heuristics.
Two important but simple heuristics are called most constrained and least constraining. That is, you pick the variable with the least options, and you set it to the value that leaves the most options for other values. These two heuristics together can produce a surprising improvement, even with an otherwise naive algorithm. The reason they work is the variable with the least options is going to be the hardest to satisfy, and the value for that variable that leaves the most options is going to give us the highest chance of success.
I think those heuristics are equally important in software design. You want to start with the biggest, hardest requirements first, because once you've got them out of the way everything else gets easier. But you also want to take care to try to solve those requirements in a way that takes the fewest decisions off the table for the rest of your code. I don't mean that you should write everything in a super-overly-abstract way, just that you should avoid doing things in a way that cuts off options unnecessarily. You can be flexible by doing less, which is preferable to being flexible by doing more.
There are some interesting implications about decisions beyond software, too. It's not unreasonable to view your life as a very complex Constraint Satisfaction Problem. If that's the case, it could be worth thinking about which decisions are the most limited, and whether you're making those decisions in a way that maximises your remaining options.
After last week I was hoping that putting a bit more effort into organisation would turn into a higher output of prototypes. Unfortunately it wasn't quite the magnitude I expected. I committed to do one every day, but I only managed 4 days.
This was attempt two. I figured out how to do a clever solver that would generate and solve numbers from a seed instead of brute forcing everthing, but then I got stuck on how to be sure that the seeds were comprehensive. In the end, I gave up on this method too.
Finally, I decided if I can't work smarter, I'll work faster. I rewrote the thing in Rust, which was actually really nice. I had to do a little bit of type juggling until I realised that because my arrays need to index themselves I should just make everything a usize even if the numbers never go above 10. I also got to use magical Rust threads, and not having to worry about synchronisation felt pretty good.
I've poked around a bit with Facebook's Flow type checker, but I thought it might be fun to do something else with it. I wanted to make a friendly type parser, so you can write "a function that takes an array of string and an optional number and returns a number" and get back (a: array, b: ?number): number. I got it going, but I'm not sure the types really helped me that much.
I've been thinking about the prototype situation over the last few weeks, and I think the simple fact is that I've been relying on the commitments too much and not respecting them enough. Commitments should be something you are very likely to achieve, and at least for the last few I've been using them quite aspirationally. If I was asked how surprising it would be if I failed that commitment, the answer would be not at all. Not exactly how I meant to do things!
I think, in particular, maintenance goals are a poor fit for the platform. Firstly it gets boring committing to the same thing constantly, but also I don't think public commitments are really granular or focused enough for the kind of thing I'm doing. Doing this properly means steady effort every day, and a system that reflects this. With writing, I have a very short feedback loop from missing a deadline to dealing with it, whereas right now for the prototypes it's often a week.
Which is all to say that I'm going to stop making prototype commitments. I think there is a better system out there, but I'm not sure what it is yet. In its absence, I still have my weekly prototype wrapups, and the habit is fairly well established to the point I don't think I'll just stop doing it. I might look into my monotonic stats idea and see if that proves to be a better motivator. Either way, I think turning down this particular kind of pressure will actually be pretty helpful, because it makes room for a different solution.
I have been playing The Witness recently, and I feel like its themes, or at least what I percieve as its themes, are powerful and interesting enough to be worth going into in some detail. I won't be spoiling anything in particular about the puzzles or the story, but in a sense discussing the themes is a meta-spoiler. You may want to play the game first to get the full value of experiencing these themes firsthand rather than having them described to you.
In mathematics, a witness is a fancy term for an example. If you say that aliens exist, a witness is an alien. If you want to prove that not all prime numbers are odd, a witness is 2. Many proofs are complicated, but existence proofs are, in this way, very simple: you just need a witness. Much of the game involves solving simple checkerboard puzzles by drawing a line from a start to the end. You are given the board, the game asserts that the board is solvable. All you have to do is provide the witness.
Of course, a witness is also a person who observes something, and observation is the other thing you do a lot of in the game. Despite its core mechanics being no more fundamentally complex than what you'd find in a standard mobile puzzler, the real magic of The Witness comes from its setting and environment. The puzzles are tangible, physical things that you find as you explore the island they inhabit. At first, this seems like a curious and extravagant way to make a puzzle menu. Later, you realise that the puzzles and the environment build on each other in ways that make them inseparable. Finally, you realise that the puzzles are the island, the island is the puzzle, and it was only your narrow-mindedness that tried to pull them apart in the first place.
It may sound surprising for a puzzle game, but the main theme of The Witness is spirituality. This manifests mostly through the game's collection of audio logs scattered throughout the island in hard-to-reach places. Each one is a few minutes of beautifully narrated monologue from a famous thinker on the nature of truth, God, the universe, and other philosophical topics. The main unifying theme behind these is that they address the relationship between the concrete and the abstract, between the physical and the ideal, and between person and God. Not God of the bushy beard and thunderbolts, mind you, but God of the unknowable ultimate truth.
All that over squares and lines? It's not as far-fetched as it sounds. In fact, the best moments of The Witness do have a kind of religious quality. You spend such a long time running around, thinking, trying, failing, trying again, and all the while completely lost in the mechanics of the puzzle, down in the figurative and often literal weeds. When you figure it out, however, when you have that spectacular "a-ha!" moment? Your thoughts fly out of the weeds and up into the clouds. You haven't just drawn a line, you've made a discovery, found a new truth with a life of its own. It is no coincidence that the game's final level is atop a mountain, overlooking the whole island of discoveries you made to get there.
And that feeling is really something beyond The Witness itself. Rather, it's about the joy of discovery, of knowledge, of having in whatever small way that window into the great truth of the universe. That truth is there, has always been there, and by just scribbling lines on a piece of paper you can be a witness to it. In that sense, this game provides a simple vehicle that lets you experience a kind of scientific spirituality, Einstein's "cosmic religion".
Experience is, not coincidentally, another big theme in the game. Why bother to make a game when you could explain this idea with a book? The thing is, you can't read a new feeling. And if someone tells you something, it never feels the same as discovering it for yourself. Indeed, I accidentally read the solution to one of the puzzles and it was immensely disappointing. Sure, I beat that puzzle, but the puzzle was just lines on the screen. I missed out on the experience of discovering the truth. In that sense, experience generation can only happen through your actions, can only work because this is a game where you participate.
Near the end, the game turns inward and begins examining itself. It becomes obvious that this experience generation is not a coincidence, but the whole point. Many games are made with a tutorial, but this is a tutorial made into a game. Normally that doesn't work, because the game ends and then what good is a tutorial? However the final, perhaps strongest message is to look around you. The puzzle isn't the board, it's the table the board is sitting on. It's the house around the table. It's the island under the house. It's the sky above the island. The Witness is a tutorial on how to think, and the final level is to turn the game off and keep thinking.
How audacious is that? There are lots of educational games out there, but this is something else entirely. Not content to merely explain or describe, The Witness is a series of experiences that walks you, one self-directed step at a time, to enlightenment. Many of us have believed games to be capable of truly amazing things, but evidence has been pretty thin on the ground. In its own way, this game is a proof by example. It is the witness. And, if you play, you can be a witness to it.
I wrote previously about the "what the hell" effect, where once you slip up a bit with something you tend to think "ah, what the hell" and slip up massively. Of course, that is objectively worse than slipping up slightly, so this falls in the category of unhelpful cognitive biases. I've been thinking recently that there might be a more general effect.
What I mean is that if you're on top of things, it tends to be easy to stay on top. But a failure in one area seems to spread to others, even if they aren't dependent in any way. My theory is that there is a kind of global "what the hell" effect, where once you've lost control in any area, you stop feeling on top of things which justifies everything else going bad too.
This is interesting because it suggests an alternative solution, both to this general "what the hell" effect and the more specific one. If being in out-of-control situations makes you generally out of control, maybe the antidote is putting yourself in unrelated situations where you are in control. That is, it may be beneficial to step away from the thing you're having trouble with and focus on something more stable.
In fact, it may be worth cultivating some easy habits specifically for this purpose. Keeping a clean desk could have benefits beyond just the outcome of having a clean desk. The very process of keeping the desk clean is an easy thing to keep on top of, which should make it easier to stay organised about other things just by virtue of putting you in an on-top mood.
A few days ago on Reddit, I saw someone post this observation: "The number 14233221 describes itself; it has one four, two threes, three twos, and two ones." I thought that was pretty interesting, so I thought I'd take a closer look. The numbers are very similar to the look-and-say sequence as studied to death by John Conway of Game of Life fame.
In the traditional look-and-say sequence you say 1, 11, 21, 1211, 111221, or "one", "one one", "one two, one one", and so on. However, we want to count the digits instead of reading them in order, so: 11, 21, 1112, 3112, [...], 21322314. This last number is actually the same as our original 14233221 with the digits in a different order. These have been described as descriptive numbers and counting sequences. What we want are the fixed points of this descriptive/counting function.
Another closely related idea is the autobiographical or self-descriptive number. These are numbers such as 1210 where the first digit represents the number of 0s, the next the number of 1s and so on. The largest of these is 6210001000. In fact, there are only 7 of them assuming you only allow digits up to 9. However, there are many more of the kinds of numbers we're looking for.
To find out just how many, I put together a program to search through them. Initially I wanted to be clever and search in some incremental way through the space of numbers by just picking some seed numbers and generating new numbers until I found the fixed points. Unfortunately, I couldn't figure out how to be sure that I'd get all the numbers. Maybe someone with more substantial mathematical chops than me would be able to figure it out, but I had to settle for a brute force approach.
However, I found an interesting shortcut: while you can have a solution that starts with 10 (like 10213223), you can't have one that starts with 20, or indeed any number of 0s other than 1. This eliminates 80% of the candidate solutions fairly quickly. I maybe could have found other clever things, but by that point my program was fast enough anyway, so I just ran it. It turns out the smallest of these numbers is 22, and there is only one solution that uses every digit. I'll put the full list in a box below, but it might be a fun exercise to find the largest one yourself.