With rare exception, software never seems to be complete. Donald Knuth famously gives TeX version numbers that asymptotically approach pi. The last major version was 3, and it's currently on 3.14159265. There will be no more major versions, no new features. Each subsequent version will only include (more and more minor) bugfixes. That is to say, Knuth considers TeX to be done, and is now only pursuing closer and closer approximations of correctness.
I think it's useful to think about software is in terms of the Platonic ideal. In the real world, of course, you run into a lot of problems trying to define a perfect abstract chair, but there is absolutely such a thing as a perfect abstract algorithm. When you're implementing Quicksort your implementation is an approximation of that algorithm. And, in a sense, all sorting algorithms are approximations of an ideal abstract sort, which returns correctly sorted elements with the minimum possible resources.
Even for less abstract problems, it can be meaningful to use Platonism to think about software. You can define a perfect calculator as one that includes every one of a set of defined mathematical operations, executes them in a defined (bug-free) way, and operates using the minimum resources to achieve that. In a sense, all testing (and in particular formal verification) relies on this idea of an abstract ideal program that your program can be measured against.
However, the more your software tries to do, the more complex that Platonic ideal is. It's rare that a piece of software will be as simple as a calculator; usually the requirements will be in some way defined by the needs of real people, which means that the software model also needs to include a user model, and if it is commercial software, the software model is dependent in some way on a business model. These result in user acceptence testing and behaviour-driven development.
In the extreme, your software's requirements can become so complex that its abstract ideal is a system of infinite size. Perhaps that sounds hyperbolic, but actually it's not so uncommon. When you define software by the features it has, ie: does x, does y, does z, it's going to be finite. But systems are more often defined in terms of goals like "people can say anything and our software will do it", or "be able to access all the content on the internet". Apple's Siri and Google Chrome are both implementations of an infinitely large abstract system.
How can you tell? The question is, when would you stop adding features? What would your software have to do to be finished? Siri will never stop adding features because we will never stop having things we want to do. Chrome will never be finished because there will always be more new kinds of content on the internet. The ultimate end goal of both systems is a fully general system capable of doing anything: a Platonic Black Hole.
If you're making a system like that, it's not necessarily a bad thing, but it does make your situation kind of unstable. While other software will slow down as it asymptotically approaches completeness, yours will just keep growing and growing forever. The eventual fate of the finite system is that the marginal cost of additional work on it drops below the marginal benefit of ever-tinier improvements. At that point, you can step away from the keyboard; it's done.
But the eventual fate of the infinite system is that it gets so large that it can't adapt to future change, and another, newer infinite system takes its place.
A while back I wrote about the Elo paradox, the problem caused by motivation-by-measurement: if your measurements start dropping for any reason, your motivation drops, which causes a further drop in performance. I suggested at the time that a better solution would be to measure cumulative output, which has the advantage that any additional effort is always positive; you're never working to get back to zero, just slowly adding more to the endlessly rising tide of progress.
I had been meaning to put my money where my mouth is on this front, but I've historically had trouble with personal statistics systems; nothing off the shelf works, so I basically wrote my own and, predictably, it broke at some point. However, I took a bit of time in my prototypes this week to start over and do something slightly more robust, and added an extra layer to pump selected stats from the stats system onto this site.
You can see the result on my new stats page. It's nothing fancy, but the underlying architecture should allow me to put all sorts of other stats in there as I write more plugins to collect them. Once I have a bit more data I can add some cute graphs and things, which I'm really looking forward to. I'm hopeful it will be as motivating as I hypothesised. Can you believe I've written over 100,000 words here?
The code for the underlying system should be usable by others fairly soon, it just needs a bit of cleanup to migrate its way out of my prototypes and onto GitHub & npm.
I was really into Minecraft when it first came out, as were most people I knew. These days it seems to be mostly popular among kids, but back then everyone played it. The game had no goals, no achievements, no storyline, and in that it was particularly elegant, but it did mean you were kind of limited by whatever you could come up with. I enjoyed that for a while, but eventually the end of Minecraft, like the end of all open world games, was just getting bored and finding something else to do.
Much later a friend showed me his heavily modded Minecraft server. "Hey, check it out", he said, "this is way better than the vanilla game". Indeed, the mods added some kind of alchemy system, lots of new materials and, most importantly, factories. The factories allowed you to craft things automatically which, along with the rest of the features, made it altogether possible to build an entire supply chain. Materials could be automatically extracted, farmed, or mined, then sorted, conveyed to the appropriate machinery, crafted into other materials, and so on.
The problem with all of this was that all this equipment took a lot of materials to produce. All the machinery had to be made from parts, which were themselves made from resources that needed mining. So the first order of business was to make the machinery to make more machinery. But that needed resources, so I made machinery to extract the resources, which required more machinery to make that machinery. And then I kept running out of electricity, so I built machines to make more solar panels, which meant more machinery, which meant more solar panels...
I never really did finish my factory factory. I mean, it certainly produced a lot of parts to make itself bigger and better, and it did get incredibly large and sophisticated, but it didn't ever do anything beyond perpetuating its own existence. It was a machine optimised for optimising itself, and little else.
Leadership is a strange concept. I'm not sure we'd have a word for it, or even consider it to be a single coherent idea, if not for our particular social hierarchical history. It's something to do with power, charisma, psychology and influence, ability, and using all of those to achieve particular goals in a group context. But there's no reason to think those ideas necessarily go together. Ants, for example, have a complex social hierarchy but nothing resembling leadership. It's possible that a group of intelligent and rational enough social animals would be the same: there's no need for leadership if everyone can just figure out what to do.
I like to think of leadership a way of approaching distributed decision-making. You see it pop up organically in distributed software systems a lot, where decisions can't be made completely independently (there has to be coordination), and they can't be made completely dependently by one dedicated decision agent (because that's not reliable or fast enough). In those systems, leadership is a way of balancing the two extremes: you have some coordination and some independence, and you mediate the two by choosing one agent to be the decisionmaker for some things some of the time.
Of course, we don't usually have anything so formal in human systems. There are formal elections for companies and governments and so on, but most leadership is done on an ad-hoc basis. Even when there is an official hierarchy, decisions are often made around, not through, that hierarchy. So how do we informally decide who makes what decisions? I believe in most cases it's on the basis of whoever has the most and best information. Much like in a distributed software system, when you want to promote one of a bunch of equal systems to be the master, the one that already has the most up-to-date information is the best choice.
It's been my experience that often the best way to get ahead, even in fairly complex political situations with lots of people and agendas, is just to know more than everyone else. If you have a clearer idea than everyone else of what's going on, or what everyone's goals are, or where to go, you're much more likely to get what you want. Partly this is because you'll be able to make propositions first and be better prepared to argue against propositions you don't like. However, a significant part is just that people are often happy to follow the lead of anyone who seems to have thought things through more than they have.
Beyond its implications for others, though, this can also be a useful technique for yourself. It is sometimes tempting, even after you've made a plan, to second-guess yourself in the moment. In a sense you are mistrusting your own prior decisions, not willing to delegate your present situation to your past self. But one way you can fight this is by vastly outmatching your future self in preparation when you make a plan. This shouldn't be too difficult, because your future self has only a limited amount of time and energy to spend second-guessing, whereas your past self can be more comprehensive.
That means projecting forward when you make the plan, imagining your future self and preparing responses to things that future-you might do. If you plan to go for a run in the morning, but then it starts raining and you didn't think of that, maybe you won't run. Better to have an answer ready, like "If it rains I'll just run in the rain and shower afterwards". In a sense it's just regular contingency planning, but the main goal is to be prepared enough that your future self can just go along with your plan.
It pays to be the most prepared person in a situation, and that's no less true when the other person is just you at a different point in time.
Last week I scaled up my prototype goal dramatically, to one prototype per day. Unfortunately, I didn't quite get there. I did one every day except for Sunday, because my prototypes got a bit too ambitious and I ran out of time.
I wanted to try making some kind of cellular automata that was more suited to sounds. I previously made The Sound of Life which was a Game of Life + sound, but 2d grids don't actually map that well to audio (adjacent notes sound discordant). I had an idea for doing something in quotient space (an x,y grid where the coordinates reduce like fractions) using x/y as the frequency. It actually took a surprisingly short time to get working, but then I spent a few hours just messing around with different sound rules. It ended up pretty decent.
Continuing on from the day before, I wanted a better visualisation of how the sound was laid out. I had two ideas of how to do it, and this was the first one: a simple logarithmic scale drawn on a number line like so.
The second half was to try laying it out in a way that reflected the quotient space of the (x, y) pairs rather than the notes that came out of them. So I put everything on a big grid and drew lines to indicate the slope of the ratios. That ended up like this, which actually turns out to be a pretty pleasing visual.
I'd had an idea ages ago for teaching programming by making someone play-act as the computer. This was a start on that by making an environment where you can implement quicksort by yourself. It's just some instructions and a simple list of numbers that you can drag around and click to highlight. Later on I'll look at adding the part that tells you what step to do and checks if you're doing it properly.
This was a big one. I had this crazy idea a while back that maybe you could do computation with recursive acronyms. It ended up turning into its own post and, although I was really happy with it, it turned out to be a much bigger idea than I thought. I inadvertently re-invented L-systems, which was neat but probably not really feasible to do in an hour.
So a few good things came out of this week. I was particularly happy with the way the Tuesday->Wednesday->Thursday transition was three prototypes on top of the same idea. That seems to be a feasible path to making projects out of prototypes. I'm still not done with Audiomata; I want to consolidate what I've done so far, clean it up, add some more customisation and input, basically finish it off to the point where it meets the complete standard rather than the prototype standard.
However, I also missed a day, and that traces itself directly back to Saturday (and to an extent Tuesday), two days when I went over because I was so invested in what I was doing that I didn't want to stop. That's a good feeling, but obviously it's not sustainable. My plan is to be more militant about stopping when I run out of time, but give myself the option of continuing on if I acknowledge that it has outgrown the "prototype" label and move it into a real project in a new directory. That actually happened both times I went over, but after the fact instead of being acknowledged up front. My hope is that by recognising that the prototype has turned into a full project, it will give me the perspective and the clean break point to decide whether I should stop or keep going legitimately.
I am committing to a prototype each day again. Overall this system seems to be working well for me so I think I will keep at it until I can work the kinks out and it becomes easier.