This is going to be a fairly short one. Things were looking fairly good last week, so I committed to do keep making a prototype every day. For reasons mostly related to my failure earlier in the week, I didn't end up doing that many. And by not many, I mean one.
I've always been a big fan of Literate Coffeescript. It's a lot more lightweight than Knuth's original, just a kind of Markdown where the code blocks are run as actual code. For a long time, I've had a dream of being able to use this style with languages other than Coffeescript, so I started writing a proof of concept in Javascript.
I'm not going to dwell exceptionally much on having fallen so far short of my commitment; this is still a fairly fragile and underdeveloped habit, and near as I can tell the main lesson here is "new habits get stomped when a lot of other things go wrong". If it becomes a recurring problem I'll take another look, but for now I'm just going to concentrate on keeping everything else in order and building the habit up. That means I'm making the same commitment again.
In positive news, I did actually fulfill my commitment to finish this post early. I think that if I can maintain that habit, it'll ease a lot of pressure when things go south, including on my prototypes. Until next week!
There's something I've been thinking about in terms of producing creative work. It's quite confusing that we use the same word to mean two different things. I think it's meaningful, even necessary, to distinguish between creation, the act of making things, and creativity, the quality of a thing being imaginative, novel, or eye-opening. I should note that I already said "creative work" at the start of this paragraph without specifying which I meant. They're easy to conflate!
Creation, that is, making things exist that didn't exist before, is often a fairly uncreative act. For example, writing a story is considered to be a creative endeavour, but actually most of the real creativity happens early on, coming up with all the great plot and character ideas. However, to actually make a story, you just need to spend a lot of time writing. What do you write? The craziest, most original, most creative words you can think of? No. You write what the plot, situation and characters need. In fact, you often have to be less creative in your words if your story is really out there, just so there's a chance people will understand it.
Creativity, on the other hand, I consider to have a very particular meaning. Just because you come up with something doesn't mean it's creative, and some things are more creative than others. The measurement is a bit tricky to pin down. For example, I would say creativity is novel and unpredictable, but so are randomly generated numbers. To me, the key factor is that exercising or experiencing creativity usually means thinking something you wouldn't otherwise think. It's this element of intellectual surprise that I think makes creativity unique.
It is possible to be creative without creating anything. Perhaps my favourite example of this is computer security, whose practitioners are nearly indistinguishable from wizards. Security is, at its best, a ratchet that only ever gets more secure. Every time a new exploit is discovered, everyone learns about it and it stops being an exploit. What that means is that to come up with new exploits requires thinking in a way that nobody has thought before. My favourite example: I once saw some researchers break an otherwise impregnable server by messing with the voltage going into it. They even extracted the secret key. All by messing with the power supply! Who even thinks of that?
I don't mean to suggest that you should strive to be creative all the time, or that all creation should be creative. Quite the opposite; I think maintaining that level of creativity on a constant basis would be impossible, or at least extraordinarily tiring. And, as in the story example, creation often involves a lot of fairly uncreative work. I think that is perfectly fine. In fact, it's important to develop those non-creative abilities so as to make the best use of your creativity.
That relationship is the main thing I'd like to emphasise. Creation and creativity are different, but they often go together, and the balance between them matters a great deal. Some people create without creativity, which is a shame because it leads to uninteresting work. However, a great many more are creative without creating, and that often means their creativity amounts to very little.
In software it's quite common to do automatic testing of code. Of course, you already do a certain degree of implicit testing as you write the code anyway; you mess around with different parts of the program as you build them, change things to see what works better, and generally put things through their paces. In most cases, the significant functionality will have been hit hundreds of times during development, so why bother to do automated testing?
The problem is that kind of testing only covers the obvious. Maybe you never thought to start clicking things before the page has finished loading, or you didn't try it on an old laptop or a bad internet connection, or even just never realised that some option that you never really use broke while you weren't looking. The problem is that the extremes and edge cases are where most of the bugs show up, and they're also the situations you're least likely to encounter. This is particularly true because you made the system in the first place, so chances are you won't write bugs that are obvious to you.
So we test. We test with random data, we test with random junk that doesn't even resemble data. We test with slow computers, artificially bad internet connections, weird browser/operating system combinations and odd screen sizes. We simulate unplugging random cables, maliciously large floods of web traffic, and random parts of the system crashing. Not always all of these, of course, but every one is a legitimate and useful kind of testing. The common factor is the extreme: in all cases, we want to push our code to the breaking point. We either want to see that it won't break, or we want to know what it takes to break it and what happens when it does.
But there's no reason this attitude has to be unique to software. Any system that is designed by people has similar flaws. We forget to consider certain possibilities, make inadvertent assumptions, or fail to realise when or how our ideas break down. As you go through life, you build up layer upon layer of these systems. You have a system for getting to work in the morning, a system for keeping your clothes clean, a system for thinking about immigration, a system for dealing with stress. Every one of these is tested by just being used in every day life, but not very well. Could we do better?
I think so. Although automated tests may not be feasible, we can still apply that spirit of pushing the extremes. For example, children are a substantial test for the getting to work in the morning system, though there are probably better reasons to have a baby. Someone with children will almost certainly have a better, more robust system than mine. Your system for thinking about immigration would become substantially more robust if you spent time as an immigrant in another country, or if you spent time talking with immigrants in your country. And your stress-handling system is only as good as the most stress it's been able to handle.
In general, it makes sense to avoid unnecessary difficulty. Why make your life harder than it has to be? But this is one argument for seeking out more difficulty. The more you can do so under controlled conditions, the better-tested your systems will be. That might have immediate beneficial consequences if it reveals a flaw in something you took for granted. However, the real benefit comes later, when you hit a difficult situation under less controlled circumstances. At that point you'll be glad to have a well-tested system behind you.
There's a trope you see sometimes in movies or TV shows, where a Technical Dweeb is doing a very important technical thing for the plot-dictated imminent deadline. "Damnit", the Important Leader Who Tells People What To Do says, "this is taking too long! Isn't there any way to go faster?" – "W-w-well, if we bypass the safety systems and reroute the engine power through this coathanger, but that would..." – "Just do it, Technical Dweeb! There's no time!". And, of course, everything works out fine.
There is a scary implication there, beyond just that coathangers are not rated for engine power, which is that there is a neat and clear negative relationship between speed and quality. That is, if you want better quality, it will take more time, and if you want it faster you can just sacrifice quality. Dangerously, it is somewhat true, in the sense that there are situations where you can clearly point to a speed/quality tradeoff. And, if you have a single event and the consequences of insufficient quality on that single event are acceptable, then it can be a good trade to make. But you have to be very careful with those assumptions.
For example, if you make screwdrivers, and each screwdriver needs to undergo a careful and lengthy sequence of heating and cooling cycles to be tough but not brittle, that's a clear speed/quality tradeoff. You could just skip that step and have a crappy screwdriver that either shatters or bends when you use it. Maybe it normally takes 8 hours, and this way you can get it down to 4 but it'll probably break the first few times you use it. That would make sense if someone wanted a screwdriver in half the time, and they only need it for ceremonial reasons, or they're giving it to someone they don't like. That's a single event where the consequences are acceptable.
Now imagine that you needed to use a screwdriver multiple times, but you're on a really tight deadline. You get the four-hour crappy screwdriver, use it a few times, and it breaks. Now you need a new screwdriver, but you're still on a really tight deadline. So you order another four-hour special. Of course, it breaks again. You're now up to 16 hours. In the single-event case, the crappy screwdriver would have saved you time, but in this case it's cost you time. If you did it right to begin with, you'd be 4 hours better off. Keep in mind, this didn't require taking some far-future view where the screwdriver had to last a thousand years, this showed up after the event repeated itself a few times.
But things get incomparably worse if you work in a screwdriver factory. You make screwdrivers, but you also use screwdrivers in their construction. Now you have a clever idea. Instead of selling crappy screwdrivers, you save time by using crappy screwdrivers internally. This is the same problem as before, but much worse; you're making bad tools with more bad tools. The bad quality slowdown becomes multiplicative: each crappy screwdriver has to be replaced more often, which means you need to make more, which puts more wear on the crappy screwdrivers you use to make them. It's a kind of endless logarithmic crap spiral.
So these two factors, the amortised cost of having to replace low quality work over time, and the multiplicative cost of bad tools, both point to a very different relationship between speed and quality. In some cases, the relationship can be positive instead of negative, and even exponentially so. That is, you can be faster and better, or slower and worse.
The stereotype is that quality is a nice ideal but that it must yield, in practice, to lower quality but faster work. However, I think that is backwards. It's actually the speed-quality tradeoff that is idealistic, and the practical reality is that cutting corners often leaves you worse off than you would be otherwise. That's not to say you can't save time by doing things more efficiently, or by doing less of them, but that you probably won't save time by doing things worse.
When I first started writing, I sometimes worried that I would run out of things to say. Maybe I'll eventually sit down at my computer, put my hands to the keyboard and... nothing. All out. These days I maintain a fairly thorough collection of ideas to write about that I've built up over time, but of course that only kicks the problem further down the road. I can't build up an infinite reserve of ideas, and if I start using more ideas than I generate, eventually I'll hit zero.
That thought tends to occur to me more when I haven't come up with any new ideas lately, but the funny thing is that I never really have difficulty coming up with ideas when I leave time for it. There's some process that trundles along in the back of my head, scooping up random fragments throughout the day and fitting them together into a sudden moment of clarity that surfaces out of nowhere. And, when I'm busy, tired or stressed, that whole process just turns off. It's not even that I try to come up with ideas and I can't. I just... don't.
But when I make time for ideas, give myself enough space away from distractions and stress, and just think for a while, it's surprising how easily it all comes back. I'm always grateful when it does, but it's strange to think how well I can get by without it. The scary thing isn't how hard it is to give up creativity, but how easy. If I wasn't specifically doing things that demand a constant flow of ideas, I might not even notice.