Why fail?

A year ago, I wrote Fail scale, about the value of considering not just success or failure, but how much. Something you just barely succeed at is worth thinking about and learning from every bit as much as a failure. I described near-misses as a "valuable signal", and I'd like to take a step back and examine the idea of signals, and how both failures and near-misses are examples of them.

An important question is: why fail at all? I mean, why bother having a notion of failure? Failure is generally considered a character flaw (outside of a few specific circles where it's celebrated), usually doesn't feel nice, and can be embarrassing. I previously wrote that frames of reference are valuable even without judgement or goals. Can't we just try our best to improve relative to those frames of reference without any thresholds of success or failure?

One answer is that thresholds match up with people's expectations better, which is true but not very interesting. I think that even in the absence of external pressure, failure is useful because it directs attention to where it's needed. Your life has hundreds of different aspects, and while it might be possible to measure them, it's not really feasible to pay attention to all of them at once. Whatever you're not thinking about might slip when you're not paying attention to it, and you want some signal when that's gone too far.

For example, I'm not currently paying a lot of attention to my diet, but if my weight increased beyond a certain level that would be a signal that I should start eating better and pay more attention. I haven't got a particular number in mind, but I do have a general idea of what would constitute an issue. I could obviously prevent that from happening in the first place by paying more attention now, but that would necessarily take attention from somewhere else, and maybe it won't be a problem at all. This signal lets me focus more judiciously.

A failure is just a particularly strong signal telling you that your plan isn't working. Unlike excessive weight, which is a signal about your eating system, a failure is a signal about your planning system. This is why I said that ignoring or hiding failures is terrible, because a failure to deal with your failures can have much wider-ranging consequences than any individual failure would. Any time you've set up a threshold for success or failure, it was to focus your attention on this particular outcome. Minimising that failure does the opposite.

With that in mind, we can now return to why near-misses are a useful signal. As in the weight example, you probably want to catch a problem before its consequences get too bad. So ideally you wouldn't get a signal to pay attention after a failure has occured, but early enough that you have time to do something about it. Essentially, a near-miss is a signal that a failure is probably coming. This means it's especially useful for situations where the failure's consequences would be particularly negative. You can get the signaling benefits of failure without having to pay its price.

One last thought on this, which is that thinking of failure as a signal that focuses your attention also means accepting that too much failure is not very useful. Your attention can't be everywhere at once, and failing over and over doesn't help you learn. In fact, thanks to the what the hell effect, it's more likely to make you give up. Conversely, too little failure is also not very useful because you're wasting a valuable signal.