Prediction vs control

Imagine you're an old timey rainmaker. You know that people can't control the weather, but lots of suckers think differently. How can you use this to your advantage? Well, maybe you can't control the weather, but you can predict it. So you go to towns that have had a long drought and, according to your predictions, are due for rain. You do your rain dance, it starts to rain, and you cash in. This is thought to be the method used by famed rainmaker Charles Hatfield, for whom it worked so well that he was once accused of flooding San Diego.

This similarity between prediction and control is something I've been thinking about, especially in reference to the idea that you can control the outcome you get or how long it takes but not both. I'm sure some people would disagree, saying that for well understood problems you can get a certain outcome within a certain time. But really, this is just confusing prediction and control. You can predict you will get the outcome by the time, but you can't control it. The causality only goes one way.

Project estimates are a great example of this. Why are they wrong so much, especially in software development? In theory, they're a pure exercise in science, just like predicting the weather. But, as in management consulting, people ask questions that they want particular answers to, making it less a question than a request for justification. I think estimates are often wrong because the estimators are often, implicitly or explicitly, asked to be wrong. The hope is that changing the prediction will change the outcome. But that's not how prediction works.

And you can tell whether your estimates are really attempts at control by what happens when they're wrong. Predictions can be wrong, but control can't. If your estimate was that the project would take a month, but it's looking like it will take longer, what happens? Do you change the estimate or are you expected to somehow make it happen anyway? If you have to make the estimate come true, then it's really just an order with a thin veneer of science. Control, not prediction.