Process pipelines

I've been thinking about what changes I need to make for my future posting plans, and along the way I started thinking about a structure for these kinds of problems in general. A lot of things I want to do involve managing two distinct processes: an internal creative one and an external expressive one. These processes work very differently, but to make the whole thing work you need some way to synchronise them. My idea for doing this is called process pipelines.

A process pipeline is a system for a given project (say, writing) where each result goes through a number of stages (say, idea -> outline -> draft -> final). Separately, you have some kind of release schedule that you want to meet. That might be the two-week cycle common in software, a daily habit, or even monthly major releases. Each bit of work in the pipeline will take a slot in your release schedule. That can either be attached to the work (ie this project is for the November release), or to the slot (ie this November I'll release whichever project is the furthest along).

This provides an orderly way of tracking the internal process on a project, the external process of releasing it, and the connection between them. It's particularly relevant for creative work because of how unruly that work can be. You don't know when an idea's going to come, so you need to be ready to store those ideas in a place where they can eventually be developed. But you may also get partway through some work and get stuck, and often the best thing to do is just leave that work where it is and focus on something else for a while. As long as you're moving something through the pipeline, that's still a result.

Crucial to this is using the data that the pipelines give you. How many things do you have in the idea stage? How long does it take you to get from draft to final on average? Which stages have lots of items and which have very few? Which stage is the most difficult, and can you make it less difficult or break it into more stages? Are some stages easier to work on at certain times, in certain places, or in a certain mood? Is the total time to finish the next piece of work less than the time until the next slot? If not, you have a problem...

The other nice thing is that it makes matching work to releases more obvious and explicit. You can allocate work to slots, which becomes easier as the work gets closer to final stage. The slots with no work allocated should be obvious, making it easy to figure out what needs more development. You can build up a buffer of work in the final stage so that releasing is always easy, but also you can explicitly decide what to do with release slots that don't have any final-stage work ready. Either you can release some earlier-stage work or just miss the slot, depending on your preferences.

A while back I wrote about consistency, and the difficult question of how to balance the structure necessary for creation and the structurelessness necessary for creativity. I'm hopeful that this might be one answer: don't try to solve both in the same way. Rather, have one system that captures the unpredictable output of the creative process and link it to another that describes the predictable output necessary to get that creativity out into the world.