Github Automaintainer

automaintainer sketch

One thing I've always had trouble with is the long tail of Open Source projects. For the larger, more popular projects there's it's usually fine; there's an established community and some particularly organised and motivated maintainer has usually stepped forward. But for smaller ones the effort of maintaining a project seems disproportionately high compared to the effort of making the project in the first place. Who wants to review pull requests or build a community for some 50-line thing you threw together one time?

I think a great way to solve this would be some kind of Github Automaintainer. You would authorise it to access your repository and it would read a special file configuring the rules by which it should maintain your project. Rules would look like:

Though obviously those numbers and even the rules themselves would be customisable.

I think a lot of smaller projects could get away with nothing but that kind of minimal rule-based organisation, and it would be easier for everyone. Maintainers wouldn't have to do much work to maintain the project, and the explicit rules would make it easy to understand how to become a contributor to the project and how commits are approved.

And who doesn't like knowing their project is managed by a robot?

Illegal npm modules

Recently, I was a little surprised to learn that the FSF claims that depending on a library makes a derivative work and thus spreads the GPL. To me, that seems obviously ridiculous (how is it possible to create a derivative work of something you haven't modified? Or distributed? Or sometimes even looked at?). Indeed, you can have some fun watching the FSF tie itself into knots trying to avoid the inevitable conclusion that if you make a derivative work by calling code, then basically all non-GPL software is in violation of the GPL.

Anyway, this got me thinking about how in node-land, the common pattern is many many small modules. Indeed, if I install the top 60 most popular npm modules I get about 3000(!) dependent modules including dupes. With so many modules it would be very easy to accidentally include a GPL library somewhere. If you did that, your npm module is (according to the FSF) in violation of the GPL and therefore (according to the FSF) in violation of international copyright law and therefore (according to the FSF) illegal.

I thought it would be fun to find out how many people are breaking Stallman's intergalactic copyright law, so I quickly grabbed the most starred modules and installed them. Npm is actually a couchapp so this was pretty easy to do.


$ wget 'https://skimdb.npmjs.com/registry/_design/app/_view/browseStarPackage?group_level=1' -O starred.json
$ coffee -e "console.log require('./starred').rows.sort((a, b) -> a.value - b.value).slice(-60).map((x) -> x.key[0]).join('\n')" | xargs npm install

Then I waited for a very long time.

For the next step I used the awesome licensecheck module (don't worry, it's not GPL - you can visit that page without creating a derivative work). Many npm modules don't include license information in their metadata, because programmers are lazy, so it employs various sophisticated techniques to figure out and normalise the licenses into a consistent output. And I got back this:


$ licensecheck --tsv | awk '{print $3}' | sort -k1 | uniq -ci | sort -n | tail -12
   2 BSD*
   2 WTFPL
   2 WTFPL2
   3 AGPLV3
   4 BSD-like
   5 Do
  11 MISSING
  15 unmatched:
  41 Apache
  51 ISC
 160 BSD
 874 MIT

Luckily, it seemed like there were no hidden time bombs deep in the dependencies of the most popular projects. The three AGPLV3 entries that turned up are all part of the fairly popular pm2 project. But if it's popular that probably means there are other things depending on it...


$ wget 'https://skimdb.npmjs.com/registry/_design/app/_view/dependedUpon?startkey=[%22pm2%22]&endkey=[%22pm2%22,{}]&reduce=false&include_docs=true' -O depends_on_pm.json
$ coffee -e 'console.log require("./depends_on_pm").rows.length'
26
26? Let's hope all of them are GPL! Especially because, if they aren't, any recursive dependents would also be in violation, probably without even realising it.

$ coffee -e 'console.log require("./depends_on_pm").rows.map((x) -> "#{x.id}: #{x.doc.license || x.doc.versions[x.doc["dist-tags"].latest].licenses?[0].type}").join("\n")'
anthtrigger: MIT
bosco: MIT
bute: MIT
debian-server: MIT
diy-build: BSD
ecrit: MIT
ezseed: GPL
foxjs: MIT
g-dns: ISC
gatewayd-4: undefined
gitbook2edx-external-grader: BSD
hls-endless: MIT
hubba: undefined
itsy: MIT
lark-bootstrap: MIT
nodemvc: MIT
npm-collection-explicit-installs: MIT
nshare-demon: MIT
pm2-auto-pull: ISC
pm2-plotly: MIT
pod: undefined
radic: MIT
tesla: MIT
wordnok: MIT
yog-pm: BSD
zorium-paper-demo: undefined

...Oh dear.

Photoshop Light

Sketch of Photoshop Light

In honour of Dorkbot, which I went to for the first time today, here's a Dorkbot-style art-meets-technology idea I had a while back: Photoshop Light.

You set up a person in a chair with lights shining on them from 5 different directions: top, bottom, left, right and front. Each light is a digital projector being coordinated by a Photoshop-style interface. As you draw in the interface, the projectors change their image to make it appear like you're drawing on the person in the chair.

The tools would be mostly designed for painting shadows and light, tinting, and adjusting colour temperature, the goal being to experience the subtlety and importance of lighting in our perceptions. There would be a few presets for, say, regular light, dim light, evil-looking underlights, that sort of thing. But mostly I expect people would just play around and try to make their friends look either good or bad, depending on their preferences.

There would, of course, be the obligatory take-a-photo-and-tweet-it functionality built in.

Feedback loops

Feedback loops

Something that I've really started to appreciate the importance of recently is feedback loops. Any time I'm doing a project there's a loop between my actions and their results. I come up with an idea, I start figuring out how to implement it, make a couple passes at it, get something I'm happy with, release it and then show it to people. If they like it, or don't, that's good information and goes back into the next cycle of the process.

But I've been beginning to realise that my feedback loops are often way too long. I have projects and ideas that sit on my hard drive for months or years without seeing the light of day. Part of that is not wanting to release something crappy that will get me bad feedback, but another more significant factor is that my projects aren't structured the right way. It's easy to make something that's useless until it's finished, but much harder to make something that is useful and can gather feedback as early as possible.

This isn't new, in fact it's a central tenet of agile development, which I've found useful in a business context, but I'm beginning to realise that I haven't really appreciated the value of thinking this way in general. The feedback loop is critical because it's a fundamental part of any optimisation process. Any time you're solving a dynamic problem you'll be limited by how quickly you can see if your solution's working.

My recent realisation is that short feedback loops are more useful for creative projects, not less. When you have an opportunity to go deep into the woods and make something really interesting is when it's most important not to get lost and forget the grounding your ideas have in reality. More creative projects are also more resistant to traditional analysis techniques; they're less cerebral and more intuitive. You won't be able to solve the whole thing at once, and making steady progress is going to be impossible without feedback.

Perhaps most importantly, a short feedback loop is very motivating. I've had many projects stall out because they've been languishing in a half-done state for too long and I just lose the energy to keep caring about them. But every bit of feedback and every minor reinforcement propels me forward and makes me want to see the next iteration. Even without the other benefits, that alone is worth it.

Rebuilding

I had a fun idea a while back for a kind of extreme real world game: get a bunch of people together with a fixed amount of time and try to recreate civilisation from nothing out in the wilderness. I call it rebuilding.

The difficulty of the game mainly depends on how you define civilisation and how you define nothing. I think it would be reasonable for beginners to start with a knife and comfortable clothes, but if you're hardcore you could start with nothing but a loincloth. It would be tempting to cheat by starting with specific things that are impractical to make, like steel or a microscope, but I think that defeats the point.

Instead, the right place to cheat is by careful definition of the word civilisation. For a first attempt I think you would pass by just setting up some kind of functional dwelling + basic agriculture. Then you could move on to more advanced goals like building a non-human-powered vehicle, launching something 100 metres into the air, or isolating penicillin.

A good goal would have a lot of creative ways to approach it. The vehicle doesn't have to use a traditional engine, maybe it could be wind powered or run off firewood. There are a few differnt mechanical and chemical ways to make something go high up in the air. Penicillin... yeah, that one's pretty tough. You would also want to set up lots of intermdiate goals as waypoints, partly to give you something to focus on on the way to the major goal, and partly so you have a string of other successes to look back on when you don't reach your chosen definition of civilisation. The game's meant to be hard, and I think it's fair if it takes a few attempts.

A big part of it would be very careful preparation. Having a plan, or even several plans, and having practiced some crucial skills early on would go a long way to making the task achievable. Picking the right location for the goal (or the right goal for the location, if there aren't many choices) would also be crucial. If something happens such that you need real-world tools or medical attention, that would qualify as "dying" in the game. Obviously you wouldn't want to have phones and cars around, so you'd need to have a plan for how to get someone out if something goes wrong.

Ultimately I think it's a game that's appealing as an exercise in realising how far we've come and getting in touch with some of the technological developments that we take for granted now. Being able to go back and, in some small way, relive the experience of being on the front line of civilisation would be an amazing thing to experience.