In the coal mine.

In a recent episode of the Ship Show podcast, Paul Reed interviewed Damon Edwards in which they made the analogy to Release Engineers being the "canaries in the coal mine." Meaning that when there's problems in the release cycle, the release engineers are usually the first to notice. Then they made the astute observation that traditional development shops don't push the detection upstream to the developers, but instead blame the Release Engineers, which is in essence saying, "send more canaries". Hence the name of this blog. 

Before joining Netflix, I was a full-time programmer, doing J2EE things, for a site with real customers. While technically I'm still a "Senior Software Engineer" my current role focuses on the tools used by engineers. It's not a Release Engineer position, since I don't actually release anything, but many of the tools used by engineers are for releasing, so I play in that space. 

If I directly help other teams 40% of the time and program the other 60%, I consider it a good day. Our team is lucky in how it's structured as a service organization. This is only possible because it's a core Netflix tenant that teams take responsibility for their releases. Developers are expected to be the ones that code, build, release and deploy. Teams might structure themselves into more traditional Dev/QA/RelEng roles, but that's up to them. Engineering Tools is there to help, but if we can't do it, the team has to take responsibility for what ever they're missing or having trouble with. Effectively meaning that we're not there to gate their release. If we do gate/block their release, we've done something wrong.

Our value comes in being domain experts (Ant, Ivy, Gradle, Jenkins, AWS), evangelizing good practices used by other teams and automating anything we can. (This means we do end up supporting our own work, e.g. Asgard, Monkeys, Aminator, Gradle builds. But we do our best to either contribute back to OSS, or just OSS ourselves, so that's its not unique to our team.) I hadn't ever seen this organization structure before, I wonder how common it is. It's amazingly effective, and I believe it's why the concept of DevOps has succeeded at Netflix. It's balances the line between making things easy enough for teams to deploy and not abstracting everything away.

I guess I'm trying to say that let's save the canaries and make great tools, so developers (like my past self) can control the entire deployment pipeline.