Tuesday, May 21, 2013

Cord cutter

What do you buy a tech geek, who just buys gift-priced items throughout the year, for their Birthday? I just had a birthday, and I have no idea how my wife can find something that I don't already have.  It's not like I'm going to wait 2 months to buy a Roku or Shoes just because its temporally near my birthday. In contrast, a big gift wouldn't be a big surprise since everyone should coordinate something big (I've been burned by that, but that's another post).

This years gift was most fortunately by-chance. At a yard sale for some kids clothes, I stumbled upon a pair of Sennheiser HD-280 PRO Headphones. I had forgotten that I was on the lookout for new headphones, but seeing them there reminded me. The model number sounded really familiar too from my research. My mom taught me how to conduct myself at a yard sale, and jumping onto your phone to do research should be avoided (we wouldn't want to show our interest too much). A little flippant negotiation and I walked away with them for $3. They retailed for $150 when I was looking, but they seem to be down to $99 now.

Just getting a deal like that is like a present, but also getting nice replacement cans was the cherry on top. I used them for a little while in Neverwinter and with some FLAC files, then I quickly realized that I couldn't go back. They're great. There was one problem, the 1/8" plug was flakey. I tried putting a 1/4" collar around it, a complex set of rubber bands, and a few other things, but it was really sensitive to any movement. The solution was to replace the cord. The first option was to buy the replacement cord directly from Sennheiser for $30. That's not cheap, but given how little I paid for them, it's very reasonable. But what about when the cat chews on the cord? Or my daughter practices her scissor skills? I had just gone to Maker Faire, and wasn't going to let this opportunity to permanently fix them slip me by.

I modded them with a 1/8" jack, so that I could replace the cable in the future really easily. The Dr Dre Beats Headphones takes this approach and consequently has made a market for 1/8" to 1/8" cords just for headphones. I was little scared of taking them apart, doing some gutting out with a Dremil and performing some soldering. But, what the hell, they were only $3. Luckily the Sennheiser cord actually can detach and uses standard headers that I had laying around.

I had planned on ordering from Amazon and crossing my fingers that the jack would fit, but it occurred to me to check Radioshack and amazingly they had the part and it was in stock. That meant I could pick one up on my way home and get started right away. It came together really fast and I had it done just before 9pm. I used a closed-circuit panel mount plug, which sat really nicely after some plastics was drilled out and looked like it was originally there. I think they turned out really well. Future replacement cords can now just be bought from monoprice for a few bucks. I now also have the possibility of buying a cord with control buttons that I can use it with my phone.

I highly recommend attending Maker Faire, if there's one in your area. I left with some motivation and eventually granted some satisfaction from "making" something.

Call all the things!

At Gradle's core is a really nice set of Collection classes, which I wish were exposed on their own. The central feature is the idea around the all(Closure action) method, which says "run this Closure on every object", including ones added later. This is defined in DomainObjectCollection and it's leveraged by a few other methods. Specifically withType(Class<S> type, Closure configureClosure)and it's generalization matching(Closure spec). While they're very similar to Groovy's (and other languages) findAll or each, the extra temporal dimension makes it much more power for an API designer. Typically an API would provide a Collection which delegates to another collection while putting some triggers in place when something is added. With DomainObjectCollection, the user could create the collection and even prep it before passing it into an API call. It also allows multiple people to essentially register listeners, sorta like a property listener. In the following example, I configure other parts of the model based on what they all, for any value added or which will be added.

Loading ....
Another useful generic Collection is NamedDomainObjectCollection<T> (with its supporting implementations). These allow a collection to be built up with the assumption that each has a unique name, and it's really not too far from a Map with the key being the name. The magic comes from being able to build up an instance with methodMissing calls. The resulting calls look really slick and screams "DSL", I wish I had it for every Groovy DSL I write. As you can see in the next sample is a minimal way to create a list, which also has the ability to be configured for each item.
Loading ....

Tuesday, May 7, 2013

Another soul lost to Ubuntu sound issues

People have very selective memory (remembering the good things and forgetting the bad things). This is very true for the first few years of being a parent. It's also true for sound support in Ubuntu. Every time I upgrade my desktop's version of Ubuntu, I seem to be effected by some odd unknown problem with my Intel HDA sound. I say "seem" since I have selective memory about having trouble with it. But the last few days have brought all the horrible memories of scouring forums and bug tracking sites from years ago. Each of which gives "advice" that potential only cripples a system in other ways. For example, installing some PPA that hasn't been updated, or using OSS instead of ALSA, or ALSA instead of pulseaudio. Let me immediately say that I don't fall for them, I only browse them hoping that someone has commented on them with some real help. I found the best summary of this problem in an article titled "Top five wrong ways to fix your audio." The problems stem from Canonical having two audio troubleshooting pages. The first one, "SoundTroubleshooting" is outdated and downright dangerous, then it says to the use the second page. The second page, "DebuggingSoundProblems", has no more than "is the speaker plugged in."

My most recent upgrade to 13.04 manifested itself as ALSA being fine and pulseaudio silently coming up with no devices. This was good since it eliminated a slew of problems with the kernel, which is where 90% of ALSA really lives and consequently problems with how the modules are loaded. The way I knew this was running "aplay -L" to see which devices ALSA knew about. You can also run "alsamixer", but it won't give you device names. The best diagnostic tool I found for ALSA was speaker-test, which can target any of the multitude of drivers available. I quickly learned that some process has taken over the default sound device, via a "Resource is busy" message. I'm still not sure why "lsof /dev/snd/*" did tell me which process was grabbing the device, but a quick scan of "ps aux" showed the culprit as Pulseaudio. Odd, since Pulseaudio was exactly what I was trying to get working.

The breakthrough was realizing that the owner of that Pulseaudio was a user called speechd, which I assumed was some sound daemon, but proved to an artifact of the speech-dispatcher package. It turns out that this package starts up early on, starts its own Pulseaudio and never gives it up. It didn't take long to know that I didn't need a speech library for Lisp. Uninstalling the package did fail, but a manual userdel and it was cleanly gone. A nice side effect of uninstalling speech-dispatcher is that Ubuntu starts up twice as fast, and now safely shutdown (which it couldn't before).

I'd like to believe that my diagnostics process was this clean. Not even close. There were hours of trying new kernels, snd_intel_hda settings, and pulseaudio configurations. At this point I'm expecting to have another sound problem come November, but hopefully there's one less way it can fail without me knowing how to fix it.

Tuesday, April 30, 2013


In Gradle, I've come upon a development pattern. It usually follows this pattern:
  1. Write plugin
  2. Write Extension, so that the user can customize the behavior of the plugin
  3. Wish I could use Convention Mappings, but then resist since they're not public
  4. Realize that I can't look at the Extension until afterEvaluate, which is after the user's code has had a chance to set values on the Extension
  5. Put all my plugins logic in project.afterEvaluate
  6. Swear at any @DeferredConfigurable plugins
  7. Fight the ordering of my plugins
When I say @DeferredConfigurable, I'm pretty specifically looking at the PublishingPlugin's DefaultPublishingExtension, which will throw an exception if you try to configure it after it's been configured. Honestly this make sense, but it doesn't make it any easier to use. The publish plugin adds it's own afterEvaluate which forces the extension to be configured, since it needs to create tasks before the task graph is built. 

This means that if I have a Plugin that uses an Extension which is going alter a Software Component, it HAS to run before the publish plugin is applied, so that my afterEvaluate is run before the PublishingExtension's afterEvaluate is, keeping in mind that afterEvaluate is evaluated in insertion order. But that's impossible since I can't configure the PublishingExtension until after the PublishingPlugin has added the PublishingExtension to the project. I think I've also seen issues where the IvyPublication is resolved before I can add in my generated/imported dependencies, which is another reason I need to run before the publish plugin's afterEvaluate.

Technically the publish plugin is @Incubating, so I'm not faulting the Gradle guys and don't even expect this to be a bug in its own right, since they're still figuring it out. In my cause, I require the ability to modify the module descriptor, and that's only really possible with the publish plugin, so I'll live with it. Though I'm starting to get the feeling that there's some fundamental ordering issues cropping up, which will eventually grow out of control.

My solution is implement a beforeAfterEvaluate method that lets me jump to the front of the dispatcher LinkedHashMap. Caveat, there's lot of internal classes being used, so I know that it can break in future versions of Gradle.

Loading ....

This doesn't answer the general ordering issue, but it can unblock a stalemate in the order of plugins. It's worth mentioning that Convention Mappings aren't the solution either, they're just an opportunistic lazy evaluation that "typically" works in your favor.

Saturday, April 27, 2013

Ahoy Matie!

At the last Java Posse Roundup, I gave a lighting talk on "Thinking with box" where I outlined some fun I have had with cardboard with my daughter. I'll link to the video once it's uploaded (I'm looking at you +Carl Quinn) . Here are some of the steps, with some tips and tricks, that I did to make it. (I've meant to write this up for a while, but I finally got a little push from @Chris Cobb which put me over the edge.)

Step 0: Surf the web

My wife was on Pinterest looking at pirate ideas for our little one's 4th birthday, and looking over her shoulder I saw a cardboard pirate ship that looked like a lot of fun. So I dug deeper by searching Google for "cardboard pirate ship", and wow, there's a lot of stuff out there that adventurous parents had done. I zero'd in on this picture pretty quickly.
Step 1: Build a Model

I'm fortunate to work at a company that has lots of spare TV boxes around, but I didn't necessarily know how to get them for myself, so I filed a Helpdesk ticket. They were fantastic in helping, they brought by samples to see exactly I was interested in. They even took a huge whiteboard out of a box, just to give to me. I had some great building materials, but I didn't want to waste any and I wasn't even sure if I had enough, so I built a model. 

This proved helpful in designing the large straight pieces, and more importantly the angle for the bow.

Step 2: Cutting

I'd say cutting cardboard is best done like drywall, where you score one side with subtle pressure, folding it, then cutting from behind. Before you score, I'd recommend getting a very long straight-edge and drawing a line where you're going to score, even if you're going to score against the straight-edge. I didn't have any rulers long enough, so I ended up using some wood scrap, which I'll talk about later. The cutting was primarily done with a utility knife.

Step 3: Attaching

The model showed me that I had to cut the bow as a trapezoid  but once I got to real cardboard I learned that there more complex angles going on in three dimensional space. But I was close enough that I thought I'd just start attaching everything. The original idea was to just staple gun two pieces of cardboard, that was a horrible idea. I don't even know why I thought that would work. The additional stress on the bow make it clear that I'd need more support, so I went back to Home Depot and picked up some wood scraps which I could screw into. The best way to describe it was 1/2" x 1". It's no more than a $1 per 8 foot piece (wood is cheap in California). You can seen an example in the picture for the mast.
I'd cut the strip to length, letting the little one pull down the chop-saw. To attach the cardboard, I put washers on a drywall screw then threw the cardboard into the wood behind it. The washers were necessary to prevent the screwhead from going through the cardboard. It was quite common for the screw to come through the wood, in which case I used a Dremil to cut it off the end sticking out. It wasn't perfectly smooth, but it was good enough for 4 year olds. If you zoom into the picture you might be able to  see the washers and wood at the intersections between pieces.

Step 4: Laying out

I worked from the bow to the back, and I honestly didn't know how each part was going to work out until I tried. The model really helped. 

I'd sometimes staple gun in between the screws to get a really good seal between pieces, but this probably wasn't needed.

Step 5: Cutting Windows

I cut a single window, then used it as a template for the other windows. I found that the utility knife had a tendency to go straight, so I used an X-Acto knife for corners. The thickness of cardboard meant I had to make a few passes before it went through, but tracing a previous cutout made it easy to follow the path. I'd usually duct tape a previous cut where I wanted the next window, so that it wouldn't drift as I cut around it.

Step 6: Painting

At most home improvement stores, they have previously mixed paint which has been returned and is now on clearance. You can get a gallon of decent paint really cheap. We did find that Home Depot was selling these clearance items at full price, our theory is that people were abusing the system so they needed to stop selling them so cheap, so we went to Lowe's instead. We let our daughter pick any color they had there, and we picked something to complement it. She wanted purple, but they only had pink. Painting cardboard proved to be really straightforward.

Step 7: Finishing touches

At this point it was looking fantastic, but we had already pretty much taken the day off of work, so we thought to fancy it up a little. My wife cut a large H and I put duct tape around the edges. And anchor was made by stapling loops of paper together. A sail was just an old sheet, which had to be cut or else the ship would literally sail away.

Wednesday, April 24, 2013

Seeing the rings

Right now is a great time to see the rings of Saturn, just follow the moon down and to the left, past Spica and the next bright thing is Saturn. Hmm. Let me rephrase that, it's a great time to see the rings of Saturn through a telescope. A few months ago we bought a telescope for the parents who took astronomy classes together the little one. We kept the budget manageable enough that we could get a nice telescope sometime in the future if this works out. We've learned some lessons, which might have been different if we had a $3,000 telescope:
  • Jupiter and Saturn are a rush to see
  • Stars are just dots in the sky, which you make big and blurry with a telescope
  • Four year olds have the patience to see one celestial object a night
  • Four year olds "get it" and are excited to see a blurry star even its just orange in color
  • The moon never gets old
I do think there's a limit to viewing. Once we saw Jupiter and the moon, there wasn't much more we could get that wouldn't be more enjoyable by just surfing images.google.com. But then we waited and saw Saturn. I think we're going to find that we bring it out every 6 months, see some new object in the sky and get a renewed interest for a week. And that's perfectly fine.

We're lucky that Orion is a local company, with a store in Cupertino. The whole family went over one Sunday and we brought home a very reasonably priced telescope. There's an argument to be had that we should have just bought really nice binoculars, especially given my above statement about bursty interest. There's just something special about having a real telescope in front of you. Our budget was <$400. To extend our budget, we opted to go without any tracking mechanism, so that we can could spend more on the optics. (It's fun to just talk about the optics, I understand optics better now than I ever did in Physics class, but I'll save that discussion for another post.) The other balancing act was portability, we wanted to be able to bring it to viewing parties and just whip it out from the garage when it's nice out.

In the end, we got an Orion SpaceProbe 130mm EQ, with a 900mm focal length. And after some initial fun we bought the Orion Telescope Accessory Kit for a few more length and some filters, including a moon filter. In both cases they were on sale in the store, so the total was around $300.

We like it a lot. It's not too big that I can't just leave it setup in the garage. It's advanced enough that I can play with different lenses. Maybe in the future we'll be able to justify $3,000 for a sweet Cassegrain, but for right now this works for us.

Tuesday, April 23, 2013

Don't dare prompt me

At Netflix, we've been trying to come up with an effective way to distribute the Gradle wrapper (more on that in another post), and the solution we're going for is to check it into a git repo, which can then be cloned locally or referenced as a submodule for CI builds. So, when we're constructing our wrapper and pointing it to a custom distribution of Gradle we need to perform git commands. Luckily Andrew Oberstar created a eponymous Gradle plugin called gradle-git, which wraps JGit. We already started to use it to publish javadocs to gh-pages in the Netflix OSS projects. In this case, I started to use it to check the new wrapper into a repo, only to find out that I was being prompted for my passphrase.

It's easy to forget that one's SSH private key has a passphrase on it, because most apps indirectly ask ssh-agent or Pageant for the key. (Note: I almost forgot my passphrase, please go and check that you still remember yours.) When I say most apps, I mean every possible app except JGit.  When JGit was written it appears that it was abstracting out the underlying implementation of SSH, but generally uses JSch to keep everything in the JVM. The problem is the JGit seems to use some defaults that bring a Swing UI to prompt a user for their passphrase every time it instantiates a session. This might make sense in an IDE, but in a CI server that is ridiculous.

A quick Google search shows that the creators of JSch figured this out and wrote jsch-agent-proxy. Great, all I have to do is wire that in and prest-o change-o, I'll no longer be prompted. Nope. I searched high and low and couldn't find an example of someone configuring jsch-agent-proxy with JGit. Some more research showed that JGit in all it's lovely abstraction layers hides the parts of JSch that I need to plug in jsch-agent-proxy.

Here's my implementation that connects the JSch under JGit to use jsch-agent-proxy:

Loading ....

Most of this code comes from jsch-agent-proxy's JSchWithAgentProxy example. jsch-agent-proxy has an impressive set of implementations. It can use Unix Sockets or JNA, or talk to ssh-agent or Pageant. Given how easy the JGit API is to use, I expect some interesting Java-based utilities to come out in the next year.

In the coal mine.

In a recent episode of the Ship Show podcast, Paul Reed interviewed Damon Edwards in which they made the analogy to Release Engineers being the "canaries in the coal mine." Meaning that when there's problems in the release cycle, the release engineers are usually the first to notice. Then they made the astute observation that traditional development shops don't push the detection upstream to the developers, but instead blame the Release Engineers, which is in essence saying, "send more canaries". Hence the name of this blog. 

Before joining Netflix, I was a full-time programmer, doing J2EE things, for a site with real customers. While technically I'm still a "Senior Software Engineer" my current role focuses on the tools used by engineers. It's not a Release Engineer position, since I don't actually release anything, but many of the tools used by engineers are for releasing, so I play in that space. 

If I directly help other teams 40% of the time and program the other 60%, I consider it a good day. Our team is lucky in how it's structured as a service organization. This is only possible because it's a core Netflix tenant that teams take responsibility for their releases. Developers are expected to be the ones that code, build, release and deploy. Teams might structure themselves into more traditional Dev/QA/RelEng roles, but that's up to them. Engineering Tools is there to help, but if we can't do it, the team has to take responsibility for what ever they're missing or having trouble with. Effectively meaning that we're not there to gate their release. If we do gate/block their release, we've done something wrong.

Our value comes in being domain experts (Ant, Ivy, Gradle, Jenkins, AWS), evangelizing good practices used by other teams and automating anything we can. (This means we do end up supporting our own work, e.g. Asgard, Monkeys, Aminator, Gradle builds. But we do our best to either contribute back to OSS, or just OSS ourselves, so that's its not unique to our team.) I hadn't ever seen this organization structure before, I wonder how common it is. It's amazingly effective, and I believe it's why the concept of DevOps has succeeded at Netflix. It's balances the line between making things easy enough for teams to deploy and not abstracting everything away.

I guess I'm trying to say that let's save the canaries and make great tools, so developers (like my past self) can control the entire deployment pipeline.