Thursday, May 19, 2016

SSD Sneakernet

Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
-Andrew S. Tanenbaum, Computer Networks, 4th ed., p. 91
Twenty years ago the standard of measure of a high-throughput Sneakernet was a station wagon full of tapes. In 2016, we can do a little better. Let's see what we can do with a station wagon and modern storage. First, let's start with a fancy Volvo XC90. It has 85.7 cu.ft. of cargo space and a maximum load of 1521 lbs (148089.6 cu.in. and 24336 ounces, respectively). (For those nostalgia Volkswagen fans, you can mentally think of 1982 Vanagon which has similar cargo capacity of 93 cu.ft.)

You might jump to a large HDD, like a WD Blue 6TB HDD. It has the dimensions of 5.8 x 4 x 1 inches and a hefty weight of 15.8 ounces. Those dimensions give us 23.2 cu.in., meaning we could fit 6383 drives into the XC90. When you look at the weight of all those drives, adds up to 100851.4 ounces (6303lbs), which is way more than the car can handle. So, we can look at it the other way, how many can we fit by weight. The answer is 1540 drives, providing 9240 TB (9.24 PB) of bandwidth.

If we were to consider the edge cases of loading the data on to and off the disks (left to the reader to add to the calculations), we have to take into consideration the transfer speeds of the drives. In which case we should also consider SSD's, since they do offer significantly superior transfer speeds. For example, the highly rated Samsung 850 PRO 2 TB SSD. It has the dimensions of 3.9 x 2.8 x 0.3 inches (3.276 cu.in.) and a feather-weight of 1.9 ounces. I almost can't believe it, but that means the XC90 could fit 45204 drives. Though we run into the weight issue again and have to limit ourselves to 12808 drives. That provides for 25616 TB (25.62 PB) of bandwidth. So, clearly SSDs are the way to go.

One angle I don't see discussed often is the units for this kind of transfer. We can't just take the total bandwidth and say "per second". The same data is transferring in half a second and over an hour. We all know that cars are driven in miles per hour. So the units we have available are TB * miles / hour. We can make the simple conversion to seconds to make it feel more like a fiber OC192 link, but we eventually have to deal with the distance aspect of the units, which for us is miles.

Let's compare transferring the data from the car (travelling at 80mph) versus a Gigabit ethernet, a 10 Gig and a 100 Gig connection:




The larger scales make it hard to see, so here are the numbers:




The times are in seconds, so 4.50E+02 is 450 seconds, which is 7.5 minutes. So, this is saying over the course ten miles, it'll take the car 7.5 minutes to transfer the 25.62 PB while the 100 Gig Ethernet will take 569 hours (2,049,280 seconds divided by 60 seconds in a minute and divided again by 60 minutes in an hour). The station wagon wins! Only until about 45,000 mile mark does the 100 Gig Ethernet catch up. Logically, the 10 Gig Ethernet catches up around 450,000 miles and the 1 Gig Ethernet catches up at 4,500,000 miles.

A quarter million miles is an interesting number because it's the distance to the moon. So.... an XC90 could drive to the moon with 25.62 PB of SSD drives and still be twice as fast as a direct 10 Gig Ethernet connection to the moon. Sneakernet FTW!

(Apologies to every country except the USA for use of the Imperial system.)

Tuesday, May 21, 2013

Cord cutter

What do you buy a tech geek, who just buys gift-priced items throughout the year, for their Birthday? I just had a birthday, and I have no idea how my wife can find something that I don't already have.  It's not like I'm going to wait 2 months to buy a Roku or Shoes just because its temporally near my birthday. In contrast, a big gift wouldn't be a big surprise since everyone should coordinate something big (I've been burned by that, but that's another post).

This years gift was most fortunately by-chance. At a yard sale for some kids clothes, I stumbled upon a pair of Sennheiser HD-280 PRO Headphones. I had forgotten that I was on the lookout for new headphones, but seeing them there reminded me. The model number sounded really familiar too from my research. My mom taught me how to conduct myself at a yard sale, and jumping onto your phone to do research should be avoided (we wouldn't want to show our interest too much). A little flippant negotiation and I walked away with them for $3. They retailed for $150 when I was looking, but they seem to be down to $99 now.


Just getting a deal like that is like a present, but also getting nice replacement cans was the cherry on top. I used them for a little while in Neverwinter and with some FLAC files, then I quickly realized that I couldn't go back. They're great. There was one problem, the 1/8" plug was flakey. I tried putting a 1/4" collar around it, a complex set of rubber bands, and a few other things, but it was really sensitive to any movement. The solution was to replace the cord. The first option was to buy the replacement cord directly from Sennheiser for $30. That's not cheap, but given how little I paid for them, it's very reasonable. But what about when the cat chews on the cord? Or my daughter practices her scissor skills? I had just gone to Maker Faire, and wasn't going to let this opportunity to permanently fix them slip me by.


I modded them with a 1/8" jack, so that I could replace the cable in the future really easily. The Dr Dre Beats Headphones takes this approach and consequently has made a market for 1/8" to 1/8" cords just for headphones. I was little scared of taking them apart, doing some gutting out with a Dremil and performing some soldering. But, what the hell, they were only $3. Luckily the Sennheiser cord actually can detach and uses standard headers that I had laying around.


I had planned on ordering from Amazon and crossing my fingers that the jack would fit, but it occurred to me to check Radioshack and amazingly they had the part and it was in stock. That meant I could pick one up on my way home and get started right away. It came together really fast and I had it done just before 9pm. I used a closed-circuit panel mount plug, which sat really nicely after some plastics was drilled out and looked like it was originally there. I think they turned out really well. Future replacement cords can now just be bought from monoprice for a few bucks. I now also have the possibility of buying a cord with control buttons that I can use it with my phone.

I highly recommend attending Maker Faire, if there's one in your area. I left with some motivation and eventually granted some satisfaction from "making" something.

Call all the things!

At Gradle's core is a really nice set of Collection classes, which I wish were exposed on their own. The central feature is the idea around the all(Closure action) method, which says "run this Closure on every object", including ones added later. This is defined in DomainObjectCollection and it's leveraged by a few other methods. Specifically withType(Class<S> type, Closure configureClosure)and it's generalization matching(Closure spec). While they're very similar to Groovy's (and other languages) findAll or each, the extra temporal dimension makes it much more power for an API designer. Typically an API would provide a Collection which delegates to another collection while putting some triggers in place when something is added. With DomainObjectCollection, the user could create the collection and even prep it before passing it into an API call. It also allows multiple people to essentially register listeners, sorta like a property listener. In the following example, I configure other parts of the model based on what they all, for any value added or which will be added.

Loading ....
Another useful generic Collection is NamedDomainObjectCollection<T> (with its supporting implementations). These allow a collection to be built up with the assumption that each has a unique name, and it's really not too far from a Map with the key being the name. The magic comes from being able to build up an instance with methodMissing calls. The resulting calls look really slick and screams "DSL", I wish I had it for every Groovy DSL I write. As you can see in the next sample is a minimal way to create a list, which also has the ability to be configured for each item.
Loading ....

Tuesday, May 7, 2013

Another soul lost to Ubuntu sound issues

People have very selective memory (remembering the good things and forgetting the bad things). This is very true for the first few years of being a parent. It's also true for sound support in Ubuntu. Every time I upgrade my desktop's version of Ubuntu, I seem to be effected by some odd unknown problem with my Intel HDA sound. I say "seem" since I have selective memory about having trouble with it. But the last few days have brought all the horrible memories of scouring forums and bug tracking sites from years ago. Each of which gives "advice" that potential only cripples a system in other ways. For example, installing some PPA that hasn't been updated, or using OSS instead of ALSA, or ALSA instead of pulseaudio. Let me immediately say that I don't fall for them, I only browse them hoping that someone has commented on them with some real help. I found the best summary of this problem in an article titled "Top five wrong ways to fix your audio." The problems stem from Canonical having two audio troubleshooting pages. The first one, "SoundTroubleshooting" is outdated and downright dangerous, then it says to the use the second page. The second page, "DebuggingSoundProblems", has no more than "is the speaker plugged in."


My most recent upgrade to 13.04 manifested itself as ALSA being fine and pulseaudio silently coming up with no devices. This was good since it eliminated a slew of problems with the kernel, which is where 90% of ALSA really lives and consequently problems with how the modules are loaded. The way I knew this was running "aplay -L" to see which devices ALSA knew about. You can also run "alsamixer", but it won't give you device names. The best diagnostic tool I found for ALSA was speaker-test, which can target any of the multitude of drivers available. I quickly learned that some process has taken over the default sound device, via a "Resource is busy" message. I'm still not sure why "lsof /dev/snd/*" did tell me which process was grabbing the device, but a quick scan of "ps aux" showed the culprit as Pulseaudio. Odd, since Pulseaudio was exactly what I was trying to get working.

The breakthrough was realizing that the owner of that Pulseaudio was a user called speechd, which I assumed was some sound daemon, but proved to an artifact of the speech-dispatcher package. It turns out that this package starts up early on, starts its own Pulseaudio and never gives it up. It didn't take long to know that I didn't need a speech library for Lisp. Uninstalling the package did fail, but a manual userdel and it was cleanly gone. A nice side effect of uninstalling speech-dispatcher is that Ubuntu starts up twice as fast, and now safely shutdown (which it couldn't before).

I'd like to believe that my diagnostics process was this clean. Not even close. There were hours of trying new kernels, snd_intel_hda settings, and pulseaudio configurations. At this point I'm expecting to have another sound problem come November, but hopefully there's one less way it can fail without me knowing how to fix it.


Tuesday, April 30, 2013

beforeEvaluate

In Gradle, I've come upon a development pattern. It usually follows this pattern:
  1. Write plugin
  2. Write Extension, so that the user can customize the behavior of the plugin
  3. Wish I could use Convention Mappings, but then resist since they're not public
  4. Realize that I can't look at the Extension until afterEvaluate, which is after the user's code has had a chance to set values on the Extension
  5. Put all my plugins logic in project.afterEvaluate
  6. Swear at any @DeferredConfigurable plugins
  7. Fight the ordering of my plugins
When I say @DeferredConfigurable, I'm pretty specifically looking at the PublishingPlugin's DefaultPublishingExtension, which will throw an exception if you try to configure it after it's been configured. Honestly this make sense, but it doesn't make it any easier to use. The publish plugin adds it's own afterEvaluate which forces the extension to be configured, since it needs to create tasks before the task graph is built. 

This means that if I have a Plugin that uses an Extension which is going alter a Software Component, it HAS to run before the publish plugin is applied, so that my afterEvaluate is run before the PublishingExtension's afterEvaluate is, keeping in mind that afterEvaluate is evaluated in insertion order. But that's impossible since I can't configure the PublishingExtension until after the PublishingPlugin has added the PublishingExtension to the project. I think I've also seen issues where the IvyPublication is resolved before I can add in my generated/imported dependencies, which is another reason I need to run before the publish plugin's afterEvaluate.

Technically the publish plugin is @Incubating, so I'm not faulting the Gradle guys and don't even expect this to be a bug in its own right, since they're still figuring it out. In my cause, I require the ability to modify the module descriptor, and that's only really possible with the publish plugin, so I'll live with it. Though I'm starting to get the feeling that there's some fundamental ordering issues cropping up, which will eventually grow out of control.

My solution is implement a beforeAfterEvaluate method that lets me jump to the front of the dispatcher LinkedHashMap. Caveat, there's lot of internal classes being used, so I know that it can break in future versions of Gradle.


Loading ....

This doesn't answer the general ordering issue, but it can unblock a stalemate in the order of plugins. It's worth mentioning that Convention Mappings aren't the solution either, they're just an opportunistic lazy evaluation that "typically" works in your favor.

Saturday, April 27, 2013

Ahoy Matie!

At the last Java Posse Roundup, I gave a lighting talk on "Thinking with box" where I outlined some fun I have had with cardboard with my daughter. I'll link to the video once it's uploaded (I'm looking at your +Carl Quinn) . Here are some of the steps, with some tips and tricks, that I did to make it. (I've meant to write this up for a while, but I finally got a little push from @Chris Cobb which put me over the edge.)

Step 0: Surf the web

My wife was on Pinterest looking at pirate ideas for our little one's 4th birthday, and looking over her shoulder I saw a cardboard pirate ship that looked like a lot of fun. So I dug deeper by searching Google for "cardboard pirate ship", and wow, there's a lot of stuff out there that adventurous parents had done. I zero'd in on this picture pretty quickly.
Step 1: Build a Model

I'm fortunate to work at a company that has lots of spare TV boxes around, but I didn't necessarily know how to get them for myself, so I filed a Helpdesk ticket. They were fantastic in helping, they brought by samples to see exactly I was interested in. They even took a huge whiteboard out of a box, just to give to me. I had some great building materials, but I didn't want to waste any and I wasn't even sure if I had enough, so I built a model. 




This proved helpful in designing the large straight pieces, and more importantly the angle for the bow.

Step 2: Cutting

I'd say cutting cardboard is best done like drywall, where you score one side with subtle pressure, folding it, then cutting from behind. Before you score, I'd recommend getting a very long straight-edge and drawing a line where you're going to score, even if you're going to score against the straight-edge. I didn't have any rulers long enough, so I ended up using some wood scrap, which I'll talk about later. The cutting was primarily done with a utility knife.


Step 3: Attaching

The model showed me that I had to cut the bow as a trapezoid  but once I got to real cardboard I learned that there more complex angles going on in three dimensional space. But I was close enough that I thought I'd just start attaching everything. The original idea was to just staple gun two pieces of cardboard, that was a horrible idea. I don't even know why I thought that would work. The additional stress on the bow make it clear that I'd need more support, so I went back to Home Depot and picked up some wood scraps which I could screw into. The best way to describe it was 1/2" x 1". It's no more than a $1 per 8 foot piece (wood is cheap in California). You can seen an example in the picture for the mast.
I'd cut the strip to length, letting the little one pull down the chop-saw. To attach the cardboard, I put washers on a drywall screw then threw the cardboard into the wood behind it. The washers were necessary to prevent the screwhead from going through the cardboard. It was quite common for the screw to come through the wood, in which case I used a Dremil to cut it off the end sticking out. It wasn't perfectly smooth, but it was good enough for 4 year olds. If you zoom into the picture you might be able to  see the washers and wood at the intersections between pieces.

Step 4: Laying out

I worked from the bow to the back, and I honestly didn't know how each part was going to work out until I tried. The model really helped. 



I'd sometimes staple gun in between the screws to get a really good seal between pieces, but this probably wasn't needed.

Step 5: Cutting Windows

I cut a single window, then used it as a template for the other windows. I found that the utility knife had a tendency to go straight, so I used an X-Acto knife for corners. The thickness of cardboard meant I had to make a few passes before it went through, but tracing a previous cutout made it easy to follow the path. I'd usually duct tape a previous cut where I wanted the next window, so that it wouldn't drift as I cut around it.

Step 6: Painting



At most home improvement stores, they have previously mixed paint which has been returned and is now on clearance. You can get a gallon of decent paint really cheap. We did find that Home Depot was selling these clearance items at full price, our theory is that people were abusing the system so they needed to stop selling them so cheap, so we went to Lowe's instead. We let our daughter pick any color they had there, and we picked something to complement it. She wanted purple, but they only had pink. Painting cardboard proved to be really straightforward.


Step 7: Finishing touches

At this point it was looking fantastic, but we had already pretty much taken the day off of work, so we thought to fancy it up a little. My wife cut a large H and I put duct tape around the edges. And anchor was made by stapling loops of paper together. A sail was just an old sheet, which had to be cut or else the ship would literally sail away.




Wednesday, April 24, 2013

Seeing the rings

Right now is a great time to see the rings of Saturn, just follow the moon down and to the left, past Spica and the next bright thing is Saturn. Hmm. Let me rephrase that, it's a great time to see the rings of Saturn through a telescope. A few months ago we bought a telescope for the parents who took astronomy classes together the little one. We kept the budget manageable enough that we could get a nice telescope sometime in the future if this works out. We've learned some lessons, which might have been different if we had a $3,000 telescope:
  • Jupiter and Saturn are a rush to see
  • Stars are just dots in the sky, which you make big and blurry with a telescope
  • Four year olds have the patience to see one celestial object a night
  • Four year olds "get it" and are excited to see a blurry star even its just orange in color
  • The moon never gets old
I do think there's a limit to viewing. Once we saw Jupiter and the moon, there wasn't much more we could get that wouldn't be more enjoyable by just surfing images.google.com. But then we waited and saw Saturn. I think we're going to find that we bring it out every 6 months, see some new object in the sky and get a renewed interest for a week. And that's perfectly fine.

We're lucky that Orion is a local company, with a store in Cupertino. The whole family went over one Sunday and we brought home a very reasonably priced telescope. There's an argument to be had that we should have just bought really nice binoculars, especially given my above statement about bursty interest. There's just something special about having a real telescope in front of you. Our budget was <$400. To extend our budget, we opted to go without any tracking mechanism, so that we can could spend more on the optics. (It's fun to just talk about the optics, I understand optics better now than I ever did in Physics class, but I'll save that discussion for another post.) The other balancing act was portability, we wanted to be able to bring it to viewing parties and just whip it out from the garage when it's nice out.

In the end, we got an Orion SpaceProbe 130mm EQ, with a 900mm focal length. And after some initial fun we bought the Orion Telescope Accessory Kit for a few more length and some filters, including a moon filter. In both cases they were on sale in the store, so the total was around $300.



We like it a lot. It's not too big that I can't just leave it setup in the garage. It's advanced enough that I can play with different lenses. Maybe in the future we'll be able to justify $3,000 for a sweet Cassegrain, but for right now this works for us.