Tuesday, February 17, 2015

Rumblings on Distributed Systems


RoDS is about scalability, load balancing, and fault tolerance

But first a disclaimer: I am by no means an acknowledged expert in the field of reliability engineering.  This is merely a topic I've spent a fair amount of time reading, thinking, and practicing with, so hopefully someone might benefit from some random insight.

What are distributed systems?

Before coming to work for Boeing, I rode the crest of the 1990's tech wave at Ocean City, working at a supercomputing cluster start-up. We designed high-availability, fault-tolerant, scalable computing systems with a rather unique recursive network topology. They would often compare their reliability goals with the legendary reputation of Boeing aircraft, where double and triple redundancy would allow the plane to keep flying after multiple equipment failures. So I was kind of surprised when I did start working for Boeing, and did not find that philosophy pervasive in a lot of the work we were doing.

At the supercomputing company we would perform this demonstration where we'd start the cluster on an intensive task such as parallel raуtracing. As the nodes were working, we'd walk up to the machine and pull out various components -- network cables, power supplies, entire computing blades -- and show how the system would keep on running. The process would continue rendering - maybe show a hiccup a bit but go back and fill in the missing data.

A lot of my understanding and perhaps obsession with distributed systems was shaped by studying and designing for these types of computing components: RAID arrays, redundant power supplies, load balancing, etc. However, a lot of these patterns and considerations can be applied to many other fields, including products, systems, and people.

When most people make mention of Distributed Operations (and yes, that was the actual name of my workgroup), they generally mean it in the geographical sense, in that the network allows to decouple people, tools, and resources from particular locations. Let's spend some time musing over some of the many other senses of distributed architectures, however.

Saturday, June 22, 2013

100kmi

Well, here's a tribute to our car, which is about to hit 100,000 miles on its odometer.  (Apologies go out to all of the flesh-n-blood people that I've forgotten to wish happy birthday to lately)

Here are a handful of lesser-discussed features and details that the engineers and designers deserve credit for:

* Rear windows that go down all the way.  (most other cars I've been in won't retract more than 75% down)
* Red instrument panel lights to help preserve your night vision.  (newer versions of the car use blue)
* The most responsive tiptronic transmission I've seen in any car I've driven with tiptronic transmission. (most have a full second or so of lag before deciding to honor your request to shift, and abruptly shift automatically for you.)
* The automatic transmission mode promptly downshifts for you when it detects you riding the brakes going downhill.
* The automatic transmission shows you what gear the car is in on the instrument panel.  (most cars I've seen just show "D" until you switch to tiptronic mode)
* The tiptronic transmission toggles the "right" way to upshift / downshift (by pulling back / pushing forwards)
* Low center console doesn't obstruct the space between you and your passenger
* Tie-down hooks in the hatch area, since the thing can pull 0.85gs laterally.
* Braking performance of a sports car.  In every accident I've seen involving a Mazda3, they've always been rear-ended by someone behind them who couldn't stop as swiftly.  So whenever I have to jam on the brakes as traffic comes to a sudden halt, I spend more time worrying about what's happening in my rear-view mirror than what's in front of me.
* Never thought I would be a sunroof person, but I do really appreciate being able to look up at tall trees and buildings.

OK, so I have one nag:
* When I switch the the temperature control to cold to get some fresh air out of the vents, it automatically turns on the A/C and recirculation.  I just want fresh air that's not heated!

Tuesday, February 12, 2013

Parsing json in bash

So maybe you're wandering around and a giant blob of json jumps up at you on the street!

 If you're on RHEL6 / CentOS6 you might have json_reformat handy from the yajl package ("yet another json library").

 But say, maybe you're on an older RHEL5 / CentOS5 box, what do you do?  Well it turns out you can probably run it through python to split it out into something grepable.

curl -s http://cbserver:8091/pools | python -mjson.tool



{
    "componentsVersion": {
        "ale": "8cffe61",
        "inets": "5.6",
        "kernel": "2.14.4",
        "mnesia": "4.4.19",
        "ns_server": "1.8.1-937-rel-enterprise",
        "os_mon": "2.2.6",
        "sasl": "2.1.9.4",
        "stdlib": "1.17.4"
    },
    "implementationVersion": "1.8.1-937-rel-enterprise",
    "isAdminCreds": false,
    "pools": [
        {
            "name": "default",
            "streamingUri": "/poolsStreaming/default",
            "uri": "/pools/default"
        }
    ],
    "uuid": "43f16796-d591-409b-a831-889eeac86984"
}