I suppose it is fitting that this week I really felt like a grownup in my job. After all, yesterday marks two years in my position. I’ve gone from being a green, inexperienced admin to the point where my position is starting to become too small for me. So what happened this week? On Tuesday evening, I happened to have my e-mail client open when I received a message from our mail file server complaining that a hard drive had failed. The timing was awful, I had just settled in with my Law & Order and my martini. I drove in anyway, and yanked the drive. I put the shelf spare in, but that’s where the routine fell apart — our spare didn’t work!
My colleague Randy got my voicemail by that point and had hopped online to help me out. We tried a few times to get the spare to be recognized, but the controller never saw it. At that point, we decided to throw the old disk back into the array to see what would happen. The controller recognized it, and said it was good. Confused by this, we decided to add it back in as the spare. Except we gave it the wrong command, and ended up expanding the array by one, meaning we no longer had a hot spare. Bad news. Fortunately we have another array, so we just changed the spare in that array to be a global spare. So that gives us one spare for 23 disks.
Fortunately, nothing bad came from this mess, and it may actually turn out to be a good opportunity. On Wednesday morning, once we got everything settled, I started working on a plan. The current disks in our first array are 400GB. If we upgrade them all to be 500GB, we get an extra terabyte of storage, and the disks in both arrays will be identical, which means we can have two global spares, which means we could lose three disks and still not have any data loss. So what caused me to feel all grown up? Randy and I figured out a plan and covered all the bases. When we presented it to the rest of our team, they couldn’t come up with anything we hadn’t already considered.
Anyone can come up with a plan for things going well. A true mark of the sysadmin is the ability to plan for massive failures. And also paranoia and social ineptitude. 🙂
Well, my job kept me from my hobby this week (more on that tomorrow, maybe). But Beonard’s Losers is finally posted, with the first game 23 minutes from kickoff. You may enjoy it here:
Week 4 has been uploaded. It may be my best job yet of reading my script, but I’ve been battling audio problems all night, so its a little rough. If that bothers you, remember that I’m doing all this for free. You’re welcome to send me money. 🙂
One of the things I love the most about Linux is how insanely customizable it is. There are several mainstream distributions, each with their own strengths, but if you don’t like any of them you can always roll your own. If there’s a specialized use for Linux, it’s not surprising to see a distribution dedicated to it. Here are a few that probably don’t exist, but should:
- Obuntu – Ubuntu for Obama supporters
- Dr SuSE – very popular in Europe, and among those who speak in rhymes
- 50CentOS – an enterprise OS supported by the gangster community
- Little Debian – a distribution popular among snack cake enthusiasts
- Fedora Crime – a RedHat-based distribution used by Carmen Sandiego and her henchmen
There are plenty of other distros out there. If you have your own humorous play on words, feel free to leave it in the comments.
The eye of Ike is still nearly 40 miles offshore, but The Weather Channel’s field personnel are getting tossed around pretty well. I’ve been in contact with my friends in Texas. One is in Dickinson, and she’s reported measured wind gusts in excess of 90 mph. My other friend is in the northwest part of Houston, about 20 miles from KHOU. He says its starting to get exciting there, with winds around 40-45 knots.
The highest official observation I’ve seen is from the Galveston Pleasure Pier, which had sustained winds of 43 knots with gusts to 64 knots, but it hasn’t reported in nearly an hour now. Galveston is now completely without power, and the downtown area has four feet of water. Several large fires are going, and emergency responders along the Texas coast are not currently responding to calls. We’ll see what the morning brings.
In 1900, Galveston, Texas experienced the deadliest hurricane in U.S. history. 108 years later, MSNBC is reporting that perhaps as many as 23,000 people in the city have chosen to ignore a mandatory evacuation order. Hurricane Ike may be “only” a Category 2 storm (with winds currently at 110 mph), but the big story here will be the storm surge. The reports I’ve heard so far indicate a 20 foot surge in Galveston, which is several feet higher than the seawall. The first floor of many buildings are already flooded, and the eyewall won’t be ashore for several more hours.
If there is anything good about this storm, it is the fact that it won’t stall out over Texas. HPC forecasts still show 13-plus inches of rain for the coastal area in the immediate vicinity of the eye, it quickly drops to 3-4″ into east-central Texas. In fact, a Springfield, MO-Detroit line is forecast to receive more rain than much of Texas.
I learned something new today, too. Jeff Masters had a write-up earlier about a measurement called ” Integrated Kinetic Energy (IKE).” Ike’s IKE is much higher than that of Hurricane Katrina. So what is IKE? Basically, it is a measure of the total energy in a hurricane. The numbers are incredibly large, too — earlier this afternoon Ike’s IKE was 149 TeraJoules, the equivalent to more than 35,000 tons of TNT. The higher the IKE, the higher the potential storm surge. I can see how the sum total of the energy in a storm is important, but I’d personally like to see the energy expressed in a density of sorts. TJ/km^2 or something. I mean, over a big enough domain, you could have a 149 TJ IKE with nothing but fair-weather cumulus. But I don’t know much about tropical weather, so maybe I’m out to lunch?
Anyway, best wishes to everyone in eastern Texas, and especially to my two friends in the area.
I’ve tried to improve on perfection. Fourteen games are covered in this week’s issue of Beonard’s Losers (http://www.funnelfiasco.com/fun_games/losers/2008w03.html). Do I have what it takes to make it two perfect weeks in a row? We’ll find out this weekend!
I got a call from my sister in eastern Virginia last night. She was pretty disappointed with Hanna. Southern Indiana has thunderstorms that are more intimidating, apparently. The results of the Hanna contest have been posted to http://funnelfiasco.com/weather/tropical/game/2008-hanna.html. The top score isn’t quite as good as for Gustav, but the worst scores are far better. Well done, everyone.
I dread scoring the Ike results. At least the Atlantic is stopping to catch its breath for a bit.
Well, in my bleary state of half-awakenness, I’ve upged the Nugget Night website (http://www.funnelfiasco.com/ben/cfa.html). This was a fun one. We got to meet a new employee and scare her away, and I got to meet challenges set before my. One friend challenged me to eat a nugget for every point Purdue scored at the game. Another challenged me to eat a nugget for every yard that Curtis Painter threw for. The totals were 42 and 57.2, respectively. When I hit 42, I just felt like going on, so I stopped at 58. I think the gallon of Powerade (that may be hyperbole, but I was oh so very thirstsy) I had limited my ability to eat nuggets.
The next one will be 30 in a row. That pretty much makes me the most awesome person ever.
The Atlantic’s rage continues. With Hurricane Ike (currently a category 4!) churning toward the Bahamas, and eventually Florida, it’s time to open a game for him. Submit your forecast at: http://funnelfiasco.com/cgi-bin/hurricane.cgi?cmd=view&year=2008&name=ike.
In Hanna news, it looks like I totally blew my forecast. The good news is that she is no longer expected to regain hurricane strength. That’s good news for the entire east coast, but especially my sister. I’m very jealous that she gets to ride out a tropical storm before I do. Just a few minutes ago, I sent her a list of last minute preparations. Hopefully she’ll follow them.