I owe my career to Unidata

You may wonder how a site called “Funnel Fiasco” came to have so much technology content. It all traces back to an email I sent my freshman year of college. But it’s also directly attributable to the work done at Unidata. Funded by the National Science Foundation for decades, Unidata is a cornerstone of atmospheric science education, providing software and data services. Tragically, Unidata furloughed almost all staff on Friday thanks to the assholes running the government.

A fateful email

Early in my freshman year, Dr. Jon Schrage was giving a tour of the Earth & Atmospheric Sciences facilities in Purdue’s Civil Engineering Building. (Ed note — Earth & Atmospheric Sciences is now Earth, Atmospheric, and Planetary Sciences. The Civil Engineering Building is now Hampton Hall. I will use the names as they were in my time as an undergraduate.) He mentioned that he’d be doing a training on the WXP software soon if anyone wanted to learn how to use it.

Reader, I very much did. So I showed up to the Civil Engineering Building on a Wednesday night. For the next two hours, I learned how to use WXP to make weather maps. At the end of the session, Jon mentioned that he was a visiting professor and his appointment was up at the end of the year. He didn’t know who would be maintaining the software the following year.

When I got back to my room, high on the thrill of weather plots, I sat down and sent him an email. With all of the confidence of a mediocre white man, I sent this: “I’m just a freshman who doesn’t know what he’s doing, but I’ll do it.” It’s been almost 24 years, but I’m pretty sure those were my exact words.

Did I know how to use Unix (specifically FreeBSD)? No! Did I know anything about the software? No! Was I going to let that stop me? Absolutely not.

Amazingly, the department hired me. That got me through my undergraduate years and set me up to accidentally fall into a career in technology. I’d say it has worked out pretty well so far.

Where Unidata fits in

The astute reader may notice that so far the tale centers on my overconfidence. So where does Unidata fit in?

Unidata created and maintains the Local Data Manager (LDM) software. LDM allows universities and other users to reliably share meteorological data in near-real time. From models, to observations, to satellite images, to radar data, LDM provides a robust transport mechanism. A big part of my job was administering the software and providing help to students and faculty who needed data.

The department flew me to Boulder for an in-person training workshop where I learned LDM in greater depth. Later on, I returned to Boulder for training on GEMPAK, another weather visualization and analysis suite.

The software and the training helped me become a valuable contributor my department’s education and research missions. This is what led to me getting a full-time Linux sysadmin role the summer after I graduated. No doubt there are many others like me out there — not to mention all of the forecasters and researchers who learned about the atmosphere with the help of Unidata’s work.

The Unidata staff — as well as so many other federal grant recipients, contractors, and employees — deserve far better than this administration has given them.

Building GEMPAK on Fedora

People who have training or experience in following severe weather rarely are content to rely on the media for severe weather information.  This isn’t to say anything against the TV and radio outlets, but we would just rather see the data for ourselves and make our own decisions.  A number of software packages exist for analyzing weather data, of varying price and quality.  For Linux and OS X users (and Windows users via Cygwin), perhaps the most powerful is GEMPAK, developed by the National Weather Service and made available by Unidata.

GEMPAK is basically what national centers, including the Storm Prediction Center and the National Hurricane Center currently use for data analysis and visualization.  It is also a pretty old suite, written in pretty old Fortran, and kind of a bear to get installed sometimes.  Unidata formerly provided pre-built binaries, but only source code is available for the 5.11.4 release.  Since there seems to be an absence of step-by-step instructions, I thought I’d post my own.  Of course, as soon as I post this, a public release of AWIPS II will be announced.

The first step is to prepare your environment.  This basically consists of creating the appropriate user and/or directory structure.  For simplicity and security reasons, I prefer not to create a separate GEMPAK user.  I prefer to just build it in a directory that I have access to and then move it to opt when I’m done.  If your account will be the only one using GEMPAK, you can just leave it in your home directory.  For the sake of this demonstration, let’s assume we’re doing it my way.  Then there’s no pre-work to do.

Next you’ll need to download the GEMPAK source code from Unidata’s website.  Unidata requires registration to download the software, but it is free and they don’t harass you.  While you’re waiting for the tarball to download, you’ll need to make sure you’ve got a few of the packages necessary for building GEMPAK.  Most of them can be found in the yum repository, so you can install them with:

su -c "yum install gcc-gfortran gcc-c++ libXp-devel  libX11-devel xorg-x11-fonts-ISO8859-1-75dpi"

You’ll also need OpenMotif packages, but they can’t be newer that 2.3.1.  This means you cannot install the packages from the RPMFusion-nonfree repository.  You’ll have to grab them manually from MotifZone.  There aren’t builds for recent Fedora releases on there, but the Fedora 9 RPMs work fine with Fedora 12.  Since we’ve installed these RPMs, you need to make sure yum won’t try to upgrade them, so make sure you exclude “openmotif*” in /etc/yum.conf.

By now, the tarball should be downloaded.  You’ll need to unpack it just like you would any other. Let’s assume you downloaded it to ~/Download, so go to the directory you want to build GEMPAK in and run:

tar xfz ~/Download/gempak_upc5.11.4.tar.gz

Now cd into the GEMPAK5.11.4 directory you just created.  The first thing you’ll need to do is edit the Gemenviron.profile (if you use the Bash shell) or Gemenviorn (for C-shell).  This file sets the myriad of environment variables that GEMPAK uses.  Fortunately, it is sanely built, so you only need to make a few changes.  Change the NAWIPS variable to point to the directory you’re building in (for example, $HOME/GEMPAK5.11.4).  USE_GFORTRAN should already be set to 1, but double-check that it is and that the references to USE_G77 and USE_PGI right below it are commented out.  Eventually, you’ll also need to make sure the GEMDATA and LDMDATA variables are set correctly, but that’s beyond the scope of this how-to.

After you’ve saved the Gemenviron[.profile] file, you’ll need to make sure config/Makeinc.common is correct (it probably is) and you’ll need to edit config/Makeinc.linux_gfortran (or config/Makeinc.linux64_gfortran if you’re on an x86_64 system. You can check that with `uname -i`).  Change the MOTIFINC variable to “-I/usr/include/openmotif” and change X11LIBDIR to “-L/usr/lib/openmotif” (replace “lib” with “lib64″ for x86_64 systems).

There’s one more step to get things working, which appears to be a result of Fedora 12 using a newer version of autotools than the source was prepared with.  You”ll need to run the following command in both the extlibs/netCDF/v3.6.2 and extlibs/JasPer/jasper directories (return to the GEMPAK5.11.4 directory when you’re done):

autoreconf --force --install --symlink

Now that that’s done, you’ll need to load the GEMPAK environment.  For the Bash shell, use `. Gemenviron.profile` and for the C-shell use `source Gemenviron`.  The next step is to actually build the programs.  It is a lengthy process (I timed it at about 8 minutes on my system), so you might want to go get a cup of coffee or something.  Since it produces a lot of output, we want to make sure we can go back and look through it if there are any problems (it took about 5 build attempts before I got these instructions to a workable state), so we’ll save the output to a file called make.out.

make >& make.out

Once that’s completed, it’s time to install the files and clean up after ourselves:

make install; make clean

If you plan to move the built GEMPAK into somewhere like /opt, now’s the time to do that (make sure you update the NAWIPS variable in Gemenviron[.profile] appropriately).  I prefer to move it to /opt and then create an /opt/gempak symbolic link to the GEMPAK5.11.4 directory so that if I install a different version, I just need to change my link and everything else works the same.  For ease of use, you should set your shell configuration file to call the Gemenviron[.profile] script at login if you plan to use GEMPAK frequently.

That’s all there is, it’s ready to use now.

Edit – 7 October 2010.  Instructions for Intel-based Mac OS X builds have been written by Kevin Tyle and posted to the Gembud mailing list.

Edit – 9 October 2010. Added the gcc-c++ package per Gerry Creager’s suggestion.