Changes to weather radar rolling out this spring

Since this week is Severe Weather Preparedness Week in Indiana, I figured it’s a good time to have a weather post. The National Weather Service is rolling out some changes to the 159 NEXRAD weather radars sites across the country. These changes affect the Volume Coverage Patterns (VCPs) – how the radars scan the sky.

How weather radar works

To put it in the simplest terms, radars work by sending out pulses of energy and listening for the echos. The radar antenna rotates in a circle in order to get a view all around. But it doesn’t just move in a circle. The antenna also tilts upward. By moving up through increasing tilts, the radar eventually gets a 3D image of precipitation.

The key word here is “eventually”. The slowest VCP takes about 10 minutes to complete a full scan. This is generally used with clear skies or light wintry precipitation. The slow speed allows for more sensitivity and saves wear on the radar’s mechanical parts. But even the fastest scan modes take 4.5-5 minutes. During rapidly-evolving severe weather events, that can be a long time.

The changes

This spring, the weather service is rolling out changes that will introduce two new VCPs. Critically, the new software build will also remove four existing VCPs. By reduce the total number of options, forecasters will have to spend less time thinking about which radar mode to select so they can spend more time interpreting the radar data.

One of the new VCPs is focused on general precipitation and is designed to include the best parts of the patterns it replaces. The other is a new clear air pattern that shares common scan elevations with the precipitation modes and can be used for non-convective precipitation. The NWS has a paper describing the new VCPs in greater detail.

The changes will happen via software updates planned to begin later this month or in early April. It may take some time to know what the daily impact of the new patterns is. Still, it’s good to see that over 25 years after the first operational NEXRAD was deployed, the system is continuing to evolve.

Reflectivity tags

Sometimes you don’t notice something until it is pointed out to you, then you see it everywhere. That was the case for me when the near-ubiquity of clear slots near tornadoes was pointed out a few years ago. Suddenly, I began to (legitimately) see them everywhere. The feature was there, just completely overlooked. I expect a similar effect after learning about reflectivity tags at the Central Indiana Severe Weather Symposium in March.

Reflectivity tags are hardly new. The concept appears to have been introduced in a 2006 paper by Llyle Barker. The basic idea is that a small blob of reflectivity overtaking an area of rotation is often an indicator of tornado formation or intensification. Ed Shimon’s presentation at the Symposium pointed out how the Washington, IL tornado of November 17 grew from a small tornado into a neighborhood-leveling monster when the reflectivity tag passed.

There’s a danger in over-relying on the new shiny you’ve just picked up of course. The vast majority of storms still don’t produce tornadoes. I suspect that the majority of storms that feature reflectivity tags also don’t produce tornadoes. The presence of a fast-moving tag shouldn’t mean immediate panic. At the same time, it’s another piece of information to consider when watching storms.

Advancing science with your smartphone

The network of Doppler radars used by the National Weather Service is a powerful tool for forecasters. It can detect the intensity of precipitation, the motion of (and in) storms. With the ongoing deployment of dual-polarization upgrades, radars can even detect the type of precipitation. But radar can’t detect precipitation at ground level. In the past, the NWS has depended on a small number of trained volunteer spotters to provide “ground truth.” A recent project at the National Severe Storms Laboratory (NSSL) aims to expand the number of precipitation reports.

Called “PING”, the Precipitation Identification Near the Ground project uses mobile phone apps and a website to make it easy for members of the general public to provide immediate feedback on precipitation conditions. Feedback is available to the NWS and the public. The apps are available for free and present an easy interface. Expanding the available pool of spotters will be of great benefit to scientific understanding and to the warning process in general.

Mobile radar page updated

After some effort, I’ve completed my to-do list for the mobile radar page and decided to call it version 1.0.  It is now available from the Mobile Weather page (http://weather.funnelfiasco.com/mobile/).  A mostly complete list of changes is below:

  • Bugfix: Fixed problem with selecting alternate products. An update in the previous version caused the site variable to not be set correctly when an alternate product was selected at the bottom of the page.  This has been fixed.
  • Added adjacent site dial. Toward the bottom of the display page, there is now a dial to select the same product from an adjacent site (if it exists).  This is really handy for times when the area of interest is right on the edge of a site’s coverage.
  • Images now have a file extension. Previously, images were saved without an extension.  This wasn’t really a problem unless you wanted to right-click on the image.  All images are now displayed with a .gif extension, even though some of the static images are actually PNG files.  This does not appear to have any adverse effects.
  • Site name now in the headline. The name of the site, as well as the ID is now given in the headline along with the product type.

NWS mobile radar page updated

I put a little bit of effort into the NWS mobile radar page this afternoon and am proud to announce that it has been bumped to version 0.2b.  It is now available from the Mobile Weather page (http://weather.funnelfiasco.com/mobile/).  A mostly complete list of changes is below:

  • Added site selection menu.  I didn’t think it was necessary initially, but several users have suggested it, and my own experience while on vacation proves that there probably aren’t too many people who know the ID for all 154 sites.  The added bonus is that I’ve begun support for a requested feature, which is to select adjacent radar sites.  The difficult part will be filling that information in for each site, so it will likely be a gradual roll-out.  Sites can still be selected manually, which is probably quicker if you already know the ID.
  • Added license information.  In line with the rest of the website, the code for this script is licensed under the CC-BY-NC-SA 3.0. That information is now contained in the comments as well in the code output.
  • Put site selection and product selection on separate lines.  This is a small tweak to (hopefully) improve usability.  While horizontal real estate is constrained, there’s more room to separate things vertically on most devices, so let’s go with that.  If nothing else, users are used to vertical scrolling.
  • Added spacing of other products under radar image. Another usability tweak, which should make the clicking process a little bit simpler, especially on touch-only devices.
  • Change radar image label from <p> to <h3>.  This makes the site and product a little more visible and adds some vertical spacing to keep things from looking too jammed together.

Presenting: Funnel Fiasco mobile weather

You may recall on Saturday that I mentioned some big things that were coming.  Fortunately, you don’t have to wait long.  I’m proud to announce that an idea I’ve been thinking about has finally been realized this weekend.  Without further ado, the Funnel Fiasco mobile weather site. The idea behind this site is simple: the National Weather Service makes a lot of data available but it isn’t always in a mobile-friendly format.  Even the NWS mobile page has some bad navigation (and more importantly, doesn’t include velocity data, which is very important to chasers).  All I’ve done is to re-package the data in a way that I want to see it when I’m away from the computer.

All of the data is mirrored and hosted locally to minimize my impact on the NWS servers (and thus save taxpayer money!).  The local storm reports (LSRs) are grabbed by a cron job every 10 minutes.  I use the comma-separated value (CSV) files hosted on the Storm Prediction Center’s storm report website.  The CSVs are parsed by a Perl script I wrote and then a static HTML page is generated.  For the radar data, the images are mirrored on demand and a Perl script generates the output on the fly.  The radar data piece is a fairly heavy-duty script (by my standards, at least) and so I still consider it in beta.  For now, it actually runs on my server at home and not on the main funnelfiasco.com server.  I plan to move it onto funnelfiasco.com (and hope it doesn’t kill my bandwidth limits) after further testing.

I have to say, I’m pretty proud of the work I’ve done, almost all in the space of a weekend.  It’s nice to be able to add some useful content to my site.  I hope that it will get some good use and continue to grow.  As I come across data that can be easily manipulated, I’ll add it to the site.  Of course, if you wonderful readers have data you’d like to see, please let me know.

Why disk utilization matters

Here’s a rare weekend post to help make up for my lack of blogging this week.  Once again it is work related.  My life is boring and uneventful otherwise. 🙂

Unless you plan on sitting around babysitting your servers  every minute of every day, it is probably a good idea to have a monitoring system like Nagios set up.  My department, eternal mooches that we are, opted to not set one up and instead use the service provided by the college-level IT staff.  It worked great, until one day when it didn’t any more.  Some config change hosed the system and the Nagios service no longer ran.  I didn’t consider it much of a big deal until about 7 days ago.

This time last week, I was enjoying a vacation with my beautiful wife in celebration of our 2nd wedding anniversary.  When I got home Sunday evening, I noticed that several people had sent in e-mails complaining that they couldn’t log in to their Linux machines.  Like a fool, I spent the last few hours of my freedom trying to resolve the issue.  We figured out it was a problem with the LDAP server.  Requests went out, but no answers were ever received.  So after a too-long e-mail exchange, we got a workaround set up and I called it good enough.  I went to bed at one o’clock, thoroughly exhausted.

The next day we started working on figuring out what was the problem.  At first it seemed like the issue was entirely with the LDAP server, which is run by the central computing group on campus.  I was pleased that it was not one of my systems.  Then they noticed that there were a lot of open connections from two of my servers: one was our weather data website, and the other was our weather data ingest server.  Both machines work pretty hard, and at first I thought maybe one of the image generation processes just choked and that tripped up everything else.

Further investigation showed that the root cause of the issue was probably that the data partition on the ingest server was full.  This caused the LDM processes to freak out, which resulted in a lot more error messages in the log, which then filled up /var.  Now the system was running so slowly that nothing was behaving right, and since the web server is tightly married to the data server, they both ended up going crazy and murdering the LDAP server.

Now there are scripts that are supposed to run to scour the data server to keep the disks from filling.  I thought perhaps something had kept them from running.  I looked through logs, through cron e-mails, and then ran some find commands by hand.  Everything suggested that the scouring was working as it should.  The more I looked, the more I realized it’s just that the radar data is ever-growing.  I just need to add more disk.

Had I been keeping an eye on the disk usage these past few months, I would have known this sooner, and been able to take care of it before critical services got beaten up.  I think on Monday, I’ll lend a hand getting the Nagios server up and running again.  Learn from my mistakes, readers!