New entries in the Forecast Discussion Hall of Fame

Radar estimates of wind speed aren’t always the most reliable for a variety of reasons, but on Friday, forecasters in Memphis, TN opted to believe the radar instead of the surface observation. Maybe because the Millington station was reporting a 343 MPH wind gust.

NWS forecasters are public servants dedicated to preserving life and property. It should come as no surprise that they are sometimes moved by uncontrollable bursts of patriotism. Chris Hattings in Riverton, WY felt very Jeffersonian on Independence Day.

Both of these discussions have been added to the Forecast Discussion Hall of Fame.

Regional weather forecast offices?

Update: The Senate Commerce Committee has amended the bill to remove the regionalization, according to The Washington PostThe bill now focuses on improving how weather hazards are communicated to the public (an effort that is already underway in both the public and private sectors). At the time of this update, it is unclear whether this morning’s post was what convinced Senator Thune to change approaches.


 

Last week, The Washington Post‘s Capital Weather Gang blog reported on a bill being introduced by the chair of the Senate Commerce Committee. The bill, dubbed “The National Weather Service Improvement Act”, would direct the National Weather Service to consolidate the current 122 weather forecast offices around the country down to six. Although the bill itself is light on details, supporters say it is designed to reduce under-staffing during severe weather outbreaks and it is apparently expected to save money (since the bill has explicit provisions for what is to be done with any savings).

The reactions among my meteorologically-inclined friends were mostly dismissive. Tim Cermak’s thoughts were fairly representative:

Others privately suggested the bill was solely political posturing. No one in Congress would want to be the one to shutter the local NWS office. Still, were the bill to become law, several people expressed concern about the loss of local knowledge.

There are some definite benefits to consolidating forecast responsibilities. Forecast area boundaries lead to discontinuous forecast grids and ugly warning polygons. It stands to reason that fair weather forecasts could be handled at the regional level. But what do the local office forecasters do while they wait for a significant weather event?

If I understand the text of the bill correctly, there are no local forecasters. Only the Warning Coordination Meteorologists would remain at local offices (and presumably electronics technicians to repair radars and other equipment). This would mean the loss of local knowledge which can be key during severe weather events. Local forecasters get to know the locations of small towns in their forecast area better than regional forecasters could. This is important not only for writing warning text, but for contextualizing storm reports.

Ubiquitous broadband Internet has removed some of the need for co- or near-location with data sources such as radars. The ham radio spotter networks would have some trouble, but that could be addressed. One concern I haven’t seen raised is what happens when an office has to take shelter. It’s one thing for an office to hand over responsibilities to surrounding offices for a few minutes as a tornado bears down (or a few hours because phone lines are down). This becomes more difficult (though likely more rare) when suddenly one sixth of the forecast offices drops offline.

Most of the thoughts above are not new. Many of them come from the National Academy of Sciences report upon which the Senate bill is supposedly based. The NAS report has a few key differences, though. Most importantly, it describes regional offices as one of several possibilities (another of which is to retain the status quo). The NAS report also explicitly keeps open local WFOs for hazardous weather, marine, and aviation products. The Senate bill (again, based on my reading) implies that these functions would also be regionalized.

Consolidating forecast operations is not without precedent; the U.S. Navy consolidated its various forecast units in 2011. I could not find any follow-up studies that evaluated a change in effectiveness or forecast accuracy. If any such studies were conducted, they may be classified.

It’s no secret that I’m generally a fan of the National Weather Service. I believe it is a steal for taxpayers. That’s not to say that there is no room for improvement. The NWS must be open to constructive criticism and changes that can improve its ability to perform its mission to protect life and property. This bill does not represent that. I’m not familiar enough with the politics to guess at the driving factor, but it seems to cherry pick from the NAS report without regard for the end result.

Tornado warning false alarm rates

Five Thirty Eight recently ran a post about the false alarm rate of tornado warnings. Tornado warnings fail to verify (i.e. have no tornado) approximately 75% of the time, a number that has held steady for years. This comes as no surprise to meteorologists, and probably not to the general public. What’s disappointing about the article is that it doesn’t address the reason that the false alarm rate hasn’t improved: because it’s not a priority.

The ideal case, of course, is a false alarm rate of zero. While the article quotes the reasoning (“you would rather have a warning out there and have it miss than have an event and not have one out there”), it doesn’t explain why that reasoning leads to a high false alarm rate.

The first reason is that an emphasis on maximizing detection means that in questionable scenarios, forecasters will lean toward issuing a warning instead of not. I’ve been in an office when an unwarned tornado has been reported. The forecasters are not happy about that. They take the National Weather Service mission of protecting life and property seriously. The impact of a false alarm (inconvenience and lost productivity) outweighs the potential loss of life from a missed event.

After inadvertently posting this when I meant to save the draft, a friend commented that the “ideal FAR is actually non-zero if you want lead time.” This leads to the second reason an emphasis on detection increases the false alarm rate. Issuing a tornado warning seconds before the tornado hits is of limited utility. People in the warned area need time to move to safety. The article does point out that lead time has increased steadily for the past few decades. But the more lead time you have, the more likely it is that a warned storm won’t produce a tornado. Tornadoes are exceptional events.

There’s a balance between detection rate and lead time on one side and false alarm rate on the other. Like a seesaw, lowering one side raises the other (if you play with the signs on the numbers, that is). Prudent policy focuses first on detection and then on lead time, so the false alarm rate has to suffer. Improvements in technology and science will hopefully move the fulcrum such that we can lower the false alarm rate without reducing the lead time or probability of detection too much.

Severe weather outlooks on TV

At the end of October, the Storm Prediction Center changed the categories used in severe weather outlooks in order to more clearly communicate risk. These outlooks, like many NWS products, started as a way of communicating information to other meteorologists, emergency managers, et cetera. Though they weren’t designed with public consumption in mind, social science has helped to shape some of the changes. The Internet means that weather products are available to anyone who is looking for them.

What I’ve noticed since then is that not all of the local TV stations have gone along for the ride. A while ago, I asked one of the local meteorologists about this. His station discussed it internally and decided fewer categories made for less viewer confusion. I don’t have any reason to dispute that.

Severe weather outlook from the Storm Prediction Center.

Severe weather outlook from the Storm Prediction Center.

Severe weather outlook from NBC affiliate WXIN.

Severe weather outlook from NBC affiliate WXIN.

Severe weather outlook from CW affiliate WISH.

Severe weather outlook from CW affiliate WISH.

Severe weather outlook from Fox affiliate WXIN.

Severe weather outlook from Fox affiliate WXIN.

My main concern isn’t that the station doesn’t use the same categories as the SPC, but that different stations in the market use different categories. Of course, they should do what they think is in the best interests of their viewers. I’m certainly not suggesting there be mandatory unification. At the same time, I think stations having different risk categories is more confusing to the public than adding “marginal” and “enhanced” categories. Then again, TV weather seems to be one place that people have a specific and unwavering loyalty. Outside of weather weenies, I’m not sure there are too many people who would even notice differences between stations.

Communicating weather safety information

Weather is complicated and hyper-local. The general public often lacks a basic understanding of weather evolution and people are generally bad at risk assessment. These facts combined make it really hard to provide general safety advice. It’s made even harder by the fact that if you give bad advice, you may be responsible for injury or death.

What to do when you’re in a car and a tornado is coming is perhaps the epitome of this issue. The National Weather Service office in Kansas City recently posted a scenario to its Facebook page. I saw some dismay expressed about how many people said they’d keep driving in that scenario. But here’s the kicker, I think that’s (conditionally) the right answer.

In the scenario you’re smack in the middle of a six mile stretch of interstate highway that’s expected to be impacted by a tornado in 15 minutes and you’re at an exit. The overpass is clearly the wrong answer. A very good answer would be to go to one of the gas stations or restaurants in the picture and seek shelter there. A car is about the worst place to be in a tornado, so why did I say “keep driving” is the right answer?

Let’s assume you’re traveling at 60 miles per hour. In three minutes, you’ve reached the edge of the warned area. The tornado won’t reach that area for another 12 minutes. Of course, there’s likely some error in the projection, but even if the forward motion is twice what was stated, you still have a cushion of over three minutes. If, in addition, the danger area is twice as large as stated, you still have 30 seconds. That’s cutting it too close, but we’re being really conservative here.

Now let’s look at all of the underlying assumptions that I made. First, I assume that you can safely travel at normal speed the necessary distance. This means no traffic, accidents, construction zones, or debris from earlier storms. In some places, you’d probably have sufficient visibility to make that determination, but certainly not in all places, and not in the picture shown. Second, I assume that you are just passing through. If you’re 10 minutes from home, it might be tempting to try to get there, but that eats into a lot of your safety buffer. Third, I assume you’re traveling south or that the main part of the supercell (another assumption) does not contain heavy rain or large hail that would slow you down or cause damage/injury on its own.

What would I do in that situation? It would depend on my familiarity with the area, my awareness of the storm type and evolution, and (most importantly), my ability to process it all quickly enough.

What should you do in that situation? See above. The best default answer is to seek shelter in one of the buildings off the exit, but that’s not always the best answer.

NWS TV studios?

The Washington Post’s excellent Capital Weather Gang blog had a post earlier this week about a new addition to the Birmingham, AL National Weather Service office. It seems that WFO Birmingham has added a high-quality setup for video recording. This has some private sector broadcasters worried that the NWS will be horning in on their business.

Let’s be honest: the NWS is hardly a digital juggernaut. Even if they wanted to, I don’t see the NWS doing live streams during severe weather events any time soon. But as times change, so too must the function of the NWS. The old model where the NWS generated products, but the media did most of the dissemination to the public is not suited to the 21st century (despite the fact that many NWS products are still in a 20th century paradigm). The public can easily connect directly with the NWS, and the nobody is served by hamstringing the NWS due to tradition.

The NWS has a mission to protect life and property. In a multimedia world, higher-quality weather briefings, interviews, and educational material advances that mission. If a broadcast meteorologist finds this disconcerting, they should seriously evaluate how much value they’re adding for their audience.

Simplifying winter weather products

Decades after the National Weather Service began issuing watches and warnings, many members of the public don’t know what the difference is. When you throw in different products, the confusion only mounts. Too often, the products are based on meteorological distinctions that don’t necessarily mean much to the public. Take, for example, a nor’easter that struck New England in December. Or the confusion around the landfall of Sandy, which became extratropical shortly before landfall.

Some products you might see in a winter event include blizzard, winter storm, high wind, wind chill, ice storm, lake effect snow, and freezing rain. Plus flood products and special weather statements. How should the public try to understand these differences?

In general, I’m a proponent of getting the important information to the consumer as quickly as possible with minimal effort required. This case is an exception. Trying to cram the important information into the headline leads to public confusion and forces forecasters to spend time trying to decide which of a handful of products are correct instead of focusing on communicating impact.

I’m in favor of a smaller set of products, with specific impacts delineated in the text. A “winter storm” and “blizzard” product with watch, warning, and advisory (maybe) levels would go along way toward making the products more clear to the public. Everyone could spend less time thinking about the differences between the products and more time focusing on the impacts and preparedness.

If you’re interested in the official specification for the current suite of winter products, see http://www.nws.noaa.gov/directives/sym/pd01005013curr.pdf

NWS products are not ready for public consumption

Decades ago, dissemination of National Weather Service products was largely done via third parties, particularly broadcast media. Then along came the Internet and suddenly NWS products became readily available to the public at-large. This should have been a benefit, but the products have not adjusted to this new paradigm.

Forget that text products are still in all-caps (I’ve found that I have a harder time reading discussions that are in mixed case). Severe weather warnings give information out of order. Warnings and even regular forecasts suffer from discontinuity at forecast area boundaries. Worst of all, forecasts do not convey uncertainty, instead providing a single number instead of a possible range.

The snow storm that hit (to one degree or another) the east coast this weekend is an excellent example of how forecast uncertainty was not well-communicated. In some areas, the forecast was quite accurate. In others, snowfall predictions were far too high. The forecasters knew there was a high degree of uncertainty about the forecast, so why did the public and civic leaders?

It’s hard to fault individual forecasters. They work hard within the system to produce valuable forecasts for the American people. It’s the management and technology that prevent the message from getting out. In recent years, the industry (including the private sector) has begun to understand the need for social science to accompany meteorological science. Hopefully this new focus will help to make products for the modern public.