NWS warnings still make me sad

Longtime readers of Blog Fiasco know that I have some opinions about how the National Weather Service (NWS) issues and communicates warnings. Just check out the “warning” tag on this site if you’re interested. But it turns out that I have as-yet-unwritten opinions. This post is inspired by a recent tweet from an NWS meteorologist:

I mentioned in a previous post that warning products tend to be technically correct instead of useful, as in the case of the non-hurricane Sandy. This is a fine example. Tornadoes over water are waterspouts and the NWS treats maritime areas (including larger lakes) differently than land areas. The end result is that forecasters are unable to properly communicate threats to the public. This is harmful.

I understand treating land and water areas differently. A storm that is unremarkable on land could be deadly to watercraft. Special marine warnings are usefully distinct. But a tornado warning over water can be useful, too, particularly to folks on land who happen to be downstream. But forecasters aren’t allowed to provide that information because it’s not technically correct.

The National Weather Service is a great agency. The dedicated forecasters are at work around the clock to provide life-saving (and life-enhancing) forecasts and warnings. I just wish it would get out of it’s own way on this issue.

The role of privatized weather warnings

Last week, the Washington Post‘s Capital Weather Gang blog ran an article titled “U-Md. used a private company for a tornado warning. That can be problematic.” They’re right, but the point gets lost in the article. By presenting a laundry list of the times AccuWeather got a forecast wrong and ignoring missed warnings from the National Weather Service, the post ends up reading like a hit piece.

I am unabashedly a National Weather Service fanboy, but I see an important role for the private sector in the weather ecosystem. Despite my general dislike for AccuWeather, I have no problem with universities working with them. They can provide a degree of hands-on service that the NWS is not equipped to provide. This includes warning-like products to augment the NWS products.

My only objection is to the use of “watch” and “warning”. It’s hard enough to get the public to understand these terms. Adding similarly-named products from other sources will not help. A Weather-Ready Nation requires a cooperative effort between public and private sector meteorologists. Private companies are free to give their customer severe weather warnings, I just wish they’d use a different name.

Please don’t argue with the warning system

“Please don’t argue with the warning system”, Indiana University told a lecturer from its meteorology department as he rightly criticized their communications on Sunday.

Despite being wrong, the university continued to insist that they were making the right choice. Now as a Boilermaker, I’m normally in favor of Indiana University embarrassing itself. But this time, it’s just bad. Warning fatigue can kill people. The false alarm rate is already too high; telling people about warnings that don’t exist only makes it worse.

The “warnings affect the entire county until notified otherwise” statement is only a decade out of date. But I get it, our warning dissemination technology hasn’t caught up with how warnings are issued. You may recall I’ve written a few words on the subject.

The fact that dissemination technology is still (mostly) stuck in a county-based paradigm 10 years after the nationwide implementation of polygon-based warnings is an embarrassment. Emergency management is more than just weather, so I don’t expect emergency managers to know as much as meteorologists. I do expect them to not act silly when they’re corrected by experts. But most of all, I expect things to get better.

I don’t know why I expect things to get better. It’s hard to imagine the large public- and private-sector investments that are necessary to fix the issue. Storm deaths are relatively low, so there’s not even mass tragedy to spur action. It’s much easier to just work around the edges and pretend the glaring issues don’t exist. But if we’re serious about being a Weather-Ready Nation, we need to fix it at some point. Otherwise public institutions will continue making themselves look bad and misinforming the public.

How to reduce artificial boundaries in severe weather warnings

If you’ve been around here a while, you’ve seen me have opinions about the shapes of so-called “storm-based warnings”. Years ago, the National Weather Service changed the shape of tornado and severe thunderstorm warnings. Instead of issuing warnings based on the county, warnings are arbitrary polygons fitted to the threatened area. The idea is that by shaping warnings to the actual threat, the public gets a more accurate warning.

The reality is a little messier. Warnings are still frequently communicated to the public on a county basis. Worse, the warnings themselves are sometimes shaped to a county line. This is sometimes done to prevent a tiny sliver of a county to be included in a warning. Other times, it’s the result of a boundary between the responsibility areas of different NWS Forecast Offices.

Last week gave a great example close to home. The NWS office in Northern Indiana issued a tornado warning on the edge of their forecast area. Because the adjacent office didn’t issue a warning for that storm, the resulting shape was comically bad.

A tornado warning (red) shaped by the boundary (blue) between the IWX and IND forecast areas.

To be clear: I don’t blame the forecasters here. It was a judgment call to issue or not issue a warning. The real problem is that the artificial boundary does the public a disservice. Most of the general public probably does not know which NWS office serves them. Bureaucratic boundaries here only add confusion.

One solution is for the offices to coordinate when issuing warnings near the edge of their area. That doesn’t hold up well in the short time frame of severe weather, especially if an office is understaffed or over-weathered. Coordination takes time and minutes matter in these situations.

My solution is simpler: allow (and encourage) offices to extend warnings beyond their area. Pick a time frame (30 minutes seems reasonable) and allow the warning to extend as far into another office’s area as it needs to in order to contain the threat at that time. Once the threat is entirely into the new area, allow that office to update the warning as they see fit.

This allows offices to draw warnings based on the actual threat. It buys some time for additional coordination if needed, or at least gives a cleaner end to the warning. It does mean that some local officials will need to have a relationship with two NWS offices, but if they’re on the edge they should be doing that anyway.

The downside is that it increases the effort in verifying warnings because you can no longer assume which office issued the warning. And it could lead to some territorial issues between offices. But the status quo provides easier bureaucracy by putting the burden on the public. That’s not right.

Sidebar: what about issuing warnings at the national level?

Another solution would be for a national center to issue warnings. This is already the case for severe weather watches, after all. While it would solve the responsibility area problems, it would also reduce the overall quality of warnings. Local offices develop relationships with local officials, spotters, etc. These relationships help them evaluate incoming storm reports, tailor warnings to local conditions and events, etc. While a national-level warning operation would clearly provide some benefit, warning response is ultimately a very personal action that benefits from putting the warning issuance as close to the public as possible.

In defense of the call-to-action

Dr. Chuck Doswell, one of the most well-known and respected severe weather researchers, wrote on his personal blog:

Personally, I believe telling people what to do, say via “call to action” statements (CTAs) is not a good idea.  What people need to do depends on their specific situations, about which we as forecasters know nothing! 

The latter part of his statement is true, as are his assertion that people need to develop their plans well ahead of time. But I strongly disagree that call-to-action statements are not important. 

Dr. Doswell is thinking like someone who has devoted his life to severe weather for decades. That makes sense, but it is not a mindset shared by the general public. Fundamentally, he misunderstands the purpose of call-to-action statements: they’re not for teaching people what to do, they’re for reminding people what to do.

In the middle of an emergency, it’s very easy to forget what you know. That’s why people train for scenarios repeatedly – to have responses be reflexive, not cognitive. Call-to-action statements serve to remind people in an emergency of the general principles of severe weather safety. The education about those principles and specific implementations must be addressed ahead of time.

To warn or not to warn?

The decision to issue a tornado warning is a difficult one for meteorologists. A timely and accurate warning can save lives, but a false alarm contributes to an already-too-high indifference among the public.

On March 31, thunderstorms moved through Indiana in the mid-afternoon. The Storm Prediction Center had a slight risk for the area, so I had been keeping an eye on them. One cell had shown rotation since it had been in east-central Illinois. After a while, it started to fall apart. Then around 4:40 PM, I happened to glance back over at the radar, and I saw this:

KIND reflectivity and velocity at 4:42 PM Eastern on March 31, 2016.

KIND reflectivity and velocity at 4:40 PM Eastern on March 31, 2016.

Oh yeah, there’s something going on there. I was talking to my friend Kevin and we were wondering why there was no warning. Even if the forecaster didn’t think a tornado warning was justified, a severe thunderstorm warning would have been a good hedge. As it turns out, the storm produced a brief EF1 tornado about a mile east of my house.

I don’t know why no warning was issued. But here’s a different take: should a warning have been issued? No one was injured and the damage that was done couldn’t have been prevented with a 10-15 minute lead time. Should this count as a false negative?

Yeah, probably. Although no one was injured, a small difference in the location could have easily changed that. But it makes me wonder if warning for every tornado is a reasonable goal.

Too many NWS products?

I’ve written before that the National Weather Service is stuck in a paradigm where the general public isn’t assumed to directly consume most products. NWS text products are largely unchanged from decades ago when most public consumption was filtered through media outlets or emergency management officials. As a result, products are more focused on being technically correct instead of usefully correct.

A prime example is how the National Hurricane Center decided not to issue hurricane warnings as “Superstorm” Sandy made landfall. While technically correct, the decision led to confusion and caused some to minimize the threat.

I suggested on Twitter that people really only want to know three things:

  1. How badly will my stuff be wrecked?
  2. When will be stuff be wrecked?
  3. What should I do about it?

To the extent that the specific threat matters, it only matters as it affects the answers to those three questions. If your roof disappears because of a wind gust instead of a tornado, is it any less gone? Of course, the distinction matters for scientific research purposes, but does the general public care?

When I saw an infographic that Ashley Atney put together, it hit me: the NWS simply has too many warning products. Maps like this drive the point home:

Watch, Warning, Advisory map of Arizona from January 31, 2016. Image capture via Rob Dale (@therobdale).

Watch, Warning, Advisory map of Arizona from January 31, 2016. Image capture via Rob Dale.

This may be heresy, but what if we condensed products? Winter storm, lake effect snow, and blizzard products could all be combined into one, with the product itself describing the details. After all, merely saying “winter storm” versus “blizzard” doesn’t really communicate much anyway. Further heresy would be to combine severe thunderstorm and tornado warnings.

I’m not saying that these (and other condensations) are necessarily the right way to go. There’s a lot of research that would need to be done first in order to make sure that the net effect on the public is positive. But the current system is in need of improvement.

NWS products are not ready for public consumption

Decades ago, dissemination of National Weather Service products was largely done via third parties, particularly broadcast media. Then along came the Internet and suddenly NWS products became readily available to the public at-large. This should have been a benefit, but the products have not adjusted to this new paradigm.

Forget that text products are still in all-caps (I’ve found that I have a harder time reading discussions that are in mixed case). Severe weather warnings give information out of order. Warnings and even regular forecasts suffer from discontinuity at forecast area boundaries. Worst of all, forecasts do not convey uncertainty, instead providing a single number instead of a possible range.

The snow storm that hit (to one degree or another) the east coast this weekend is an excellent example of how forecast uncertainty was not well-communicated. In some areas, the forecast was quite accurate. In others, snowfall predictions were far too high. The forecasters knew there was a high degree of uncertainty about the forecast, so why did the public and civic leaders?

It’s hard to fault individual forecasters. They work hard within the system to produce valuable forecasts for the American people. It’s the management and technology that prevent the message from getting out. In recent years, the industry (including the private sector) has begun to understand the need for social science to accompany meteorological science. Hopefully this new focus will help to make products for the modern public.

Thoughts on the Weather Forecasting Improvement Act

Insurance Journal reported last week on a bill sponsored by Representative Jim Bridenstine (R-Oklahoma). In a fit of poor reporting, the author says the bill makes the “protection of people and property a priority.” Unfortunately, the National Weather Service mission statement has included “protection of life and property” for years. The bill itself contains no such insulting verbiage. On the surface, it’s actually a welcome relief: a Congressman looking to direct over half a billion dollars of new funding to scientific research and operations. In reality, it strikes me as more of a pipe dream.

The average tornado warning lead time is currently around 13 minutes. The goal of Bridenstine’s bill is a lead time of 60 minutes or more. Stretch goals are good, but a 4x increase is not, perhaps, the most appropriate for legislation. Even so, there’s a question of how valuable such an increase would really be. Increased protection of property is probably not going to be that dramatic with hour-long lead times. It’s not like people can move their houses and businesses out of the way. Some damage could be prevented by securing loose objects and boarding windows, but it’s not likely to be significant.

Protecting life is the more important aspect, but would a one-hour lead time help? I’ve argued for years that there’s definitely an upper bound to lead times after which the returns diminish. My suspicion is that as the lead time grows beyond that point, people become more and more complacent. This argument has been based on hunches and unsubstantiated reasoning. It turns out, there’s evidence that increased lead time has no impact on injuries from tornadoes.

Even if the benefits are minimal, the amount of learning that would have to take place to get lead times up to an hour would aid our understanding of severe weather. The improvements to observation networks and modeling would benefit all areas of weather forecasting. Even  if tornado warning lead times remain unchanged, the scientific impact of this bill would be dramatic. I just worry that it’s setting the National Weather Service up for “failure”.

Outdoor warning sirens

After adding my last post to Google Plus, a friend asked my thoughts on tornado sirens. I replied that I thought they provided a rather poor return on investment. This eventually lead to a day-long discussion with a coworker who disagrees with my assessment. Since I’ve never put my opinion on sirens in a blog post, I figure it’s time to do just that.

First, I am of the opinion that sirens serve an important role in public safety. In places like parks, golf courses, and common outdoor gathering areas, sirens are an excellent way of communicating a single message: “seek shelter”. Sirens are unable to communicate why shelter should be sought, when it is safe to come out (though some jurisdictions will re-sound sirens as an all clear), or what the threat is. Some siren systems lack battery backup, occasionally rendering them inoperable at the most inopportune times. Sirens also suffer, as do most alerting systems, of being stuck in a county-based warning system that no longer exists.

Although they are not intended to be indoor warning devices, some people still rely on them as their primary means of receiving severe weather alerts. When close enough to a siren, this can be fairly reliable, but it’s not always the case. My house is 1000 feet from the nearest siren and during tests it is quite audible (uncomfortably so when the windows are open). During heavy rain, it is not nearly as noticeable. I have little doubt that I would sleep through an overnight siren sounding if it happened during a heavy rain.

So what, then? Since the 1970’s, NOAA Weather Radio has served the country well. It provides both alerts and routine weather information around the clock. Unfortunately, it’s also stuck in the 20th century. County-based alerting simply cannot continue to dominate our warning dissemination systems. If the weather radio system were to send warning coordinates along with SAME codes, receivers could optionally determine if the alert needs to be sounded. In addition, most counties are served by a single transmitter. Each transmitter should have a redundant backup, located far enough away to be unlikely to fail from the same event (e.g. power outage, tornado), but still able to cover the assigned counties.

The current state of sirens in Tippecanoe County includes coverage of all areas I propose require it (and probably some that don’t). There are still, by my calculations (see note 1), approximately 25,000 people in the county who live outside the audible range of sirens. In order to cover the entire county’s land area, the initial investment would be $960,000 to $2.7 million (see note 2) with an annual maintenance cost of $115,200-324,000. (see note 3)

I wanted to look at the costs for all thirteen counties served by the WXK74 transmitter in Monticello, but it turns out the siren count information is not easy to find. WTHR in Indianapolis did some of the work for me, but the rest had to be independently researched. Sadly, some Emergency Managers don’t want to disclose even a count of the sirens in their county. As a result, I was only able to obtain authoritative siren counts for Cass, Clinton, Howard, and Tippecanoe Counties.

Using the same strategy as for Tippecanoe County, I calculated what it would take to get these four counties to 100% siren coverage. The recurring costs are $774,00 to $1.44 million. This after an initial investment of $6.42-12 million dollars. And remember, that’s just for four of the fourteen counties, only 30% of the land area covered by WXK74 (roughly 65% of the population).

What would happen if instead of sirens, we added a second transmitter site and bought every household in the covered area a $50 weather radio? The cost of the transmitter would be about $75,000 (see note 4). Buying the radios would bring the cost up to roughly $9.84 million, which is in the range of covering 65% of residents with sirens.

Of course, this is entirely academic. Sirens are funded at the county level, whereas weather radio is a federal project. It’s not easy to just move the money around. There’s also the alert-granularity issue that needs to be resolved.

Given the economics and the richness of information, it makes sense to push for more radios instead of more sirens. Sirens have their place, but those places are limited. The Indiana Department of Homeland Security guidelines would suggest that many local cities and towns should not have sirens at all (except for parks). Ultimately, weather alerting requires a defense-in-depth approach. Sirens are one layer in certain situations. Weather radios are another, more broadly applicable layer.

A third layer is the Wireless Emergency Alert (WEA) system that is being deployed. Unfortunately, it requires a relatively modern smart phone, so I expect the penetration rate is still fairly low. It, too, suffers from a lack of geographic granularity (although probably better than either weather radio or sirens), and sparseness of information (WEA messages are limited to 90 characters). There’s also people who don’t have their phone by their side at all times. Some people actually leave their mobile devices in other rooms sometimes? Quelle horreur! Disturbingly, WEA does not interrupt phone calls, meaning a long gab session will result in you receiving a warning after it has expired.

Given budgets and politics at all levels of government these days, I’m sad to say that I don’t see any of the existing deficiencies being resolved any time soon.

Continue reading