Please don’t argue with the warning system

“Please don’t argue with the warning system”, Indiana University told a lecturer from its meteorology department as he rightly criticized their communications on Sunday.

Despite being wrong, the university continued to insist that they were making the right choice. Now as a Boilermaker, I’m normally in favor of Indiana University embarrassing itself. But this time, it’s just bad. Warning fatigue can kill people. The false alarm rate is already too high; telling people about warnings that don’t exist only makes it worse.

The “warnings affect the entire county until notified otherwise” statement is only a decade out of date. But I get it, our warning dissemination technology hasn’t caught up with how warnings are issued. You may recall I’ve written a few words on the subject.

The fact that dissemination technology is still (mostly) stuck in a county-based paradigm 10 years after the nationwide implementation of polygon-based warnings is an embarrassment. Emergency management is more than just weather, so I don’t expect emergency managers to know as much as meteorologists. I do expect them to not act silly when they’re corrected by experts. But most of all, I expect things to get better.

I don’t know why I expect things to get better. It’s hard to imagine the large public- and private-sector investments that are necessary to fix the issue. Storm deaths are relatively low, so there’s not even mass tragedy to spur action. It’s much easier to just work around the edges and pretend the glaring issues don’t exist. But if we’re serious about being a Weather-Ready Nation, we need to fix it at some point. Otherwise public institutions will continue making themselves look bad and misinforming the public.

How to reduce artificial boundaries in severe weather warnings

If you’ve been around here a while, you’ve seen me have opinions about the shapes of so-called “storm-based warnings”. Years ago, the National Weather Service changed the shape of tornado and severe thunderstorm warnings. Instead of issuing warnings based on the county, warnings are arbitrary polygons fitted to the threatened area. The idea is that by shaping warnings to the actual threat, the public gets a more accurate warning.

The reality is a little messier. Warnings are still frequently communicated to the public on a county basis. Worse, the warnings themselves are sometimes shaped to a county line. This is sometimes done to prevent a tiny sliver of a county to be included in a warning. Other times, it’s the result of a boundary between the responsibility areas of different NWS Forecast Offices.

Last week gave a great example close to home. The NWS office in Northern Indiana issued a tornado warning on the edge of their forecast area. Because the adjacent office didn’t issue a warning for that storm, the resulting shape was comically bad.

A tornado warning (red) shaped by the boundary (blue) between the IWX and IND forecast areas.

To be clear: I don’t blame the forecasters here. It was a judgment call to issue or not issue a warning. The real problem is that the artificial boundary does the public a disservice. Most of the general public probably does not know which NWS office serves them. Bureaucratic boundaries here only add confusion.

One solution is for the offices to coordinate when issuing warnings near the edge of their area. That doesn’t hold up well in the short time frame of severe weather, especially if an office is understaffed or over-weathered. Coordination takes time and minutes matter in these situations.

My solution is simpler: allow (and encourage) offices to extend warnings beyond their area. Pick a time frame (30 minutes seems reasonable) and allow the warning to extend as far into another office’s area as it needs to in order to contain the threat at that time. Once the threat is entirely into the new area, allow that office to update the warning as they see fit.

This allows offices to draw warnings based on the actual threat. It buys some time for additional coordination if needed, or at least gives a cleaner end to the warning. It does mean that some local officials will need to have a relationship with two NWS offices, but if they’re on the edge they should be doing that anyway.

The downside is that it increases the effort in verifying warnings because you can no longer assume which office issued the warning. And it could lead to some territorial issues between offices. But the status quo provides easier bureaucracy by putting the burden on the public. That’s not right.

Sidebar: what about issuing warnings at the national level?

Another solution would be for a national center to issue warnings. This is already the case for severe weather watches, after all. While it would solve the responsibility area problems, it would also reduce the overall quality of warnings. Local offices develop relationships with local officials, spotters, etc. These relationships help them evaluate incoming storm reports, tailor warnings to local conditions and events, etc. While a national-level warning operation would clearly provide some benefit, warning response is ultimately a very personal action that benefits from putting the warning issuance as close to the public as possible.

In defense of the call-to-action

Dr. Chuck Doswell, one of the most well-known and respected severe weather researchers, wrote on his personal blog:

Personally, I believe telling people what to do, say via “call to action” statements (CTAs) is not a good idea.  What people need to do depends on their specific situations, about which we as forecasters know nothing! 

The latter part of his statement is true, as are his assertion that people need to develop their plans well ahead of time. But I strongly disagree that call-to-action statements are not important. 

Dr. Doswell is thinking like someone who has devoted his life to severe weather for decades. That makes sense, but it is not a mindset shared by the general public. Fundamentally, he misunderstands the purpose of call-to-action statements: they’re not for teaching people what to do, they’re for reminding people what to do.

In the middle of an emergency, it’s very easy to forget what you know. That’s why people train for scenarios repeatedly – to have responses be reflexive, not cognitive. Call-to-action statements serve to remind people in an emergency of the general principles of severe weather safety. The education about those principles and specific implementations must be addressed ahead of time.

To warn or not to warn?

The decision to issue a tornado warning is a difficult one for meteorologists. A timely and accurate warning can save lives, but a false alarm contributes to an already-too-high indifference among the public.

On March 31, thunderstorms moved through Indiana in the mid-afternoon. The Storm Prediction Center had a slight risk for the area, so I had been keeping an eye on them. One cell had shown rotation since it had been in east-central Illinois. After a while, it started to fall apart. Then around 4:40 PM, I happened to glance back over at the radar, and I saw this:

KIND reflectivity and velocity at 4:42 PM Eastern on March 31, 2016.

KIND reflectivity and velocity at 4:40 PM Eastern on March 31, 2016.

Oh yeah, there’s something going on there. I was talking to my friend Kevin and we were wondering why there was no warning. Even if the forecaster didn’t think a tornado warning was justified, a severe thunderstorm warning would have been a good hedge. As it turns out, the storm produced a brief EF1 tornado about a mile east of my house.

I don’t know why no warning was issued. But here’s a different take: should a warning have been issued? No one was injured and the damage that was done couldn’t have been prevented with a 10-15 minute lead time. Should this count as a false negative?

Yeah, probably. Although no one was injured, a small difference in the location could have easily changed that. But it makes me wonder if warning for every tornado is a reasonable goal.

NWS products are not ready for public consumption

Decades ago, dissemination of National Weather Service products was largely done via third parties, particularly broadcast media. Then along came the Internet and suddenly NWS products became readily available to the public at-large. This should have been a benefit, but the products have not adjusted to this new paradigm.

Forget that text products are still in all-caps (I’ve found that I have a harder time reading discussions that are in mixed case). Severe weather warnings give information out of order. Warnings and even regular forecasts suffer from discontinuity at forecast area boundaries. Worst of all, forecasts do not convey uncertainty, instead providing a single number instead of a possible range.

The snow storm that hit (to one degree or another) the east coast this weekend is an excellent example of how forecast uncertainty was not well-communicated. In some areas, the forecast was quite accurate. In others, snowfall predictions were far too high. The forecasters knew there was a high degree of uncertainty about the forecast, so why did the public and civic leaders?

It’s hard to fault individual forecasters. They work hard within the system to produce valuable forecasts for the American people. It’s the management and technology that prevent the message from getting out. In recent years, the industry (including the private sector) has begun to understand the need for social science to accompany meteorological science. Hopefully this new focus will help to make products for the modern public.

Thoughts on the Weather Forecasting Improvement Act

Insurance Journal reported last week on a bill sponsored by Representative Jim Bridenstine (R-Oklahoma). In a fit of poor reporting, the author says the bill makes the “protection of people and property a priority.” Unfortunately, the National Weather Service mission statement has included “protection of life and property” for years. The bill itself contains no such insulting verbiage. On the surface, it’s actually a welcome relief: a Congressman looking to direct over half a billion dollars of new funding to scientific research and operations. In reality, it strikes me as more of a pipe dream.

The average tornado warning lead time is currently around 13 minutes. The goal of Bridenstine’s bill is a lead time of 60 minutes or more. Stretch goals are good, but a 4x increase is not, perhaps, the most appropriate for legislation. Even so, there’s a question of how valuable such an increase would really be. Increased protection of property is probably not going to be that dramatic with hour-long lead times. It’s not like people can move their houses and businesses out of the way. Some damage could be prevented by securing loose objects and boarding windows, but it’s not likely to be significant.

Protecting life is the more important aspect, but would a one-hour lead time help? I’ve argued for years that there’s definitely an upper bound to lead times after which the returns diminish. My suspicion is that as the lead time grows beyond that point, people become more and more complacent. This argument has been based on hunches and unsubstantiated reasoning. It turns out, there’s evidence that increased lead time has no impact on injuries from tornadoes.

Even if the benefits are minimal, the amount of learning that would have to take place to get lead times up to an hour would aid our understanding of severe weather. The improvements to observation networks and modeling would benefit all areas of weather forecasting. Even  if tornado warning lead times remain unchanged, the scientific impact of this bill would be dramatic. I just worry that it’s setting the National Weather Service up for “failure”.

Outdoor warning sirens

After adding my last post to Google Plus, a friend asked my thoughts on tornado sirens. I replied that I thought they provided a rather poor return on investment. This eventually lead to a day-long discussion with a coworker who disagrees with my assessment. Since I’ve never put my opinion on sirens in a blog post, I figure it’s time to do just that.

First, I am of the opinion that sirens serve an important role in public safety. In places like parks, golf courses, and common outdoor gathering areas, sirens are an excellent way of communicating a single message: “seek shelter”. Sirens are unable to communicate why shelter should be sought, when it is safe to come out (though some jurisdictions will re-sound sirens as an all clear), or what the threat is. Some siren systems lack battery backup, occasionally rendering them inoperable at the most inopportune times. Sirens also suffer, as do most alerting systems, of being stuck in a county-based warning system that no longer exists.

Although they are not intended to be indoor warning devices, some people still rely on them as their primary means of receiving severe weather alerts. When close enough to a siren, this can be fairly reliable, but it’s not always the case. My house is 1000 feet from the nearest siren and during tests it is quite audible (uncomfortably so when the windows are open). During heavy rain, it is not nearly as noticeable. I have little doubt that I would sleep through an overnight siren sounding if it happened during a heavy rain.

So what, then? Since the 1970’s, NOAA Weather Radio has served the country well. It provides both alerts and routine weather information around the clock. Unfortunately, it’s also stuck in the 20th century. County-based alerting simply cannot continue to dominate our warning dissemination systems. If the weather radio system were to send warning coordinates along with SAME codes, receivers could optionally determine if the alert needs to be sounded. In addition, most counties are served by a single transmitter. Each transmitter should have a redundant backup, located far enough away to be unlikely to fail from the same event (e.g. power outage, tornado), but still able to cover the assigned counties.

The current state of sirens in Tippecanoe County includes coverage of all areas I propose require it (and probably some that don’t). There are still, by my calculations (see note 1), approximately 25,000 people in the county who live outside the audible range of sirens. In order to cover the entire county’s land area, the initial investment would be $960,000 to $2.7 million (see note 2) with an annual maintenance cost of $115,200-324,000. (see note 3)

I wanted to look at the costs for all thirteen counties served by the WXK74 transmitter in Monticello, but it turns out the siren count information is not easy to find. WTHR in Indianapolis did some of the work for me, but the rest had to be independently researched. Sadly, some Emergency Managers don’t want to disclose even a count of the sirens in their county. As a result, I was only able to obtain authoritative siren counts for Cass, Clinton, Howard, and Tippecanoe Counties.

Using the same strategy as for Tippecanoe County, I calculated what it would take to get these four counties to 100% siren coverage. The recurring costs are $774,00 to $1.44 million. This after an initial investment of $6.42-12 million dollars. And remember, that’s just for four of the fourteen counties, only 30% of the land area covered by WXK74 (roughly 65% of the population).

What would happen if instead of sirens, we added a second transmitter site and bought every household in the covered area a $50 weather radio? The cost of the transmitter would be about $75,000 (see note 4). Buying the radios would bring the cost up to roughly $9.84 million, which is in the range of covering 65% of residents with sirens.

Of course, this is entirely academic. Sirens are funded at the county level, whereas weather radio is a federal project. It’s not easy to just move the money around. There’s also the alert-granularity issue that needs to be resolved.

Given the economics and the richness of information, it makes sense to push for more radios instead of more sirens. Sirens have their place, but those places are limited. The Indiana Department of Homeland Security guidelines would suggest that many local cities and towns should not have sirens at all (except for parks). Ultimately, weather alerting requires a defense-in-depth approach. Sirens are one layer in certain situations. Weather radios are another, more broadly applicable layer.

A third layer is the Wireless Emergency Alert (WEA) system that is being deployed. Unfortunately, it requires a relatively modern smart phone, so I expect the penetration rate is still fairly low. It, too, suffers from a lack of geographic granularity (although probably better than either weather radio or sirens), and sparseness of information (WEA messages are limited to 90 characters). There’s also people who don’t have their phone by their side at all times. Some people actually leave their mobile devices in other rooms sometimes? Quelle horreur! Disturbingly, WEA does not interrupt phone calls, meaning a long gab session will result in you receiving a warning after it has expired.

Given budgets and politics at all levels of government these days, I’m sad to say that I don’t see any of the existing deficiencies being resolved any time soon.

Continue reading

More thoughts on warning polygons

On Tuesday, Patrick Marsh wanted a distraction from his dissertation and embarked on an idle investigation of tornado warnings and impacted areas (my thoughts on what “impact” means are below). Using some very rough approximations, he calculated the percentage of warned persons who are impacted by a tornado. Even under the most generous set of assumptions, the verification by population is generally below 20%. It’s worth noting that 2011 (the most recent year that official tornado data is available) was the best year in the analysis, but there is no indication of a general improvement trend.

Despite some of the problems I’ve previously noted in the polygon warning system, it’s still better than warning entire counties. Still there’s a lot of room to improve the false alarm rate. Much of the population-based false alarm comes from warnings that have no tornado at all. The rest either comes from too-large warnings or not-small-enough warnings (“not-small-enough” warnings are small enough to be justifiable, but still larger than absolutely necessary).

It’s not always easy to shrink warnings. Only the supercell storms relatively close to a radar site seem suitable. In those cases, it’s possible to make the warning only a few miles wide, or the width of the mesocyclone with uncertainty added as you go downstream. This would minimize the area under the warning, but it got me wondering: would that be too small?

At the scale of a mile or two, how do you explain the warned area to the public? Storm-based warnings are already difficult to communicate quickly, and microwarnings would only compound the problem. Even in Lafayette, the 10th largest city in Indiana, the covered area might look something like:

...TEAL ROAD BETWEEN 4TH STREET AND 26TH ST...
...KOSSUTH ST BETWEEN 9TH STREET AND SAGAMORE PARKWAY...
...SOUTH ST BETWEEN FIVE POINTS AND FARABEE DR...
...18TH ST BETWEEN BECK LN AND FERRY ST...

And so on. Or maybe it would use neighborhoods and landmarks instead:

...LAFAYETTE COUNTRY CLUB...
...HIGHLAND PARK...
...JEFFERSON HIGH SCHOOL...
...FIVE POINTS...
...WALLACE TRIANGLE...
...COLUMBIAN PARK...

Either way, it’s much more complicated than a simple “LAFAYETTE”. Yes, it’s more detailed, does that help? First, it takes much longer to read the text. Secondly, can you count on people, especially those who are new to the area, to know the streets, neighborhoods, and landmarks well enough to quickly figure out if they’re affected or not? I suspect the answer is “no”.  Perhaps some day someone with the time, energy, and funding can look at this.

Sidebar: What does it mean to be “affected” by a tornado?

When Patrick commented on Twitter about his post from Tuesday, I remarked that the results depend on how “affected” is defined. His analysis was based on population, but that doesn’t necessarily convey all impacts. If my office is wiped out by a tornado but my house is untouched, I am still affected. You can expand this out even further and incorporate businesses that saw decreased revenue as a result of a tornado, even if they were not directly hit. Businesses that see increased revenue (e.g. home improvement stores) might also be included, even though the effect is a positive one. The broader (and, I would argue, more accurately) we define being affected, the more difficult it becomes to get accurate data.

Disseminating storm-based warnings

Earlier this month, I wrote about the format and wording of severe weather warnings, and how to effectively shape those warnings. In those previous posts, we came to the conclusion that county sections are pretty lousy ways to define warnings. Defining warnings based on counties leads to over-warning, and using county subsections are ambiguous to the public. Storm-based polygon warnings are the most accurate way to define warnings, but they come with their own problems. First, as I previously discussed, there’s the issue of having to shape around county boundaries. Secondly, they present some challenges in dissemination.

Storm-based warnings are easily transmitted visually (though they still require a basic level of geographical knowledge that I’m not sure we can assume), so they work well on TV, Internet images, and smartphone apps. They are really poor in text or audio formats. Warnings disseminated through audio or text must still reference vaguely-defined county regions. As a result, storm-based warnings lose some of their benefit immediately.

NOAA All-Hazards Radio’s Specific Area Message Encoding (SAME), which is used to selectively activate weather radio receivers, works on a county basis. As a result, the NWS obsoleted its own technology when it switched to storm-based warnings in 2007. On the other hand, county-based warning distribution like SAME has one distinct advantage: the ability to pre-warn. A common recommendation when programming weather radios is to have the radio activate for warnings for your own county and also the surrounding counties. This allows additional lead time in some events. The current warning paradigm does not allow for such a setup.

The future of warning dissemination will be hitting people in the pocket. Mobile phones are the best way to reach a large (and growing) portion of the population. The Wireless Emergency Alert system is a good start. WEA will automatically send warnings to cell phones in affected areas (this also helps to address the issue of people driving, especially through areas they wouldn’t normally be), but it has some room for improvement. The character limit of WEA messages is 90, which is just over half the length of a traditional text message, resulting in a very information-sparse alert. In addition, it will be based on the tower‘s location, not the phone’s. This means that people will receive warnings that do not include them (or worse, will not receive warnings that do include them). Of course, it also requires that people have a WEA-capable device.

In the end, a multi-layered approach is required. Broadcast media must continue to remain a valuable partner. WEA and third-party smartphone apps should continue to get warnings to people’s phones. Weather radio technology should either be upgraded to support location-based alerting or be gracefully retired. Warning siren systems should be upgraded so that they can be sounded selectively (most systems still sound county-wide), or better yet scrapped entirely.

Effectively shaping warnings

Earlier this week, I wrote about the format and wording of severe weather warnings in order to most effectively communicate the necessary information to the public. In that post (and the excellent discussion that took place in the comments), I referred several times to the problems that can arise from the shape of warnings. Before I let loose on that, let’s set some historical context. From 1953-2005, the National Weather Service issued warnings on a by-county basis. This lead to over-warning unaffected areas.

In 2005, the NWS began a pilot program to issue warnings with a forecaster-defined polygon shape. In this way, the warnings could be issued such that they reflect actual threats and not political boundaries. All forecast offices began using these “storm-based warnings” in 2007, but the system still isn’t perfect.

Radar image showing two separate polygon warnings side-by-side

Sample polygon warning from WFO Indianapolis. Source: http://www.crh.noaa.gov/ind/?n=polywarn

Even though forecasters can issue warnings that are based on the atmosphere, they still must consider the political boundaries. Many warning dissemination systems (more on this in a future post) don’t support polygon warnings, so if a warning would normally clip a corner of a county, the forecaster must consider whether or not to cut a notch out of the warning. (I asked for clarification from a friend who is an NWS forecaster. He said there’s a setting that optionally excludes tiny slivers of counties. “We try to do our best to serve two masters between county based communication systems and scientifically based warnings. The main focus at all times, however, is getting information to people who are threatened as fast as possible and in as useful a manner as possible.”)

The problem is further compounded when a storm exists along the boundary between the County Warning Areas (CWAs) of two forecast offices. Current NWS practice does not allow a warning to extend outside an office’s CWA. Since CWAs edges are determined by the county borders, they are frequently uneven. A storm may clip a small portion of another office’s CWA, and the issuing forecaster must shape the polygon to avoid that portion. In order to include said portion, the other office must issue its own warning. While offices will often coordinate when storms are near a CWA boundary, I seriously doubt that any forecasters will take the time to make sure their polygons match precisely. The resulting discontinuity can be confusing to the public and makes absolutely no sense from a threat perspective. The storm does not respect political boundaries.

Radar image with severe thunderstorm warning polygons. The shape of the middle polygon is influenced by the boundary between county warning areas

A recent severe weather event in central Indiana. The warning in the middle of the image was issued by the NWS office in Northern Indiana. The southern extent of the warning is defined by the boundary between the Northern Indiana and Indianapolis county warning areas.

Removing the county-border issues from the polygon system still doesn’t lead to perfection. Polygons themselves suffer from some issues. Notably, they’re ripe for being over-large in order to improve verification scores, as Patrick Marsh posted earlier this week. The other key concern is that, as I mentioned above, some warning systems have no concept of polygons. NOAA All-Hazards Radio (also known as “weather radio”) is a prime example., as are outdoor warning sirens (some locations have the ability to sound sirens selectively, but it is by no means ubiquitous). Until these systems are modernized, even the best polygons will still lead to over-warning.