ForecastWatch recently issued a report on the accuracy of weather forecasts from 2010 through June 2016 (PDF here). While many readers will focus on who was more accurate, what stood out to me was how forecast accuracy has improved. Meteorologists have long “enjoyed” a reputation for inaccuracy — often more due to perception than fact. But those in the know are aware that skill is increasing.

Forecast accuracy over time

ForecastWatch’s U.S. analysis shows a clear — if small — improvement in the average accuracy since 2010.

Average U.S. forecast accuracy from 2010 – June 2016.

The chart above shows the average for all of the forecast sources Forecast Watch analyzed. To be frank, World Weather Online is a stinker, and brings the averages down by a considerable margin. Examining the best and worst forecast shows more interesting results.

Best and worst U.S. forecast accuracy from 2010 – June 2016.

Forecasts get less skillful over time, thanks to subtle inaccuracies in the initial conditions (see also: butterfly effect). That’s obvious in both graphs. What this second chart shows is that the best 6-9 forecast is now roughly as skillful as the worst 3-5 forecast was in 2010. And the best 3-5 day forecast is in the middle of the 1-3 day forecast skill from just a few years ago.

Forecasts are definitely improving. This is due in part to better modeling — both more powerful computers and also the ability to ingest more data. Research and improved tooling helps as well.

Forecasts still bust, of course, and forecasters hate bad forecasts as much as the public does. As I write this, forecasters in North Carolina are dealing with an inaccurate snow forecast (winter weather forecasting sucks due to reasons I explained in a previous post). Missed forecasts can cost money and lives, so it’s good to see a trend of improvement.

Forecast accuracy in your city

The ForecastWatch report breaks down by broad regions: United States, Europe, and Asia/Pacific. But weather is variable on much smaller scales. The ForecastAdvisor tool compares forecasts at the local level giving you the ability to see who does the best for your city. As of early January 2017, AccuWeather had the most accurate forecasts for Lafayette, Indiana, but they only place fourth when considering the past year.

On my to-do list, this post is titled “Chad Evans, you son of a bitch.” Though the specifics are about the failings of a specific local TV meteorologist, the broader lesson is that weather forecasts longer than about a week aren’t worth the time it takes to make or read them. AccuWeather’s 45-day forecasts have caught some flack for being awful, as everyone expected they would be. Less attention has been paid to verifying the long-range forecasts from WLFI meteorologist Chad Evans.

I decided to take a look at the September 2011 forecast to see how it fared (there’s probably a forecast from September 2012, but I’m too lazy to search for it). As the graphs below show, it’s hard to beat climatology for long-range forecasts. Interestingly, there’s not a noticeable drop in skill over time with temperatures. The precipitation forecast does seem to get worse over the life of the forecast, with the exception of a lucky break in the summer.

Forecast and climatology monthly average temperature errors.

Forecast and climatology precipitation total errors.

Mr. Evans was smart enough not to include day-by-day specifics, except for Christmas. This year, he claimed  claimed to be 4-0 on his white Christmas forecasts. The forecast called for 1″ or more of snow on Christmas morning. Unfortunately, there was none. Several inches fell the week before, but warm and rainy weather the weekend prior took care of that. Speaking of snowfall, 10″ was forecast for January 2014. In six days, we’ve already passed that, and the snow continues to fall as I write.

In the first two months of the most recent annual forecast, the temperature errors aren’t awful, but the precip forecasts miss the mark pretty hard (though the direction of the error was right in both cases). As the year progresses, you’d expect to see the skill diminish.

And that’s really the point here: seasonal (or longer) outlooks are really bad at giving specific information. You can sometimes make use of them for trends, but even then they’re not very reliable. I can’t fault a forecaster for busting a forecast, I’ve had plenty of busts. But presenting skill-less forecasts to the public is a disservice to the public and to the reputation of the meteorology profession.