Believe it or not, your weather reporter has been getting it right more and more often over the last few decades.
So says New York Times statistician Nate Silver. In an excerpt from his forthcoming book about our ability to make better forecasts in the era of “Big Data”, Silver explains that the field of weather prediction is a major success story.
A generation of meteorologists has been gradually perfecting their blend of supercomputer modeling and qualitative human rigor since long before “data science” had a name. And their work has made a huge difference. Some examples:
- Daily Highs – The US National Weather Service has halved its margin of error for high-temperature forecasts (made 3 days in advance), from six degrees in 1972 to three degrees today.
- Hurricanes – In the 1980s, the National Hurricane Center’s three-days-ahead prediction for pinpointing hurricane landfall used to miss by an average of 350 miles; today it’s 100 miles.
- Lightning Storms – The chance of an American in 1940 being killed by lightning was 1 in 400,000. Nowadays, it’s 1 in 11 million, in large part thanks to better weather forecasts.
And the importance of improved weather forecasts goes beyond whether your picnic plans go smoothly. Accurate weather prediction also promises big benefits to help optimize how we produce and use energy. A few examples:
- With improved daily weather forecasts, utilities can plan ahead more effectively for how much energy is needed to accommodate heating and cooling demand (which together represent 50% of US home energy usage).
- Real-time weather forecasts can be automatically incorporated into buildings’ hourly thermostat settings (e.g. if hot temperatures are approaching, the system could start cooling things down ahead of time).
- Accurate forecasts about cloud cover, wind, and precipitation
make it easier to manage electricity generation from hydropower and intermittent renewables like solar and wind energy.
- Reliable forecasts of severe weather enable speedy resolution of power outages: if a utility knows where a lightning storm will hit, crews and materials can be prepared in advance to address the damage.
As big-data companies like Opower and other researchers delve into large datasets to uncover trends about the way our world works (both mother nature and human nature), we will be served well by emulating the successes of weather forecasters.
For more on what we can learn from meteorologists’ knack for big data, check out Nate Silver’s thought-provoking new book chapter, “The Weatherman is Not a Moron.”