Today, the National Weather Service is much better about noting the uncertainty level of its forecasts, Silver said, citing the "cone of uncertainty" that comes along with projected hurricane paths. Showing uncertainty "in a visual way is important" in helping people evaluate forecasts.
Probability forecasts are a "way point between ignorance and knowledge," but they are not certainties.
2. "Know where you're coming from" -- that is, know your weak points, the incentives to reach certain conclusions and the biases against others. "You are defined by your weakest link," he said.
He noted an experiment on gender bias where people were shown similar technical resumes -- one with a female name and one with a male name. People who claimed to have no gender bias were in fact more likely to discriminate against the female's resume. Why? Those who were aware of their tendencies toward bias were more likely to take action to counteract it, Silver said.
3. Survey the data landscape, and make sure you have some variance in your data before having confidence in a forecast. (In other words, accurately forecasting the weather in San Diego is not as impressive feat as doing so in Buffalo.)
Likewise, forecasting a stable economy is easier than in times of a lot of booms and busts, which helps explain why many forecasters were unprepared for the most recession. The forecasters were creating models based on data from 1986-2006, when the economy was unusually stable. A detailed and sophisticated model based on silly assumptions won't do you much good, he noted.
4. Finally, trial and error are helpful.
Models tend to work well when they are developed slowly with a lot of feedback. As with many things in life: "You should be suspicious of miraculous results."
Sharon Machlis is online managing editor at Computerworld. Her e-mail address is email@example.com. You can follow her on Twitter @sharon000, on Facebook, on Google+ or by subscribing to her RSS feeds: articles; and blogs.