Of these examples, NOAA weather data most obviously present big data challenges. The value in such large data sets doesn't simply depend on the weather data themselves, but in the ability to forecast weather based upon those data-a classic big data problem. From the perspective of the American citizen, we value accurate forecasts; the immense quantity of historical weather data that feed the forecasting engines is merely the ore we must mine to find the nuggets we desire.
Such is the challenge facing the Open Data Initiative. The more data we have, the less we value the data sets themselves. The information we truly desire lies buried under increasing quantities of irrelevant or otherwise useless information. The danger is that the more data the government provides us, the better hidden are the nuggets we desire. In other words, in the absence of effective big data solutions, truly open government may be out of reach-or, worse, misapplied to obscure the very information that citizens would find most valuable.
Big Data Challenges Heightened by Citizens' Right to Information
This undesirable outcome is clearly not the intention of President Barack Obama's Open Government Initiative, which calls for a presumption of openness. True, there are types of information that the government may not or should not share, including military secrets, private data about individuals, and information relevant to ongoing criminal investigations. However, the list of such sensitive information categories is explicit and limited. All other government information is up for grabs.
News: Obama Promotes New Open Government Initiative
If you want access to such information, typically all you have to do is go to the relevant agency website, as the Obama Administration ordered them to proactively make information available to all citizens. If you can't find what youre looking for, you may make a Freedom of Information Act request. The act was passed in the 1960s, and Congress extended FOIA in 1974 as a result of Watergate. Today, the Government receives more than 500,000 FOIA requests per year, with a current backlog of more than 80,000 requests.
Typically a citizen makes a FOIA request for a particular document or other information- Steve Job's FBI background check, for example. While such documents have a historical as well as human interest value, their worth pales in comparison to the nuggets of gold that Big Data analyses can potentially reveal.