Public Open Data: The Good, the Bad, the Future | Idea Lab | PBS

" ... The White House under the Obama administration has been a leader in its approach to transparency and launched the website data.gov in 2009. To date, nearly 100,000 datasets are available on the site. Other countries soon followed: the U.K., Kenya, Brazil, India and more than 30 other countries have created portals for public data. The European Union Open Data Portal offers more than 6,000 datasets from its member countries. International organizations from the UN to the World Bank add their own repositories to the surfeit of online information. The trend is growing also at the state and local level. Chicago apparently boasts the most public datasets (950) among cities. San Francisco has an extensive open data policy and is one of the first cities in the nation to hire a Chief Data Officer. The benefit of providing public information in accessible formats is undeniable. If entrepreneurs and citizens develop their own tools, it may encourage a sense of community and reduce reliance on outside experts, advantages California Lt. Gov. Gavin Newsom espouses in his book “Citizenville.” Citizens identify the apps that will be of greatest use to them; their work to create designs and prototypes reduces pressure on strained city budgets and circumvents the cumbersome process of competitive procurement. Some of the most powerful tools combine official public data with social media or other citizen input, such as the recent partnership between Yelp and the public health departments in New York and San Francisco for restaurant hygiene inspection ratings. In other contexts, such tools can help uncover and ultimately reduce corruption by making it easier to 'follow the money.' Despite the opportunities offered by 'free data,' this trend also raises new challenges and concerns, among them, personal privacy and security. While attention has been devoted to the unsettling power of big data analysis and 'predictive analytics' for corporate marketing, similar questions could be asked about the value of public data. Does it contribute to community cohesion that I can find out with a single query how much my neighbors paid for their house or (if employed by public agencies) their salaries? Indeed, some studies suggest that greater transparency leads not to greater trust in government but to resignation and apathy. Exposing certain law enforcement data also increases the possibility of vigilantism. California law requires the registration and publication of the home addresses of known sex offenders, for instance. Or consider the controversy and online threats that erupted when, shortly after the Newtown tragedy, a newspaper in New York posted an interactive map of gun permit owners in nearby counties ... Ad hoc apps built on public open data represent all the advantages and pitfalls of volunteer labor: free, committed, passionate, but perhaps unsustainable or unrepresentative of a diverse constituency. Whom does open data omit? What kind of apps will be created by programmers and designers with discretionary time and tech tools to build them? Policymakers and officials must still mind the 'big data gap.'  So what does the future hold for open data? Publishing data is only one part of the information ecosystem. To be useful, tools must be developed for cleaning, sorting, analyzing and visualizing it as well. The much-heralded new field of 'data science' has applications in industry as well as in non-profit or government sectors. (UC Berkeley’s School of Information just launched an online master’s degree program to train professionals in this area.) Fluency in parsing and presenting public data also has clear relevance for journalists, and the field of 'data journalism' is a bright spot in the struggling newspaper industry.

For-profit companies and non-profit watchdog organizations will continue to emerge and expand, building on the foundation of this data flood. Public-private partnerships such as those between San Francisco and Appallicious or Granicus, startups created by Code for America’s Incubator, and non-partisan organizations like the Sunlight Foundation and MapLight rely on public data repositories for their innovative applications and analysis.  Making public data more accessible is an important goal and offers enormous potential to increase civic engagement. To make the most effective and equitable use of this resource for the public good, cities and other government entities should invest in the personnel and equipment — hardware and software — to make it universally accessible. At the same time, Chief Data Officers (or equivalent roles) should also be alert to