When can we legally share protected data? - Sunlight Foundation Blog

"As Sunlight explores the public use of individual-level data — with a special focus on how this works in the criminal justice system — we are also exploring the practical, legal and ethical challenges of working with individual-level data. One big reason why individual-level data is often prevented from being shared publicly is for the protection of individual privacy. In thinking about this tension between privacy rights and open government data, we have so far focused on the legal and technical context underlying the open sharing of individual-level 'microdata.' At the same time we investigate open data sharing — meaning data that anyone can download and analyze anytime, anywhere — we recognize that it can also be useful to consider how the public can benefit from data shared privately, not openly. Though average citizens cannot access a number of data sets maintained by government because of privacy concerns, they do nonetheless benefit from the work of researchers, both inside and outside of government, who have been able to share that data with each other. For instance, many states have created integrated justice information systems to allow law enforcement, courts and corrections to coordinate more effectively. States like Washington and Oregon have also used interagency data sharing programs to evaluate the effectiveness of programs, like re-entry programs for prisoners and rehabilitation for minors with mental health concerns. While these datasets cannot be legally shared in the open, they can be shared privately in highly valuable ways. Because of privacy law, some datasets are not only not 'open,' but they are in fact very 'closed' and only made available through highly restrictive procedures. Under typical privacy law, data users must meet high standards and establish relationships with the data providers to create credibility and trust. While one of the strengths of the open data movement is the potential for unexpected benefits that come from the free and unrestricted release of data, the open release of individual-level data can come with certain risks. Fears of harm stemming from the release of identified data has led to many legal restrictions on publishing it openly. What is "microdata" — and why is it so powerful? We're exploring the challenges and opportunities of utilizing individual-level data. Read more about microdata in our "OpenData1" collection. In the arena of criminal justice, there is an unusual degree of variation in whether personally identifiable information (PII) is protected. On the one hand, some criminal justice data is entirely unprotected — think of mugshots on page one of the morning paper or details from public courtroom testimony. On the other hand, the field has many, many datasets that are protected with the same care and confidentiality granted to medical records. While the criminal justice system disseminates sensitive, PII-containing data every day, it restricts access to much of it. Knowing this, we sought to understand what is currently happening with this restricted information. How are the agencies which hold this valuable, individual-level data currently getting value from it? ..."