Open Data Charter Measurement Guide - URENIO Watch

"This guide is a collaborative effort of the Charter’s Measurement and Accountability Working Group (MAWG) of the Open Knowledge International. It analyses the Open Data Charter principles and how they are assessed based on current open government data measurement tools. Governments, civil society, journalists, and researchers may use it to better understand how they can measure open data activities according to the Charter principles.

Executive Summary

The Measurement Guide helps governments, civil society, and researchers to understand how to assess open data activities based on the Open Data Charter (the ‘Charter’) principles. It seeks to shed light on the often opaque and jargon-filled world of open data measurement. The Measurement Guide is an analysis of the Charter principles and how they are assessed based on current open government data measurement tools – with a focus on commitments that can be measured, commitments that cannot be measured, and existing gaps (e.g. commitments that have not been measured). The Measurement Guide is made for governments, civil society, and researchers to understand how the Charter principles can be measured. It provides an analysis of the indicators, which includes comprehensive tables of global indicators (e.g. indicator tables) per each Charter principle. For governments, the guide summarizes the most important insights in this section, the Executive Summary. For civil society and communicators, the indicator tables and our analysis provide transparency about existing measurement tools (‘Five open data assessment tools’) and what they measure. This can help civil society to oversee the progress of open data policy at a country level. For researchers, the guide explains the methodology to map open data indicators against Charter commitments. The indicator tables created can be used to compare existing data measurement tools and develop new indicators.

The Measurement Guide provides insights from open data experts and members of organizations who work on open data measurement tools. Analysis of the coverage of the five leading open data measurement tools – the Open Data Barometer (ODB), Global Open Data Index (GODI), Open Data Inventory (ODIN), Open Useful Reusable Government Data (OURdata), and the European Open Data Maturity Assessment (EODMA) – reveals that only parts of Charter principle commitments, and their components, are being measured; or that some commitments could be measured in the future. However, some Charter concepts are either too broad (e.g. “high-quality data”, “usability by the widest range of users”), or lack a shared interpretation, which makes them difficult to find a common indicator.

The Measurement Guide also covers how existing indicators metrify key open data concepts. It is important to note that not all aspects of a commitment are clearly defined. Multiple ways of measuring currently exist for some commitments. Some commitments need to be defined and measured on a country-by-country basis to incorporate local context...."