UK open government data: the results of the official audit

“This should be a good week for open government data in the UK. The British government is one of the key drivers in the Open Government Partnership, presently meeting in Brasilia, where it is being lauded for the way it has released a ‘tsunami of data’. And yet, according to the National Audit Office, all is not entirely rosy. Read between the lines of its report out today, Implementing Transparency, and you will see a government which has been chucking out tonnes of data, that no-one looks at and without a complete strategy. Oh and it's cost an awful lot of money. Crucially, it found the Cabinet office seemed to have no idea of how much the transparency agenda would cost or what it would do: ‘The Cabinet Office … not yet systematically assessed the costs and benefits of the Government's specific transparency initiatives’ Also that the Cabinet Office hadn't done enough to make sure departments knew how to release the data, making it less useful: ‘The Cabinet Office has not yet defined how departments should prepare and disclose data inventories to facilitate wider use’ How bad are things? These are the key findings. The report praises the government for the sheer amount of information that it has released. Data.gov.uk now has 8,300 datasets on it - compared to 5,786 on data.gov (the US version). It has become possibly the biggest government-owned open data site in the world. At the same time, the government has done most of what it said it would. David Cameron wrote two open letters to heads of departments in May 2010, having just come into power, and in July 2011 – you can read the text here. They called for the releases of key datasets, and the report finds that 23 out of 25 key commitments had been achieved. And, besides those 'standard' datasets it has also published the big ones of the government's data vaults: such as the sentencing data by courtand GP prescribing data by practice... The report lifts the veil on how much the government spends on open data... The report shows that [1[ Government departments reckon on spending from £53,000 to £500,000 each year on just providing and publishing open data [2] data.gov.uk was originally run by the Central Office of Information and received funding of £1.2m in 2010-11 from the Department for Business, Innovation and Skills. In 2011-12, the project was brought inside the Cabinet Office, and what the report calls "further engagement activity with stakeholders" increased the annual running costs to £2m [3] The police crime maps cost £300,000 to set up and have annual running costs of more than £150,000. The National Policing Improvement Agency has budgeted £216,000 in 2011-12 to further develop the site, including linking crime data to police and justice outcomes [4] the Department for International Development estimates that to deliver the commitment to provide full information on international development projects with a value of more than £500 by January 2011, it incurred capital costs of £250,000, administrative set-up costs of £156,000 and has ongoing annual running costs of £64,000... While data.gov.uk has had more than 1.75m visits since it was launched in January 2010 (which is pretty much what the Datablog gets in a good month) most of its visitors leave from either the home page or the data page on the website. Page views for transparency data on the Ministry of Justice website represented just 0.02% of the overall site traffic from April to September 2011... The NAO does point out that... ‘there has been much greater interest in releases related to the operation of public services. The police crime map website has had an estimated 47m visits between February and December 2011. The Department for Education has reported an 84% increase in the use of its comparative data on schools, compared with the same period last year, since it was consolidated in one location and data were made more accessible’ We were interested in that 47m figure for the crime maps site and tested it using Nielsen data. There is no guidance on what exactly constitutes 'visits' - is it page views or unique users? Our figures show that while the site did get a lot of visitors when it was initially launched in February last year - and had a brief peak during the England riots last year (ironically, the data on the site is all historical, so visitors looking for riot offences would have been disappointed), in December it appeared to only have 47,000 viewers, looking at 364,000 pages... One of the most significant commitments by the coalition was the idea that councils should start publishing detailed spending data - with every item over £500. It was supposed to herald a new age of local accountability, with councils publishing information in a format we could all use. As part of putting the report together, the NAO sampled the data releases of 202 councils. It found that the vast majority (89%) had published the data by month - with one doing it very week. They also found that 91% had published the data in a decent format - ie, not pdfs but in excel or as CSV files. 7% of the sample had ancient