Impact and Learning: Open knowledge spells murky waters for M & E

"In mid-January I ran a session on monitoring and evaluation at the Eldis Open Knowledge Hub Partnerships Meeting. The meeting housed a group of individuals united by a concern with opening up access to research evidence and, particularly, increasing the visibility of research from developing countries. The partnerships meeting was undertaken as part of the Global Open Knowledge Hub (GOKH) – a 3 year DFID funded project. The vision for GOKH is that IDS, and partners, will build on their existing information services to create an open data architecture for exchange and sharing of research evidence – the so-called Hub. For insight into the issues that need to be addressed in trying to set up an open knowledge hub see Radhika Menon’s recent blog The Global Open Knowledge Hub: building a dream machine-readable world. Our hope is that through the open data approach the partners and third-party users of the Hub will be in a position to extract and re-purpose information about research evidence that is relevant and contextual to their audiences. This in turn will contribute to research content being more visible thereby enabling otherwise unheard voices to contribute to global debate and decision making. My session on M & E then was concerned with how we can know if this is being achieved ... There are ways to implement tracking but these are easily circumvented, so the statistics you can obtain do not reliably represent the whole picture. So, depending on how they implement the content, if organisation A is consuming data from the hub  that organisation B has contributed to the Hub then it may be that the ‘hits’ register on organisation A’s web statistics, not organisation B’s. Even if/when we do identify the most suitable metric for measuring impact in open knowledge, as we discussed at the workshop, numbers aren’t really enough ..."