Times Higher Education - Research Intelligence - Alt-metrics: fairer, faster impact data?

“For those who long for scientific assessment to get beyond journal impact factors, the road to Mendeley could well also lead to redemption.  The discontent with the status quo was eloquently demonstrated last week in a widely read blog post by Stephen Curry, professor of structural biology at Imperial College London, which began: ‘I am sick of impact factors and so is science...’ The perceived statistical illiteracy of such practices - which, according to Professor Curry, are spreading like a ‘cancer’ through the academy - stems from the fact that impact factors are typically skewed by a small number of papers with a large number of citations. Most of the other papers in a journal will have had fewer citations than the impact factor figure... Professor Curry... said he preferred metrics that tapped into the kinds of views and expertise typically exchanged during coffee breaks at academic conferences. And, for him, the closest measurable approximations to such interactions occur on social networking sites... Because their primary function is typically to allow scientists to share and discuss papers, online resources such as Zotero, ResearchGate, CiteULike, BibSonomy and Mendeley particularly lend themselves to the development of what are often known as alternative - or ‘alt’ - metrics... According to Mendeley's co-founder and chief executive Victor Henning, the 65 million unique documents it contains - uploaded by nearly 2 million users - make it about 30 per cent larger even than mainstream citation databases Scopus and the Web of Knowledge, while three recent studies estimated that its coverage of current peer-reviewed research papers, either via abstracts or full documents, was between 93 and 98 per cent.  Some publishers have even begun to supply the site with abstracts and previews of their articles. Springer is one such, and according to Wim van der Stelt, its executive vice-president for corporate strategy, the move has already boosted traffic to the full versions of articles on its own site. He said Springer would soon be experimenting with displaying alt-metrics for its own articles.  Mendeley statistics are drawn on - with the site's blessing - by alt-metrics providers such as altmetric.com and total-impact.org. The latter measures how many public Mendeley groups - formed around specific topics - reference a paper, as well as how many people have added it or its abstract to their personal library, and how many of those people are students or from developing countries.  Dr Henning said Mendeley also tracks which documents and pages are read in its PDF viewer, and for how long... But David Colquhoun, professor of pharmacology at University College London, said...  ‘The people who propose things like alt-metrics are kids playing with computers, with no appreciation of what real science is and, worse, no appreciation that hypotheses need to be tested...’  For his part, he was highly sceptical that alt-metrics would ever be able meaningfully to measure a paper's quality, not least because they could be more easily ‘gamed’ than standard metrics. ‘The idea that Twitter will substitute for reading a paper is just ludicrous beyond words... If [alt-metrics] were taken seriously for selection and promotion, it would kill serious science,’ he said.  But for Cameron Neylon, director of advocacy at the Public Library of Science, alt-metrics aspire not to assess a paper's ‘universal quality’ but to indicate whether it is useful for a particular purpose. Sometimes I need data, sometimes methods, and sometimes I want the argument well laid out,’ he said. ‘What I really want to know is...whether [the paper was] useful for people like me doing things like those I'm doing. [Mendeley] bookmarks are one tool that can help with that, but we need the full suite of measures to really build discovery tools that will solve this problem.’”