“... 'Measuring the impact of research through the traditional methods—counting citations in published literature—is important, but it doesn’t tell the whole story,' said Timothy Deliyannides, Director of the Office of Scholarly Communication and Publishing and Head of Information Technology for the University of Pittsburgh.
Pittsburgh recently wrapped up a pilot test with Plum Analytics, one of several new companies in the emerging field of altmetrics. By examining how often a paper is downloaded, mentioned in the news, or linked to on social media sites, altmetrics providers offer researchers, funding agencies, and librarians a more immediate, quantifiable view of the impact an article is having on its field. The company in January announced the official launch of PlumX, a tool that harvests and aggregates alternative impact metrics in five major categories: usage, captures, mentions, social media, and citations.
Altmetrics 'gives a whole new layer, looking at additional dimensions that used to be hidden, and it’s so much more timely,' Deliyannides said. 'We can start seeing where research has an impact right away.'
When explaining altmetrics, the concept of tracking Facebook 'likes,' or links on Twitter and other social media is often used as a straightforward point of reference. But social media tracking represents only one aspect of the field, noted Plum Analytics co-founder Mike Buschman. Mentions in news outlets, for example, are tracked separately from social media. And, usage metrics track how often a book has been checked out or how often a paper has been accessed online. And of course, classic citations are included as a metric. There’s also the notion of 'capturing' works. If someone reads an article and then bookmarks it, or saves the bibliographic information to their Zotero account, 'that’s a different metric that in some ways is a little more powerful even than usage,' Buschman explained. 'It’s the notion that I’ve read it, or I’ve read enough of it that I know I want to come back to it. Or I want someone else to come back to it.'Currently, the field is still studying how these different metrics interact. An article that generated a large volume of mentions on social media, but was rarely downloaded or bookmarked may have been more controversial than substantive, for example. And weighting these different metrics, for now, can be somewhat subjective. 'What is the relationship to a tweet versus a PDF download versus a slideshow favorite?' Buschman asked. It’s a field that is ripe for the type of analysis to which librarians are ideally suited, he said. Academic librarians could maintain more relevance in the research process by studying these metrics and helping students and faculty get more impact out of their research output and publications... At the University of Pittsburgh, “I think some researchers weren’t aware of the extent to which their research was talked about in social media,” Deliyannides said. 'And then you’ve got communities of scholars developed around services like Mendeley, for instance. And I don’t think researchers are always aware which of their works are most widely downloaded, saved, referred to, or commented on. These are a whole range of measures that are different from the citation counts that still show impact. There were some surprises for some of the researchers there.' Altmetrics tools also help analyze the impact of a research group in aggregate, he added. While only one or two principal investigators might be mentioned in a journal article or monograph, 'you’ve got all kinds of other people on the research team that are talking about the research and sharing it in different venues. So, some of the impact there that was invisible before becomes visible with a tool like Plum Analytics ...'”