York librarian validates social media metrics for research impact evaluation

“York Business Librarian Xuemei Li has become the first researcher to ever have a study validating the usefulness of altmetrics published in an academic journal. Altmetrics is the study of social media metrics used for analyzing and informing scholarship. Li’s first research study was published in April, 2012 in Scientometrics and Li’s second study was accepted by the 17th International Conference on Science and Technology Indicatorstaking place September 2012. ‘Researchers are integrating various social media tools such as blogs, wikis, Twitter and social bookmarks into their research processes to save, organize, share, and disseminate various research sources. It is even more difficult for traditional bibliometric indicators to capture the totality of research influence on the web,’ Li explains. ‘Nevertheless, the traces left by researchers and the general public through those social media tools hold big potential for measuring different research influences, and this is what altmetrics aims to measure. Altmetrics can be used to complement traditional citation-based measurements.’ Li’s first study of altmetrics sampled 1613 papers published in Nature and Science in 2007 and compared citations with reader counts. She found significant statistical correlations between citations from Web of Science (now the Web of Knowledge) and Google Scholar and reader counts from the social media bookmark tools CiteULike and Mendeley. The findings suggest that the type of scholarly influence one’s research has  – as measured by these social media tools  – is related to traditional citation-based impact.Li’s second study compared nearly 1400 Faculty of 1000 (F1000) post-publication peer reviews and Mendeley usage data with traditional bibliometric indicators. This study suggests that F1000 – a database that stores only the best quality biomedical articles after they’ve been published, as selected by over 10,000 faculty members worldwide – is good at acknowledging the merit of an article from (the F1000) experts’ point of view while Mendeley reader counts are more closely related to citation counts. ‘Faculty are striving to demonstrate the impact of their research in a world where the web has become a critical communications channel,’ says Cynthia Archer, York University librarian. ‘Li’s ground-breaking research serves to validate the usefulness of social media based altmetrics to monitor and track faculty research impact.’ Li and other researchers are working hard to identify, monitor, and evaluate potential social media tools towards building reliable altmetric indicators.”