Are stakeholders measuring the publishing metrics that matter?: Putting research into context

"Perhaps the most fundamental aspect of compiling and implementing more meaningful research metrics that the NISO panelists discussed is the importance of putting data into context. And, as the speakers noted, there are multiple facets of context to consider, including:

• The strengths and limitations of different metrics by discipline/subject matter (e.g., some metrics are better suited to certain types of research)
• The intended uses and overall strengths and limitations of particular data points (e.g., altmetrics are “indicators” of impact, not measures of quality and the JIF was never meant to be used to measure the impact of individual articles or scholars)
• The cultural context that a researcher is operating within and the opportunities, challenges, and biases they have experienced
• How and where a research output fits within scholars’ other professional contributions (e.g., recognizing how individual research outputs are part of broader bodies of work and also measuring the impacts of scholarly outputs that do not fit within traditional publication-based assessment systems) ..."