Defining, Measuring, and Rewarding Scholarly Impact: Mind the Level of Analysis

Abstract:  We address the pervasive practice of using a journal's impact factor (JIF) as a proxy to assess individual researcher and article scholarly impact, and the grossly incorrect inferences this process generates. This invalid practice occurs because of confusion about the definition and measurement of impact at different levels of analysis. We illustrate the severity of the errors that occur when using JIF to evaluate individual scholarly impact, and advocate for an immediate moratorium on the exclusive use of JIF and other journal-level (i.e., higher level of analysis) measures when assessing the impact of individual researchers and individual articles (i.e., lower level of analysis). Given the importance and interest in assessing the scholarly impact of researchers and articles, we delineate measures that are level-appropriate and readily available. We discuss implications for the careers of researchers and educators and the administration and future of business schools, and conclude with actionable recommendations for internal and external stakeholders regarding the assessment of scholarly impact that can drive positive change in academic environments.

[Undated]