OSF | Reimagining and diversifying assessment of the credibility of research findings

"Knowing whether research findings are worth taking seriously is critically important to inform decisions about what to study next, how to allocate resources, and whether the evidence is sufficient to translate into practical use. However, at least two challenges interfere with credibility assessment: it is very resource-intensive to do well, and current popular indicators have weak validity. Popular indicators of research credibility are whether the finding is published in a peer-reviewed journal, the prestige of that journal, and the number of times the findings have been cited. In addition to evidence of their weak validity as quality assessments (Dougherty & Horne, 2022), these indicators are, at best, simplistic, proxy indicators that do not provide any information about the qualities of the research itself. They are popular, nevertheless, partly because they are easy to compute. The SCORE program (Abatayo et al., 2026; Aczel et al., 2026; Miske et al., 2026) leaned into the challenge of creating valid, scalable indicators on one dimension of credibility -- replicability (Tyner et al., 2026)."