Evaluation of untrustworthy journals: Transition from formal criteria to a complex view

Not all the journals included in credible indexes meet the ethical rules of COPE, DOAJ, OASPA and WAME and equally there may be trustworthy journals excluded from these indexes which means they cannot be used as definitive whitelists for trustworthy journals. Equally the many methods suggested to determine trustworthiness are not reliable due to including questionable criteria. The question arises whether valid criteria for identifying an untrustworthy journal can be determined and whether other assessment procedures are necessary. Since 2017, the Masaryk University Campus Library has been developing a suitable evaluation method for journals. A list of 19 criteria based on those originally suggested by Beall, COPE, DOAJ, OASPA and WAME were reduced to 10 objectively verifiable criteria following two workshops with librarians. Anevaluation of 259 biomedical journals using both the list of 19 and then 10 criteria revealed that 74 journals may have been incorrectly assessed as untrustworthy using the longer list. The most common reason for failure to comply was in the provision of sufficient editorial information and declaration of article processing charges. However our investigation revealed that no criteria can reliably identify predatory journals. Therefore, a complex evaluation is needed combining objectively verifiable criteria with analysis of a journal’s content and knowledge of the journal’s background.