Assessing the Coverage of PubMed, Embase, OpenAlex and Semantic Scholar for Automated Single Database Searches in Living Guideline Evidence Surveillance: A Case Study of the International PCOS Guidelines 2023
database[Title] 2025-04-20
J Clin Epidemiol. 2025 Apr 16:111789. doi: 10.1016/j.jclinepi.2025.111789. Online ahead of print.
ABSTRACT
BACKGROUND: Living guideline maintenance is underpinned by manual approaches towards evidence retrieval, limiting long term sustainability. Our study aimed to evaluate the feasibility of using only PubMed, Embase, OpenAlex or Semantic Scholar in automatically retrieving articles that were included in a high-quality international guideline - the 2023 International Polycystic Ovary Syndrome (PCOS) Guidelines.
METHODS: The digital object identifiers (DOIs) and PubMed ID (PMIDs) of articles included after full text screening in the 2023 International PCOS Guidelines were extracted. These IDs were used to automatically retrieve article metadata from all tested databases. A title only search was then conducted on articles that were not initially retrievable. The extent of coverage, and overlap of coverage, was determined for each database. An exploratory analysis of the risk of bias of articles that were unretrievable was then conducted for each database.
RESULTS: OpenAlex had the best coverage (98.6%), followed by Semantic Scholar (98.3%), Embase (96.8%) and PubMed (93.0%). However, 90.5% of all articles were retrievable from all four databases. All articles that were not retrievable from OpenAlex and Semantic Scholar were either assessed as medium or high risk of bias. In contrast, both Embase and PubMed missed articles that were of high quality (low risk of bias).
CONCLUSION: OpenAlex should be considered as a single source for automated evidence retrieval in living guideline development, due to high coverage, and low risk of missing high-quality articles. These insights are being leveraged as part of transitioning the 2023 International PCOS Guidelines towards a living format.
PMID:40250535 | DOI:10.1016/j.jclinepi.2025.111789