Improving the reliability of literature reviewsdetection of retracted articles through academic search engines

  1. Elena Pastor Ramón
  2. Iván Herrera Peco 1
  3. Oskia Agirre 2
  4. María García-Puente
  5. José María Morán 3
  1. 1 Universitat Pompeu Fabra

    Universitat Pompeu Fabra

    Barcelona, España


  2. 2 Universidad del País Vasco/Euskal Herriko Unibertsitatea

    Universidad del País Vasco/Euskal Herriko Unibertsitatea

    Lejona, España


  3. 3 Universidad de Extremadura

    Universidad de Extremadura

    Badajoz, España


EJIHPE: European Journal of Investigation in Health, Psychology and Education

ISSN: 2174-8144

Argitalpen urtea: 2022

Alea: 12

Zenbakia: 5

Orrialdeak: 458-464

Mota: Artikulua

DOI: 10.3390/EJIHPE12050034 DIALNET GOOGLE SCHOLAR lock_openDialnet editor

Beste argitalpen batzuk: EJIHPE: European Journal of Investigation in Health, Psychology and Education


Jasotako aipamenak

  • Aipamenak Scopus atarian: 2 (24-09-2023)
  • Aipamenak 'Web of Science' atarian: 1 (20-09-2023)
  • Aipamenak Dimensions atarian: 1 (23-03-2023)

SCImago Journal Rank

  • Urtea 2022
  • Aldizkariaren SJR eragina: 0.542
  • Kuartil nagusia: Q2
  • Arloa: Clinical Psychology Kuartila: Q2 Postua arloan: 138/307
  • Arloa: Applied Psychology Kuartila: Q3 Postua arloan: 128/249
  • Arloa: Developmental and Educational Psychology Kuartila: Q3 Postua arloan: 177/349

Índice Dialnet de Revistas

(Atari honetan eskuragarri dagoen azken urteko adierazlea, 2021 urtea)
  • Urtea 2021
  • Aldizkariaren eragin faktorea: 1,450
  • Eremua: PSICOLOGÍA Kuartila: C1 Postua eremuan: 15/111
  • Eremua: EDUCACIÓN Kuartila: C1 Postua eremuan: 20/228


  • Gizarte Zientziak: B

Scopus CiteScore

  • Urtea 2022
  • Aldizkariaren CiteScore-a: 3.5
  • Arloa: Clinical Psychology Pertzentila: 61
  • Arloa: Developmental and Educational Psychology Pertzentila: 59
  • Arloa: Applied Psychology Pertzentila: 54

Journal Citation Indicator (JCI)

  • Urtea 2022
  • Aldizkariaren JCI eragina: 0.87
  • Kuartil nagusia: Q2
  • Arloa: PSYCHOLOGY, CLINICAL Kuartila: Q2 Postua arloan: 68/178


(23-03-2023 datan eguneratutako datuak)
  • Aipamenak guztira: 1
  • Azken hitzorduak: 1


Nowadays, a multitude of scientific publications on health science are being developed that require correct bibliographic search in order to avoid the use and inclusion of retracted literature in them. The use of these articles could directly affect the consistency of the scientific studies and could affect clinical practice. The aim of the present study was to evaluate the capacity of the main scientific literature search engines, both general (Gooogle Scholar) and scientific (PubMed, EMBASE, SCOPUS, and Web of Science), used in health sciences in order to check their ability to detect and warn users of retracted articles in the searches carried out. The sample of retracted articles was obtained from RetractionWatch. The results showed that although Google Scholar was the search engine with the highest capacity to retrieve selected articles, it was the least effective, compared with scientific search engines, at providing information on the retraction of articles. The use of different scientific search engines to retrieve as many scientific articles as possible, as well as never using only a generic search engine, is highly recommended. This will reduce the possibility of including retracted articles and will avoid affecting the reliability of the scientific studies carried out.

Erreferentzia bibliografikoak

  • Ali, P.A.; Watson, R. Peer review and the publication process. Nurs. Open 2016, 3, 193–202.
  • Audisio, K.; Robinson, N.B.; Soletti, G.J.; Cancelli, G.; Dimagli, A.; Spadaccio, C.; Olaria, R.P.; Chadow, D.; Rahouma, M.; Demetres, M.; et al. A survey of retractions in the cardiovascular literature. Int. J. Cardiol. 2021, 349, 109–114.
  • Bar-Ilan, J.; Halevi, G. Temporal characteristics of retracted articles. Scientometrics 2018, 116, 1771–1783.
  • Bucci, E.M. On zombie papers. Cell Death Dis. 2019, 10, 189.
  • Budd, J.M.; Sievert, M.; Schultz, T.R.; Scoville, C. Effects of article retraction on citation and practice in medicine. Bull. Med. Libr. Assoc. 1999, 87, 437–443.
  • Chapman, D. Health-related databases. J. Can. Acad. Child. Adolesc. Psychiatry 2009, 18, 148–149.
  • Couzin, J.; Unger, K. Cleaning up the Paper Trail. Science 2006, 312, 38–43.
  • Dal-Ré, R.; Ayuso, C. For how long and with what relevance do genetics articles retracted due to research misconduct remain active in the scientific literature. Account. Res. 2020, 28, 280–296.
  • Ferragut, J.; Pinto, N.; Amorim, A.; Picornell, A. Improving publication quality and the importance of Post Publication Peer Review: The illustrating example of X chromosome analysis and calculation of forensic parameters. Forensic Sci. Int. Genet. 2019, 38, e5–e7.
  • Frampton, G.; Woods, L.; Scott, D.A. Inconsistent and incomplete retraction of published research: A cross-sectional study on COVID-19 retractions and recommendations to mitigate risks for research, policy and practice. PLoS ONE 2021, 16, e0258935.
  • García-Puente, M.; Pastor-Ramon, E.; Agirre, O.; Morán, J.-M.; Herrera-Peco, I. Research note. Open letter to the users of the new PubMed: A critical appraisal. Prof. Inf. 2020, 29, e290336.
  • García-Puente, M.; Pastor-Ramon, E.; Agirre, O.; Moran, J.M.; Herrera-Peco, I. The use of Sci-Hub in systematic reviews of the scholarly literature. Clin. Implant Dent. Relat. Res. 2019, 21, 816.
  • Gasparyan, A.Y.; Ayvazyan, L.; Kitas, G. Multidisciplinary Bibliographic Databases. J. Korean Med. Sci. 2013, 28, 1270–1275.
  • Gehanno, J.-F.; Rollin, L.; Darmoni, S. Is the coverage of google scholar enough to be used alone for systematic reviews. BMC Med. Inform. Decis. Mak. 2013, 13, 7.
  • Giustini, D.; Boulos, M.N.K. Google Scholar is not enough to be used alone for systematic reviews. Online J. Public Health Inform. 2013, 5, 214.
  • Gusenbauer, M.; Haddaway, N.R. Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Res. Synth. Methods 2019, 11, 181–217.
  • Herrera-Peco, I.; Santillán-García, A.; Morán, J.; Goodman-Casanova, J.; Cuesta-Lozano, D. The Evidence-Based Practice Silent Enemy: Retracted Articles and Their Use in Systematic Reviews. Healthcare 2020, 8, 465.
  • King, E.G.; Oransky, I.; Sachs, T.E.; Farber, A.; Flynn, D.; Abritis, A.; Kalish, J.A.; Siracuse, J.J. Analysis of retracted articles in the surgical literature. Am. J. Surg. 2018, 216, 851–855.
  • Morán, J.M.; Santillán-García, A.; Herrera-Peco, I. SCRUTATIOm: How to detect retracted literature included in systematics reviews and metaanalysis using SCOPUS© and ZOTERO©. Gac. Sanit. 2020, 36, 64–66.
  • Moylan, E.C.; Kowalczuk, M.K. Why articles are retracted: A retrospective cross-sectional study of retraction notices at BioMed Central. BMJ Open 2016, 6, e012047.
  • Nair, S.; Yean, C.; Yoo, J.; Leff, J.; Delphin, E.; Adams, D.C. Reasons for article retraction in anesthesiology: A comprehensive analysis. Can. J. Anaesth. 2019, 67, 57–63.
  • Neale, A.V.; Northrup, J.; Dailey, R.; Marks, E.; Abrams, J. Correction and use of biomedical literature affected by scientific misconduct. Sci. Eng. Ethics 2007, 13, 5–24.
  • Nogueira, T.E.; Gonçalves, A.S.; Leles, C.R.; Batista, A.C.; Costa, L.R. A survey of retracted articles in dentistry. BMC Res. Notes 2017, 10, 253.
  • Orduña Malea, E.; Martín-Martín, A.; Delgado-López-Cózar, E. Google Scholar as a source for scholarly evaluation: A bibliographic review of database errors. Rev. Esp. Doc. Cient. 2017, 40, 1–33.
  • Rapani, A.; Lombardi, T.; Berton, F.; Del Lupo, V.; Di Lenarda, R.; Stacchi, C. Retracted publications and their citation in dental literature: A systematic review. Clin. Exp. Dent. Res. 2020, 6, 383–390.
  • Scite: Evaluate the Veracity of Scientific Work.
  • Serghiou, S.; Marton, R.M.; Ioannidis, J.P.A. Media and social media attention to retracted articles according to Altmetric. PLoS ONE 2021, 16, e0248625.
  • Singh, V.K.; Singh, P.; Karmakar, M.; Leta, J.; Mayr, P. The journal coverage of Web of Science, Scopus and Dimensions: A comparative analysis. Scientometrics 2021, 126, 5113–5142.
  • Sox, H.C.; Rennie, D. Research Misconduct, Retraction, and Cleansing the Medical Literature: Lessons from the Poehlman Case. Ann. Intern. Med. 2006, 144, 609–613.
  • The Retraction Watch Database [Internet]. New York: The Center for Scientific Integrity. 2018.
  • Theis-Mahon, N.R.; Bakker, C.J. The continued citation of retracted publications in dentistry. J. Med. Libr. Assoc. 2020, 108, 389–397.