Improving the reliability of literature reviewsdetection of retracted articles through academic search engines

  1. Elena Pastor Ramón
  2. Iván Herrera Peco 1
  3. Oskia Agirre 2
  4. María García-Puente
  5. José María Morán 3
  1. 1 Universitat Pompeu Fabra

    Universitat Pompeu Fabra

    Barcelona, España


  2. 2 Universidad del País Vasco/Euskal Herriko Unibertsitatea

    Universidad del País Vasco/Euskal Herriko Unibertsitatea

    Lejona, España


  3. 3 Universidad de Extremadura

    Universidad de Extremadura

    Badajoz, España


EJIHPE: European Journal of Investigation in Health, Psychology and Education

ISSN: 2174-8144

Any de publicació: 2022

Volum: 12

Número: 5

Pàgines: 458-464

Tipus: Article

DOI: 10.3390/EJIHPE12050034 DIALNET GOOGLE SCHOLAR lock_openDialnet editor

Altres publicacions en: EJIHPE: European Journal of Investigation in Health, Psychology and Education


Cites rebudes

  • Cites en Scopus: 2 (24-09-2023)
  • Citas en Web of Science: 1 (20-09-2023)
  • Cites en Dimensions: 1 (23-03-2023)

SCImago Journal Rank

  • Any 2022
  • Impacte SJR de la revista: 0.542
  • Quartil major: Q2
  • Àrea: Clinical Psychology Quartil: Q2 Posició en l'àrea: 138/307
  • Àrea: Applied Psychology Quartil: Q3 Posició en l'àrea: 128/249
  • Àrea: Developmental and Educational Psychology Quartil: Q3 Posició en l'àrea: 177/349

Índice Dialnet de Revistas

(Indicador corresponent a lúltim any disponible en aquest portal, any {0,number,#})
  • Any 2021
  • Impacte de la revista: 1,450
  • Àmbit: PSICOLOGÍA Quartil: C1 Posició en l'àmbit: 15/111
  • Àmbit: EDUCACIÓN Quartil: C1 Posició en l'àmbit: 20/228


  • Ciències Socials: B

Scopus CiteScore

  • Any 2022
  • CiteScore de la revista: 3.5
  • Àrea: Clinical Psychology Percentil: 61
  • Àrea: Developmental and Educational Psychology Percentil: 59
  • Àrea: Applied Psychology Percentil: 54

Journal Citation Indicator (JCI)

  • Any 2022
  • JCI de la revista: 0.87
  • Quartil major: Q2
  • Àrea: PSYCHOLOGY, CLINICAL Quartil: Q2 Posició en l'àrea: 68/178


(Dades actualitzats a data de 23-03-2023)
  • Cites totals: 1
  • Cites recents: 1


Nowadays, a multitude of scientific publications on health science are being developed that require correct bibliographic search in order to avoid the use and inclusion of retracted literature in them. The use of these articles could directly affect the consistency of the scientific studies and could affect clinical practice. The aim of the present study was to evaluate the capacity of the main scientific literature search engines, both general (Gooogle Scholar) and scientific (PubMed, EMBASE, SCOPUS, and Web of Science), used in health sciences in order to check their ability to detect and warn users of retracted articles in the searches carried out. The sample of retracted articles was obtained from RetractionWatch. The results showed that although Google Scholar was the search engine with the highest capacity to retrieve selected articles, it was the least effective, compared with scientific search engines, at providing information on the retraction of articles. The use of different scientific search engines to retrieve as many scientific articles as possible, as well as never using only a generic search engine, is highly recommended. This will reduce the possibility of including retracted articles and will avoid affecting the reliability of the scientific studies carried out.

Referències bibliogràfiques

  • Ali, P.A.; Watson, R. Peer review and the publication process. Nurs. Open 2016, 3, 193–202.
  • Audisio, K.; Robinson, N.B.; Soletti, G.J.; Cancelli, G.; Dimagli, A.; Spadaccio, C.; Olaria, R.P.; Chadow, D.; Rahouma, M.; Demetres, M.; et al. A survey of retractions in the cardiovascular literature. Int. J. Cardiol. 2021, 349, 109–114.
  • Bar-Ilan, J.; Halevi, G. Temporal characteristics of retracted articles. Scientometrics 2018, 116, 1771–1783.
  • Bucci, E.M. On zombie papers. Cell Death Dis. 2019, 10, 189.
  • Budd, J.M.; Sievert, M.; Schultz, T.R.; Scoville, C. Effects of article retraction on citation and practice in medicine. Bull. Med. Libr. Assoc. 1999, 87, 437–443.
  • Chapman, D. Health-related databases. J. Can. Acad. Child. Adolesc. Psychiatry 2009, 18, 148–149.
  • Couzin, J.; Unger, K. Cleaning up the Paper Trail. Science 2006, 312, 38–43.
  • Dal-Ré, R.; Ayuso, C. For how long and with what relevance do genetics articles retracted due to research misconduct remain active in the scientific literature. Account. Res. 2020, 28, 280–296.
  • Ferragut, J.; Pinto, N.; Amorim, A.; Picornell, A. Improving publication quality and the importance of Post Publication Peer Review: The illustrating example of X chromosome analysis and calculation of forensic parameters. Forensic Sci. Int. Genet. 2019, 38, e5–e7.
  • Frampton, G.; Woods, L.; Scott, D.A. Inconsistent and incomplete retraction of published research: A cross-sectional study on COVID-19 retractions and recommendations to mitigate risks for research, policy and practice. PLoS ONE 2021, 16, e0258935.
  • García-Puente, M.; Pastor-Ramon, E.; Agirre, O.; Morán, J.-M.; Herrera-Peco, I. Research note. Open letter to the users of the new PubMed: A critical appraisal. Prof. Inf. 2020, 29, e290336.
  • García-Puente, M.; Pastor-Ramon, E.; Agirre, O.; Moran, J.M.; Herrera-Peco, I. The use of Sci-Hub in systematic reviews of the scholarly literature. Clin. Implant Dent. Relat. Res. 2019, 21, 816.
  • Gasparyan, A.Y.; Ayvazyan, L.; Kitas, G. Multidisciplinary Bibliographic Databases. J. Korean Med. Sci. 2013, 28, 1270–1275.
  • Gehanno, J.-F.; Rollin, L.; Darmoni, S. Is the coverage of google scholar enough to be used alone for systematic reviews. BMC Med. Inform. Decis. Mak. 2013, 13, 7.
  • Giustini, D.; Boulos, M.N.K. Google Scholar is not enough to be used alone for systematic reviews. Online J. Public Health Inform. 2013, 5, 214.
  • Gusenbauer, M.; Haddaway, N.R. Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Res. Synth. Methods 2019, 11, 181–217.
  • Herrera-Peco, I.; Santillán-García, A.; Morán, J.; Goodman-Casanova, J.; Cuesta-Lozano, D. The Evidence-Based Practice Silent Enemy: Retracted Articles and Their Use in Systematic Reviews. Healthcare 2020, 8, 465.
  • King, E.G.; Oransky, I.; Sachs, T.E.; Farber, A.; Flynn, D.; Abritis, A.; Kalish, J.A.; Siracuse, J.J. Analysis of retracted articles in the surgical literature. Am. J. Surg. 2018, 216, 851–855.
  • Morán, J.M.; Santillán-García, A.; Herrera-Peco, I. SCRUTATIOm: How to detect retracted literature included in systematics reviews and metaanalysis using SCOPUS© and ZOTERO©. Gac. Sanit. 2020, 36, 64–66.
  • Moylan, E.C.; Kowalczuk, M.K. Why articles are retracted: A retrospective cross-sectional study of retraction notices at BioMed Central. BMJ Open 2016, 6, e012047.
  • Nair, S.; Yean, C.; Yoo, J.; Leff, J.; Delphin, E.; Adams, D.C. Reasons for article retraction in anesthesiology: A comprehensive analysis. Can. J. Anaesth. 2019, 67, 57–63.
  • Neale, A.V.; Northrup, J.; Dailey, R.; Marks, E.; Abrams, J. Correction and use of biomedical literature affected by scientific misconduct. Sci. Eng. Ethics 2007, 13, 5–24.
  • Nogueira, T.E.; Gonçalves, A.S.; Leles, C.R.; Batista, A.C.; Costa, L.R. A survey of retracted articles in dentistry. BMC Res. Notes 2017, 10, 253.
  • Orduña Malea, E.; Martín-Martín, A.; Delgado-López-Cózar, E. Google Scholar as a source for scholarly evaluation: A bibliographic review of database errors. Rev. Esp. Doc. Cient. 2017, 40, 1–33.
  • Rapani, A.; Lombardi, T.; Berton, F.; Del Lupo, V.; Di Lenarda, R.; Stacchi, C. Retracted publications and their citation in dental literature: A systematic review. Clin. Exp. Dent. Res. 2020, 6, 383–390.
  • Scite: Evaluate the Veracity of Scientific Work.
  • Serghiou, S.; Marton, R.M.; Ioannidis, J.P.A. Media and social media attention to retracted articles according to Altmetric. PLoS ONE 2021, 16, e0248625.
  • Singh, V.K.; Singh, P.; Karmakar, M.; Leta, J.; Mayr, P. The journal coverage of Web of Science, Scopus and Dimensions: A comparative analysis. Scientometrics 2021, 126, 5113–5142.
  • Sox, H.C.; Rennie, D. Research Misconduct, Retraction, and Cleansing the Medical Literature: Lessons from the Poehlman Case. Ann. Intern. Med. 2006, 144, 609–613.
  • The Retraction Watch Database [Internet]. New York: The Center for Scientific Integrity. 2018.
  • Theis-Mahon, N.R.; Bakker, C.J. The continued citation of retracted publications in dentistry. J. Med. Libr. Assoc. 2020, 108, 389–397.