Skip to main content
SpringerLink
Log in

Google Scholar as a Sole Literature Source for Meta-analyses in Otolaryngology Head and Neck Surgery

  • Original Article
  • Published:
Indian Journal of Otolaryngology and Head & Neck Surgery Aims and scope Submit manuscript

Abstract

The strength of meta-analyses lies in the synthesis of data from multiple studies. Current guidelines require a thorough systematic search to maximize results, which usually includes searching multiple academic search systems (ASS). Google Scholar (GS) is considered a promising tool for searching the scientific literature. We aimed to determine whether GS is a valid and sufficient solitary data source for meta-analyses in the field of otolaryngology. Selected ENT-HNS journal was searched for meta-analyses published between 2010 and 2021 that adhered to the systematic reviews and meta-analyses guidelines and precisely followed the search algorithm. The latter was reproduced with GS, and the position of each enrolled study in each meta-analysis was determined. Ten meta-analyses were enrolled, the total number of search results ranged from 57 to 17,949. The number of GS search results was significantly greater than those of other ASS combinations (range 1,360–25,400, P = .006). The number of included papers for each meta-analysis ranged from 5 to 26. The position of all enrolled papers throughout GS searching was in the first 200 GS results in four of 10 meta-analyses. The reference lists of all included papers in the first 200 GS results identified 106 papers out of 108 (98%), while searching until the 500th GS output results identified 107 papers out of 108 papers (99%). GS can serve as a solitary ASS for systematic literature reviews and meta-analyses in the field of otolaryngology. Searching the first 500 or 200 results and including reference lists yields 99% and 98% coverage, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Stjernswärd J (1974) Decreased survival related to irradiation postoperatively in early operable breast cancer. Lancet 2(7892):1285–1286. https://doi.org/10.1016/s0140-6736(74)90142-1. (PMID: 4139524)

    Article  PubMed  Google Scholar 

  2. Leucht S, Chaimani A, Cipriani AS, Davis JM, Furukawa TA, Salanti G (2016) Network meta-analyses should be the highest level of evidence in treatment guidelines. Eur Arch Psychiatry Clin Neurosci 266(6):477–480. https://doi.org/10.1007/s00406-016-0715-4. (PMID: 27435721)

    Article  PubMed  Google Scholar 

  3. McCarthy JE, Chatterjee A, McKelvey TG, Jantzen EMG, Kerrigan CL (2010) A detailed analysis of level I evidence (randomized controlled trials and meta-analyses) in five plastic surgery journals to date: 1978 to 2009. Plast Reconstr Surg 126(5):1774–1778. https://doi.org/10.1097/PRS.0b013e3181efa201. (PMID: 21042137)

    Article  CAS  PubMed  Google Scholar 

  4. Dixon E, Hameed M, Sutherland F, Cook DJ, Doig C (2005) Evaluating meta-analyses in the general surgical literature: a critical appraisal. Ann Surg 241(3):450–459. https://doi.org/10.1097/01.sla.0000154258.30305.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ (1996) Users' guides to the medical literature. IX. A method for grading health care recommendations. Evidence-Based Medicine Working Group. JAMA. 1995 Dec 13;274(22):1800–4. doi: https://doi.org/10.1001/jama.274.22.1800. Erratum in: JAMA; 275(16):1232. PMID: 7500513

  6. Stevinson C, Lawlor DA (2004) Searching multiple databases for systematic reviews: added value or diminishing returns? Complement Ther Med 12(4):228–232. https://doi.org/10.1016/j.ctim.2004.09.003. (PMID: 15649836)

    Article  CAS  PubMed  Google Scholar 

  7. Lemeshow AR, Blum RE, Berlin JA, Stoto MA, Colditz GA (2005) Searching one or two databases was insufficient for meta-analysis of observational studies. J Clin Epidemiol 58(9):867–873. https://doi.org/10.1016/j.jclinepi.2005.03.004. (PMID: 16085190)

    Article  PubMed  Google Scholar 

  8. Beyer FR, Wright K (2013) Can we prioritise which databases to search? A case study using a systematic review of frozen shoulder management. Health Info Libr J 30(1):49–58. https://doi.org/10.1111/hir.12009. (Epub 2012 Dec 12 PMID: 23413793)

    Article  PubMed  Google Scholar 

  9. Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH (2017) Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev 6(1):1–12

    Article  Google Scholar 

  10. Gray JE, Hamilton MC, Hauser A, Janz MM, Peters JP, Taggart F (2012) Scholarish: google scholar and its value to the sciences. Issues in Science and Technology Librarianship, 70(Summer)

  11. Walters WH (2011) Comparative recall and precision of simple and expert searches in Google Scholar and eight other databases. Portal Libraries and the Academy 11(4):971–1006

    Article  Google Scholar 

  12. Anker MS, Hadzibegovic S, Lena A, Haverkamp W (2019) The difference in referencing in web of science, scopus, and google scholar. ESC Heart Fail. 6(6):1291–1312. https://doi.org/10.1002/ehf2.12583. (Epub 2019 Dec 30. PMID: 31886636; PMCID: PMC6989289)

    Article  PubMed  PubMed Central  Google Scholar 

  13. Gusenbauer M, Haddaway NR (2020) Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Res Synth Methods. 11(2):181–217. https://doi.org/10.1002/jrsm.1378. (Epub 2020 Jan 28. PMID: 31614060; PMCID: PMC7079055)

    Article  PubMed  PubMed Central  Google Scholar 

  14. Freeman MK, Lauderdale SA, Kendrach MG, Woolley TW (2009) Google Scholar versus PubMed in locating primary literature to answer drug-related questions. Ann Pharmacother 43(3):478–484. https://doi.org/10.1345/aph.1L223. (Epub 2009 Mar 3 PMID: 19261965)

    Article  PubMed  Google Scholar 

  15. Wadhwa V, Vilanilam GK, Chick JFB (2020) Disparities in citation metrics amongst web of science, scopus, and google scholar for interventional radiology journals. Cardiovasc Intervent Radiol 43(10):1583–1586. https://doi.org/10.1007/s00270-020-02535-0. (Epub 2020 Jun 3 PMID: 32494832)

    Article  PubMed  Google Scholar 

  16. Nourbakhsh E, Nugent R, Wang H, Cevik C, Nugent K (2012) Medical literature searches: a comparison of PubMed and Google Scholar. Health Info Libr J 29(3):214–222. https://doi.org/10.1111/j.1471-1842.2012.00992.x. (Epub 2012 Jun 19 PMID: 22925384)

    Article  PubMed  Google Scholar 

  17. Wang Y, Howard P (2012) Google scholar usage: an academic library’s experience. J Web Librariansh 6(2):94–108

    Article  CAS  Google Scholar 

  18. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6(7):1000097. https://doi.org/10.1371/journal.pmed.1000097. (Epub 2009 Jul 21. PMID: 19621072; PMCID: PMC2707599)

    Article  Google Scholar 

  19. Toth B, Gray JA, Brice A (2005) The number needed to read-a new measure of journal value. Health Inf Libr J 22(2):81–82

    Article  Google Scholar 

  20. Bramer WM, Rethlefsen ML, Kleijnen J et al (2017) Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev 6:245. https://doi.org/10.1186/s13643-017-0644-y

    Article  PubMed  PubMed Central  Google Scholar 

  21. Bramer WM, Giustini D, Kramer BM (2016) Comparing the coverage, recall, and precision of searches for 120 systematic reviews in Embase, MEDLINE, and Google Scholar: a prospective study. Syst Rev 1(5):39. https://doi.org/10.1186/s13643-016-0215-7.PMID:26932789;PMCID:PMC4772334

    Article  Google Scholar 

  22. Goossen K, Hess S, Lunny C, Pieper D (2020) Database combinations to retrieve systematic reviews in overviews of reviews: a methodological study. BMC Med Res Methodol 20(1):138. https://doi.org/10.1186/s12874-020-00983-3.PMID:32487023;PMCID:PMC7268249

    Article  PubMed  PubMed Central  Google Scholar 

  23. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Phys Ther 89(9):873–880. https://doi.org/10.1093/ptj/89.9.873

    Article  PubMed  Google Scholar 

  24. Sataloff RT, Bush ML, Chandra R, Chepeha D, Rotenberg B, Fisher EW, Goldenberg D, Hanna EY, Kerschner JE, Kraus DH, Krouse JH, Li D, Link M, Lustig LR, Selesnick SH, Sindwani R, Smith RJ, Tysome J, Weber PC, Welling DB (2021) Systematic and other reviews: criteria and complexities. Ann Otol Rhinol Laryngol 130(7):649–652. https://doi.org/10.1177/00034894211004324

    Article  PubMed  Google Scholar 

  25. Ho S, Patel P, Ballard D, Rosenfeld R, Chandrasekhar S (2021) Systematic review and meta-analysis of endoscopic vs microscopic stapes surgery for stapes fixation. Otolaryngol Head Neck Surg 165(5):626–635. https://doi.org/10.1177/0194599821990669

    Article  PubMed  Google Scholar 

  26. Simani L, Oron Y, Handzel O, Eta RA, Warshavsky A, Horowitz G, Muhanna N, Ungar OJ (2021a) Paper patching versus watchful waiting of traumatic tympanic membrane perforations: a meta-analysis. Laryngoscope 131(9):2091–2097. https://doi.org/10.1002/lary.29580

    Article  PubMed  Google Scholar 

  27. Simani L, Shilo S, Oron Y, Eta RA, Handzel O, Muhanna N, Warshavsky A, Horowitz G, Ungar OJ (2021b) Residual perforation risk assessment of intratympanic steroids via tympanostomy tube versus transtympanic injections. Laryngoscope 131(9):E2583–E2591. https://doi.org/10.1002/lary.29609

    Article  PubMed  Google Scholar 

  28. Barbieri D, Indelicato P, Vinciguerra A, Di Marco F, Formenti AM, Trimarchi M, Bussi M (2021) Autofluorescence and indocyanine green in thyroid surgery: a systematic review and meta-analysis. Laryngoscope 131(7):1683–1692. https://doi.org/10.1002/lary.29297

    Article  PubMed  Google Scholar 

  29. Miller SC, Nguyen SA, Ong AA, Gillespie MB (2017) Transoral robotic base of tongue reduction for obstructive sleep apnea: a systematic review and meta-analysis. Laryngoscope 127(1):258–265. https://doi.org/10.1002/lary.26060

    Article  PubMed  Google Scholar 

  30. Di Maio P, Iocca O, De Virgilio A, Giudice M, Pellini R, D'Ascanio L, Golusiński P, Ricci G, Spriano G (2020) Narrow band imaging in head and neck unknown primary carcinoma: a systematic review and meta-analysis. Laryngoscope 130(7):1692–1700. https://doi.org/10.1002/lary.28350

    Article  PubMed  Google Scholar 

  31. Alsubaie KM, Alsubaie HM, Alzahrani FR, Alessa MA, Abdulmonem SK, Merdad MA, Al-Khatib T, Marzouki HZ, Algarni MA, Alherabi AZ (2022) Prophylactic central neck dissection for clinically node-negative papillary thyroid carcinoma. Laryngoscope 132(6):1320–1328. https://doi.org/10.1002/lary.29912

    Article  PubMed  Google Scholar 

  32. Ding R, Sun Q, Wang Y (2021) Association between human papilloma virus infection and malignant sinonasal inverted papilloma. Laryngoscope 131(6):1200-1205. https://doi.org/10.1002/lary.29125.

    Article  PubMed  Google Scholar 

  33. Camacho M, Chang ET, Song SA, Abdullatif J, Zaghi S, Pirelli P, Certal V, Guilleminault C (2017) Rapid maxillary expansion for pediatric obstructive sleep apnea: a systematic review and meta-analysis. Laryngoscope 127(7):1712-1719. https://doi.org/10.1002/lary.26352

    Article  PubMed  Google Scholar 

  34. Naples JG, Spiro J, Tessema B, Kuwada C, Kuo CL, Brown SM (2016) Neck recurrence and mortality in esthesioneuroblastoma: implications for management of the N0 neck. Laryngoscope 126(6):1373-1379. https://doi.org/10.1002/lary.25803

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Otolaryngology Head and Neck Surgery and Maxillofacial Surgery, Tel Aviv Sourasky Medical Center, 6 Weizman Street, 6423906, Tel Aviv, Israel

    Omer J. Ungar, Nidal Muhanna & Hen Chaushu

  2. Sackler School of Medicine, Tel-Aviv University, Tel Aviv, Israel

    Omer J. Ungar, Nidal Muhanna & Hen Chaushu

Authors
  1. Omer J. Ungar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nidal Muhanna
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hen Chaushu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Omer J. Ungar.

Ethics declarations

Conflicts of interest

The authors have no financial conflicts of interest relevant to this article to disclose.

Ethical Approval

This study of collective published data did not require approval from the institutional review board nor that of the ethical committee according to local law because it does not use individualized patient data.

Consent for Publication

This is an original manuscript reviewed by all authors and not under consideration for publication elsewhere.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ungar, O.J., Muhanna, N. & Chaushu, H. Google Scholar as a Sole Literature Source for Meta-analyses in Otolaryngology Head and Neck Surgery. Indian J Otolaryngol Head Neck Surg 75, 864–870 (2023). https://doi.org/10.1007/s12070-023-03532-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12070-023-03532-8

Keywords

Search

Navigation