Skip to main content
SpringerLink
Log in

ChatGPT-4 accuracy for patient education in laryngopharyngeal reflux

  • Laryngology
  • Published:
European Archives of Oto-Rhino-Laryngology Aims and scope Submit manuscript

Abstract

Introduction

Chatbot Generative Pre-trained Transformer (ChatGPT) is an artificial intelligence-powered language model chatbot able to help otolaryngologists in practice and research. The ability of ChatGPT in generating patient-centered information related to laryngopharyngeal reflux disease (LPRD) was evaluated.

Methods

Twenty-five questions dedicated to definition, clinical presentation, diagnosis, and treatment of LPRD were developed from the Dubai definition and management of LPRD consensus and recent reviews. Questions about the four aforementioned categories were entered into ChatGPT-4. Four board-certified laryngologists evaluated the accuracy of ChatGPT-4 with a 5-point Likert scale. Interrater reliability was evaluated.

Results

The mean scores (SD) of ChatGPT-4 answers for definition, clinical presentation, additional examination, and treatments were 4.13 (0.52), 4.50 (0.72), 3.75 (0.61), and 4.18 (0.47), respectively. Experts reported high interrater reliability for sub-scores (ICC = 0.973). The lowest performances of ChatGPT-4 were on answers about the most prevalent LPR signs, the most reliable objective tool for the diagnosis (hypopharyngeal-esophageal multichannel intraluminal impedance-pH monitoring (HEMII-pH)), and the criteria for the diagnosis of LPR using HEMII-pH.

Conclusion

ChatGPT-4 may provide adequate information on the definition of LPR, differences compared to GERD (gastroesophageal reflux disease), and clinical presentation. Information provided upon extra-laryngeal manifestations and HEMII-pH may need further optimization. Regarding the recent trends identifying increasing patient use of internet sources for self-education, the findings of the present study may help draw attention to ChatGPT-4’s accuracy on the topic of LPR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Briganti G (2023) How ChatGPT works: a mini review. Eur Arch Otorhinolaryngol. https://doi.org/10.1007/s00405-023-08337-7

    Article  PubMed  Google Scholar 

  2. Vaira LA, Lechien JR, Abbate V, Allevi F, Audino G, Beltramini GA et al (2023) Accuracy of ChatGPT-generated information on head and neck and oromaxillofacial surgery: a multicenter collaborative analysis. Otolaryngol Head Neck Surg. https://doi.org/10.1002/ohn.489

    Article  PubMed  Google Scholar 

  3. Davis RJ, Ayo-Ajibola O, Lin ME, Swanson MS, Chambers TN, Kwon DI, Kokot NC (2023) Evaluation of oropharyngeal cancer information from revolutionary artificial intelligence Chatbot. Laryngoscope. https://doi.org/10.1002/lary.31191

    Article  PubMed  PubMed Central  Google Scholar 

  4. Lechien JR, Vaezi MF, Chan WW, Allen JE, Karkos PD, Saussez S et al (2023) The Dubai definition and diagnostic criteria of laryngopharyngeal reflux: the IFOS consensus. Laryngoscope. https://doi.org/10.1002/lary.31134

    Article  PubMed  Google Scholar 

  5. Lechien JR, Akst LM, Hamdan AL, Schindler A, Karkos PD, Barillari MR, Calvo-Henriquez C, Crevier-Buchman L, Finck C, Eun YG, Saussez S, Vaezi MF (2019) Evaluation and management of laryngopharyngeal reflux disease: state of the art review. Otolaryngol Head Neck Surg 160(5):762–782. https://doi.org/10.1177/0194599819827488

    Article  PubMed  Google Scholar 

  6. Lechien JR, Saussez S, Schindler A, Karkos PD, Hamdan AL, Harmegnies B, De Marrez LG, Finck C, Journe F, Paesmans M, Vaezi MF (2019) Clinical outcomes of laryngopharyngeal reflux treatment: a systematic review and meta-analysis. Laryngoscope 129(5):1174–1187. https://doi.org/10.1002/lary.27591

    Article  PubMed  Google Scholar 

  7. Kamal AN, Dhar SI, Bock JM, Clarke JO, Lechien JR, Allen J, Belafsky PC, Blumin JH, Chan WW, Fass R, Fisichella PM, Marohn M, O’Rourke AK, Postma G, Savarino EV, Vaezi MF, Carroll TL, Akst LM (2023) Best practices in treatment of laryngopharyngeal reflux disease: a multidisciplinary modified Delphi study. Dig Dis Sci 68(4):1125–1138. https://doi.org/10.1007/s10620-022-07672-9

    Article  PubMed  Google Scholar 

  8. Lechien JR, Muls V, Dapri G, Mouawad F, Eisendrath P, Schindler A, Nacci A, Barillari MR, Finck C, Saussez S, Akst LM, Sataloff RT (2019) The management of suspected or confirmed laryngopharyngeal reflux patients with recalcitrant symptoms: a contemporary review. Clin Otolaryngol 44(5):784–800. https://doi.org/10.1111/coa.13395

    Article  PubMed  Google Scholar 

  9. Kamani T, Penney S, Mitra I, Pothula V (2012) The prevalence of laryngopharyngeal reflux in the English population. Eur Arch Otorhinolaryngol 269(10):2219–2225. https://doi.org/10.1007/s00405-012-2028-1

    Article  PubMed  Google Scholar 

  10. Spantideas N, Drosou E, Karatsis A, Assimakopoulos D (2015) Voice disorders in the general Greek population and in patients with laryngopharyngeal reflux. Prevalence and risk factors. J Voice 29(3):389.e27–32. https://doi.org/10.1016/j.jvoice.2014.08.006

    Article  PubMed  Google Scholar 

  11. Kang JW, Lee MK, Lee YC, Ko SG, Eun YG (2023) Somatic anxiety in patients with laryngopharyngeal reflux. Laryngoscope Investig Otolaryngol 8(5):1288–1293. https://doi.org/10.1002/lio2.1138

    Article  PubMed  PubMed Central  Google Scholar 

  12. Wong MW, Hsiao SH, Wang JH, Yi CH, Liu TT, Lei WY, Hung JS, Liang SW, Lin L, Gyawali CP, Chen PR, Chen CL (2023) Esophageal hypervigilance and visceral anxiety contribute to symptom severity of laryngopharyngeal reflux. Am J Gastroenterol 118(5):786–793. https://doi.org/10.14309/ajg.0000000000002151

    Article  CAS  PubMed  Google Scholar 

  13. Salgado S, Borges LF, Cai JX, Lo WK, Carroll TL, Chan WW (2022) Symptoms classically attributed to laryngopharyngeal reflux correlate poorly with pharyngeal reflux events on multichannel intraluminal impedance testing. Dis Esophagus 36(1):doac041. https://doi.org/10.1093/dote/doac041

    Article  PubMed  Google Scholar 

  14. Capelleras M, Soto-Galindo GA, Cruellas M, Apaydin F (2023) ChatGPT and rhinoplasty recovery: an exploration of AI’s role in post-operative guidance. Facial Plast Surg. https://doi.org/10.1055/a-2219-4901

    Article  PubMed  Google Scholar 

  15. Cheong RCT, Unadkat S, Mcneillis V, Williamson A, Joseph J, Randhawa P, Andrews P, Paleri V (2023) Artificial intelligence chatbots as sources of patient education material for obstructive sleep apnoea: ChatGPT versus Google Bard. Eur Arch Otorhinolaryngol. https://doi.org/10.1007/s00405-023-08319-9

    Article  PubMed  Google Scholar 

  16. Campbell DJ, Estephan LE, Sina E, Mastrolonardo EV, Alapati R, Amin DR, Cottrill E (2023) Evaluating ChatGPT responses on thyroid nodules for patient education. Thyroid. https://doi.org/10.1089/thy.2023.0491

    Article  PubMed  PubMed Central  Google Scholar 

  17. Chiesa-Estomba CM, Lechien JR, Vaira LA, Brunet A, Cammaroto G, Mayo-Yanez M, Sanchez-Barrueco A, Saga-Gutierrez C (2023) Exploring the potential of Chat-GPT as a supportive tool for sialendoscopy clinical decision making and patient information support. Eur Arch Otorhinolaryngol. https://doi.org/10.1007/s00405-023-08104-8

    Article  PubMed  Google Scholar 

  18. Lechien JR, Briganti G, Vaira LA (2024) Accuracy of ChatGPT-3.5 and -4 in providing scientific references in otolaryngology-head and neck surgery. Eur Arch Otorhinolaryngol

Download references

Funding

None.

Author information

Authors and Affiliations

  1. Research Committee, Young Otolaryngologists of the International Federation of Otorhinolaryngological Societies (IFOS), Paris, France

    Jerome R. Lechien & Matthew R. Naunheim

  2. Division of Laryngology and Broncho-Esophagology, Department of Otolaryngology-Head Neck Surgery, EpiCURA Hospital, UMONS Research Institute for Health Sciences and Technology, University of Mons (UMons), Mons, Belgium

    Jerome R. Lechien

  3. Department of Otorhinolaryngology and Head and Neck Surgery, Foch Hospital, School of Medicine, Phonetics and Phonology Laboratory (UMR 7018 CNRS, Université Sorbonne Nouvelle/Paris 3), Paris, France

    Jerome R. Lechien

  4. Polyclinique Elsan de Poitiers, Poitiers, France

    Jerome R. Lechien

  5. Division of Otolaryngology-Head and Neck Surgery, Brigham and Women’s Hospital, Department of Otolaryngology-Head and Neck Surgery, Harvard Medical School, Boston, MA, USA

    Thomas L. Carroll

  6. Department of Otolaryngology, Washington University School of Medicine in St. Louis, St. Louis, MO, USA

    Molly N. Huston

  7. Department of Otolaryngology-Head and Neck Surgery, Harvard Medical School, Boston, MA, USA

    Matthew R. Naunheim

  8. Division of Laryngology, Massachusetts Eye and Ear, Boston, MA, USA

    Matthew R. Naunheim

Authors
  1. Jerome R. Lechien
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas L. Carroll
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Molly N. Huston
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Matthew R. Naunheim
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Jerome R. Lechien: design, acquisition of data, drafting, final approval, and accountability for the work; final approval of the version to be published; agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Matt Naunheim: data analysis and interpretation, and proofread of the paper, final approval of the version to be published; agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Thomas Carroll: design, acquisition of data, drafting, final approval, and accountability for the work; final approval of the version to be published; agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Molly Huston: design, acquisition of data, drafting, final approval, and accountability for the work; final approval of the version to be published; agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Corresponding author

Correspondence to Jerome R. Lechien.

Ethics declarations

Conflict of interest

The authors have no conflict of interest.

Informed consent

Not applicable.

Ethic committee

The institutional review board of CHU Saint-Pierre was not required for this study (ref.CHUST23).

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 60 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lechien, J.R., Carroll, T.L., Huston, M.N. et al. ChatGPT-4 accuracy for patient education in laryngopharyngeal reflux. Eur Arch Otorhinolaryngol 281, 2547–2552 (2024). https://doi.org/10.1007/s00405-024-08560-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00405-024-08560-w

Keywords

Search

Navigation