Skip to main content

Advertisement

SpringerLink
Log in

Artificial intelligence insights into osteoporosis: assessing ChatGPT’s information quality and readability

  • Original Article
  • Published:
Archives of Osteoporosis Aims and scope Submit manuscript

Abstract

Summary

Accessible, accurate information, and readability play crucial role in empowering individuals managing osteoporosis. This study showed that the responses generated by ChatGPT regarding osteoporosis had serious problems with quality and were at a level of complexity that that necessitates an educational background of approximately 17 years.

Purpose

The use of artificial intelligence (AI) applications as a source of information in the field of health is increasing. Readable and accurate information plays a critical role in empowering patients to make decisions about their disease. The aim was to examine the quality and readability of responses provided by ChatGPT, an AI chatbot, to commonly asked questions regarding osteoporosis, representing a major public health problem.

Methods

“Osteoporosis,” “female osteoporosis,” and “male osteoporosis” were identified by using Google trends for the 25 most frequently searched keywords on Google. A selected set of 38 keywords was sequentially inputted into the chat interface of the ChatGPT. The responses were evaluated with tools of the Ensuring Quality Information for Patients (EQIP), the Flesch-Kincaid Grade Level (FKGL), and the Flesch-Kincaid Reading Ease (FKRE).

Results

The EQIP score of the texts ranged from a minimum of 36.36 to a maximum of 61.76 with a mean value of 48.71 as having “serious problems with quality.” The FKRE scores spanned from 13.71 to 56.06 with a mean value of 28.71 and the FKGL varied between 8.48 and 17.63, with a mean value of 13.25. There were no statistically significant correlations between the EQIP score and the FKGL or FKRE scores.

Conclusions

Although ChatGPT is easily accessible for patients to obtain information about osteoporosis, its current quality and readability fall short of meeting comprehensive healthcare standards.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. International Osteoporosis Foundation (IOF). Epidemiology. https://www.osteoporosis.foundation/health-professionals/about-osteoporosis/epidemiology . IOF. Accessed 20 Nov 2023

  2. NIH Consensus Development Panel on Osteoporosis Prevention, Diagnosis, and Therapy (2001) Osteoporosis prevention, diagnosis, and therapy. JAMA 285(6):785-795. https://doi.org/10.1001/jama.285.6.785

  3. Genant HK, Cooper C, Poor G et al (1999) Interim report and recommendations of the World Health Organization Task-Force for Osteoporosis. Osteoporos Int 10(4):259–264. https://doi.org/10.1007/s001980050224

    Article  CAS  PubMed  Google Scholar 

  4. Bartl C, Bartl P (2019) Epidemiology of osteoporotic fractures. In: Bartl R, Bartl C (ed) The osteoporosis manual. Springer Nature, Switzerland, 231–32. doi:https://doi.org/10.1007/978-3-030-00731-7_28

  5. Hamet P, Tremblay J (2017) Artificial intelligence in medicine. Metabolism 69S:36–40. https://doi.org/10.1016/j.metabol.2017.01.011

    Article  CAS  Google Scholar 

  6. Marr B (2018) How is AI used in healthcare—5 powerful real-world examples that show the latest advances. Forbes. https://www.forbes.com/sites/bernardmarr/2018/07/27/how-is-ai-used-in-healthcare-5-powerful-real-world-examples-that-show-the-latest-advances. Accessed 15 Nov 2023

  7. Hopkins AM, Logan JM, Kichenadasse G, Sorich MJ (2023) Artificial intelligence chatbots will revolutionize how cancer patients access information: ChatGPT represents a paradigm-shift. JNCI Cancer Spectr 7(2):pkad010. https://doi.org/10.1093/jncics/pkad010

    Article  PubMed  PubMed Central  Google Scholar 

  8. Jovanović M, Baez M, Casati F (2020) Chatbots as conversational healthcare services. IEEE Internet Comput 25(3):44–51. https://doi.org/10.1109/MIC.2020.3037151

    Article  Google Scholar 

  9. OpenAI. CHATGPT: optimizing language models for dialogue. OpenAI. https://openai.com/blog/chatg pt/. Accessed 20 Nowember 2023

  10. Xue VW, Lei P, Cho WC (2023) The potential impact of ChatGPT in clinical and translational medicine. Clin Transl Med 13(3):e1216. https://doi.org/10.1002/ctm2.1216

    Article  PubMed  PubMed Central  Google Scholar 

  11. Crawford-Manning F, Greenall C, Hawarden A et al (2021) Evaluation of quality and readability of online patient information on osteoporosis and osteoporosis drug treatment and recommendations for improvement. Osteoporos Int 32:1567–1584. https://doi.org/10.1007/s00198-020-05800-7

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Yurdakul OV, Kilicoglu MS, Bagcier F (2021) Evaluating the reliability and readability of online information on osteoporosis. Arch Endocrinol Metab 65(1):85–92. https://doi.org/10.20945/2359-3997000000311

    Article  PubMed  Google Scholar 

  13. Moult B, Franck LS, Brady H (2004) Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information. Health Expect 7(2):165–175. https://doi.org/10.1111/j.1369-7625.2004.00273.x

    Article  PubMed  PubMed Central  Google Scholar 

  14. Boles CD, Liu Y, November-Rider D (2016) Readability levels of dental patient education brochures. Am Dental Hygienists’ Assoc 90(1):28–34

    Google Scholar 

  15. Morales-Torres J, Gutiérrez-Ureña S (2004) Osteoporosis Committee of Pan-American League of Associations for Rheumatology The burden of osteoporosis in Latin America. Osteoporos Int 15(8):625–632. https://doi.org/10.1007/s00198-004-1596-3

    Article  PubMed  Google Scholar 

  16. Aziziyeh R, Amin M, Habib M et al (2019) The burden of osteoporosis in four Latin American countries: Brazil, Mexico, Colombia, and Argentina. J Med Econ 22(7):638–644. https://doi.org/10.1080/13696998.2019.1590843

    Article  PubMed  Google Scholar 

  17. Fareed N, Swoboda CM, Jonnalagadda P, Walker DM, Huerta TR (2021) Differences between races in health information seeking and trust over time: evidence from a cross-sectional, pooled analyses of HINTS data. Am J Health Promot 35(1):84–92. https://doi.org/10.1177/0890117120934609

    Article  PubMed  Google Scholar 

  18. Lohana CK, Samir N (2016) Risk management of osteoporosis in postmenopausal women; a study of women in a teaching hospital. Global J Health Sci 8(11):55505. https://doi.org/10.5539/gjhs.v8n11p36

    Article  Google Scholar 

  19. Reginster JY, Sarlet N, Lecart MP (2005) Fractures in osteoporosis: the challenge for the new millennium. Osteoporos Int 16(Suppl 1):S1–S3. https://doi.org/10.1007/s00198-004-1752-9

    Article  PubMed  Google Scholar 

  20. Fontenot HB, Harris AL (2014) Pharmacologic management of osteoporosis. J Obstet Gynecol Neonatal Nurs 43(2):236-E21. https://doi.org/10.1111/1552-6909.12285

    Article  PubMed  Google Scholar 

  21. Caraccio DM, Jones MS (2021) Principles of diagnosis and treatment of osteoporosis. In: Esther RJ (ed) Clinical Foundations of Musculoskeletal Medicine. Springer Nature, Switzerland, 77–93.   https://doi.org/10.1007/978-3-030-42894-5_7

  22. Agrawal NK, Sharma B (2023) Prevalence of osteoporosis in otherwise healthy Indian males aged 50 years and above. Arch Osteoporos 8:116. https://doi.org/10.1007/s11657-012-0116-x

    Article  Google Scholar 

  23. Rao SS, Budhwar N, Ashfaque A (2010) Osteoporosis in men. Am Fam Physician 82(5):503–508

    PubMed  Google Scholar 

  24. Richardson JE, Reid MC (2013) The promises and pitfalls of leveraging mobile health technology for pain care. Pain Med 14(11):1621–1626. https://doi.org/10.1111/pme.12206

    Article  PubMed  Google Scholar 

  25. Temel MH, Batıbay S, Bağcıer F (2023) Quality and readability of online information on cerebral palsy. J Consum Health Internet 27(3):266–281. https://doi.org/10.1080/15398285.2023.2235531

    Article  Google Scholar 

  26. Khan B, Fatima H, Qureshi A, et al (2023) Drawbacks of artificial intelligence and their potential solutions in the healthcare sector [published online ahead of print, 2023 Feb 8]. Biomed Mater Devices 1–8.  https://doi.org/10.1007/s44174-023-00063-2

  27. Eysenbach G (2002) Infodemiology: the epidemiology of (mis)information. Am J Med 113(9):763–765. https://doi.org/10.1016/s0002-9343(02)01473-0

    Article  PubMed  Google Scholar 

  28. Millenson ML, Baldwin JL, Zipperer L, Singh H (2018) Beyond Dr Google: the evidence on consumer-facing digital tools for diagnosis. Diagnosis (Berl) 5(3):95–105. https://doi.org/10.1515/dx-2018-0009

    Article  PubMed  Google Scholar 

  29. Lee JWK, Gotay C, Sawatzky R, Kazanjian A (2019) Interrelatedness of distress among Chinese-speaking patients and family caregivers. Cancer Nurs 42(6):439–447. https://doi.org/10.1097/NCC.000000000000062

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Clinic of Physical Medicine and Rehabilitation, İzzet Baysal Physical Treatment and Rehabilitation Training and Research Hospital, Orüs Street, No. 59, 14020, Bolu, Turkey

    Yakup Erden

  2. Department of Physical Medicine and Rehabilitation, Üsküdar State Hospital, Barbaros, Veysi Paşa Street, No. 14, 34662, Istanbul, Turkey

    Mustafa Hüseyin Temel

  3. Clinic of Physical Medicine and Rehabilitation, Başakşehir Çam and Sakura City Hospital, Olympic Boulevard Road, 34480, Istanbul, Turkey

    Fatih Bağcıer

Authors
  1. Yakup Erden
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mustafa Hüseyin Temel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fatih Bağcıer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yakup Erden.

Ethics declarations

Conflicts of interest

None.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 30 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Erden, Y., Temel, M.H. & Bağcıer, F. Artificial intelligence insights into osteoporosis: assessing ChatGPT’s information quality and readability. Arch Osteoporos 19, 17 (2024). https://doi.org/10.1007/s11657-024-01376-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11657-024-01376-5

Keywords

Search

Navigation