Skip to main content
SpringerLink
Log in

Artificial Intelligence for Clinically Meaningful Outcome Prediction in Orthopedic Research: Current Applications and Limitations

  • REVIEW
  • Published:
Current Reviews in Musculoskeletal Medicine Aims and scope Submit manuscript

Abstract

Purpose of Review

Patient-reported outcome measures (PROM) play a critical role in evaluating the success of treatment interventions for musculoskeletal conditions. However, predicting which patients will benefit from treatment interventions is complex and influenced by a multitude of factors. Artificial intelligence (AI) may better anticipate the propensity to achieve clinically meaningful outcomes through leveraging complex predictive analytics that allow for personalized medicine. This article provides a contemporary review of current applications of AI developed to predict clinically significant outcome (CSO) achievement after musculoskeletal treatment interventions.

Recent Findings

The highest volume of literature exists in the subspecialties of total joint arthroplasty, spine, and sports medicine, with only three studies identified in the remaining orthopedic subspecialties combined. Performance is widely variable across models, with most studies only reporting discrimination as a performance metric. Given the complexity inherent in predictive modeling for this task, including data availability, data handling, model architecture, and outcome selection, studies vary widely in their methodology and results. Importantly, the majority of studies have not been externally validated or demonstrate important methodological limitations, precluding their implementation into clinical settings.

Summary

A substantial body of literature has accumulated demonstrating variable internal validity, limited scope, and low potential for clinical deployment. The majority of studies attempt to predict the MCID—the lowest bar of clinical achievement. Though a small proportion of models demonstrate promise and highlight the utility of AI, important methodological limitations need to be addressed moving forward to leverage AI-based applications for clinical deployment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance

  1. Pasqualini I, Piuzzi NS. New CMS policy on the mandatory collection of patient-reported outcome measures for total hip and knee arthroplasty by 2027: what orthopaedic surgeons should know. J Bone Joint Surg Am. 2024.

  2. Makhni EC. Meaningful clinical applications of patient-reported outcome measures in orthopaedics. J Bone Joint Surg Am. 2021;103(1):84–91.

    Article  PubMed  Google Scholar 

  3. Porter ME. What is value in health care? N Engl J Med. 2010;363(26):2477–81.

    Article  CAS  PubMed  Google Scholar 

  4. Makhni EC, Baumhauer JF, Ayers D, Bozic KJ. Patient-reported outcome measures: how and why they are collected. Instr Course Lect. 2019;68:675–80.

    PubMed  Google Scholar 

  5. Chung AS, Copay AG, Olmscheid N, Campbell D, Walker JB, Chutkan N. Minimum Clinically important difference: current trends in the spine literature. Spine (Phila Pa 1976). 2017;42(14):1096–105.

  6. Copay AG, Chung AS, Eyberg B, Olmscheid N, Chutkan N, Spangehl MJ. Minimum clinically important difference: current trends in the orthopaedic literature, Part I: Upper Extremity: A Systematic Review. JBJS Rev. 2018;6(9): e1.

    Article  PubMed  Google Scholar 

  7. Copay AG, Eyberg B, Chung AS, Zurcher KS, Chutkan N, Spangehl MJ. Minimum clinically important difference: current trends in the orthopaedic literature, Part II: lower extremity: a systematic review. JBJS Rev. 2018;6(9): e2.

    Article  PubMed  Google Scholar 

  8. Baumhauer JF, Bozic KJ. Value-based healthcare: patient-reported outcomes in clinical decision making. Clin Orthop Relat Res. 2016;474(6):1375–8.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Kunze KN, Bart JA, Ahmad M, Nho SJ, Chahla J. Large heterogeneity among minimal clinically important differences for hip arthroscopy outcomes: a systematic review of reporting trends and quantification methods. Arthroscopy. 2021;37(3):1028-37 e6.

    Article  PubMed  Google Scholar 

  10. Menendez ME, Sudah SY, Cohn MR, Narbona P, Ladermann A, Barth J, et al. Defining minimal clinically important difference and patient acceptable symptom state after the latarjet procedure. Am J Sports Med. 2022;50(10):2761–6.

    Article  PubMed  Google Scholar 

  11. Bernstein DN, Karhade AV, Bono CM, Schwab JH, Harris MB, Tobert DG. Sociodemographic factors are associated with patient-reported outcome measure completion in orthopaedic surgery: an analysis of completion rates and determinants among new patients. JB JS Open Access. 2022;7(3):e22.00026.

  12. Jolback P, Rolfson O, Mohaddes M, Nemes S, Karrholm J, Garellick G, et al. Does surgeon experience affect patient-reported outcomes 1 year after primary total hip arthroplasty? Acta Orthop. 2018;89(3):265–71.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Langlotz CP, Allen B, Erickson BJ, Kalpathy-Cramer J, Bigelow K, Cook TS, et al. A roadmap for foundational research on artificial intelligence in medical imaging: from the 2018 NIH/RSNA/ACR/The Academy Workshop. Radiology. 2019;291(3):781–91.

    Article  PubMed  Google Scholar 

  14. Padash S, Mickley JP, Vera Garcia DV, Nugen F, Khosravi B, Erickson BJ, et al. An overview of machine learning in orthopedic surgery: an educational paper. J Arthroplasty. 2023;38(10):1938–42.

    Article  PubMed  Google Scholar 

  15. Langenberger B, Schrednitzki D, Halder AM, Busse R, Pross CM. Predicting whether patients will achieve minimal clinically important differences following hip or knee arthroplasty. Bone Joint Res. 2023;12(9):512–21.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Fontana MA, Lyman S, Sarker GK, Padgett DE, MacLean CH. Can machine learning algorithms predict which patients will achieve minimally clinically important differences from total joint arthroplasty? Clin Orthop Relat Res. 2019;477(6):1267–79.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Nwachukwu BU, Beck EC, Lee EK, Cancienne JM, Waterman BR, Paul K, et al. Application of machine learning for predicting clinically meaningful outcome after arthroscopic femoroacetabular impingement surgery. Am J Sports Med. 2020;48(2):415–23.

    Article  PubMed  Google Scholar 

  18. Kunze KN, Krivicich LM, Clapp IM, Bodendorfer BM, Nwachukwu BU, Chahla J, et al. Machine learning algorithms predict achievement of clinically significant outcomes after orthopaedic surgery: a systematic review. Arthroscopy. 2022;38(6):2090–105.

    Article  PubMed  Google Scholar 

  19. El-Othmani MM, Zalikha AK, Shah RP. Comparative analysis of the ability of machine learning models in predicting in-hospital postoperative outcomes after total hip arthroplasty. J Am Acad Orthop Surg. 2022;30(20):e1337–47.

    Article  PubMed  Google Scholar 

  20. Rouzrokh P, Mickley JP, Khosravi B, Faghani S, Moassefi M, Schulz WR, et al. THA-AID: deep learning tool for total hip arthroplasty automatic implant detection with uncertainty and outlier quantification. J Arthroplasty. 2023;9(4):966–73.

  21. Khosravi B, Rouzrokh P, Mickley JP, Faghani S, Larson AN, Garner HW, et al. Creating high fidelity synthetic pelvis radiographs using generative adversarial networks: unlocking the potential of deep learning models without patient privacy concerns. J Arthroplasty. 2023;38(10):2037-43 e1.

    Article  PubMed  Google Scholar 

  22. Jang SJ, Fontana MA, Kunze KN, Anderson CG, Sculco TP, Mayman DJ, et al. An interpretable machine learning model for predicting 10-year total hip arthroplasty risk. J Arthroplasty. 2023;38(7S):S44–50.

  23. Huber M, Kurz C, Leidl R. Predicting patient-reported outcomes following hip and knee replacement surgery using supervised machine learning. BMC Med Inform Decis Mak. 2019;19(1):3.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Harris AHS, Kuo AC, Bowe TR, Manfredi L, Lalani NF, Giori NJ. Can machine learning methods produce accurate and easy-to-use preoperative prediction models of one-year improvements in pain and functioning after knee arthroplasty? J Arthroplasty. 2021;36(1):112-7 e6.

    Article  PubMed  Google Scholar 

  25. Katakam A, Karhade AV, Collins A, Shin D, Bragdon C, Chen AF, et al. Development of machine learning algorithms to predict achievement of minimal clinically important difference for the KOOS-PS following total knee arthroplasty. J Orthop Res. 2022;40(4):808–15.

    Article  PubMed  Google Scholar 

  26. Zhang S, Lau BPH, Ng YH, Wang X, Chua W. Machine learning algorithms do not outperform preoperative thresholds in predicting clinically meaningful improvements after total knee arthroplasty. Knee Surg Sports Traumatol Arthrosc. 2022;30(8):2624–30.

    Article  PubMed  Google Scholar 

  27. Kunze KN, Karhade AV, Sadauskas AJ, Schwab JH, Levine BR. Development of machine learning algorithms to predict clinically meaningful improvement for the patient-reported health state after total hip arthroplasty. J Arthroplasty. 2020;35(8):2119–23.

    Article  PubMed  Google Scholar 

  28. Bourne RB, Chesworth BM, Davis AM, Mahomed NN, Charron KD. Patient satisfaction after total knee arthroplasty: who is satisfied and who is not? Clin Orthop Relat Res. 2010;468(1):57–63.

    Article  PubMed  Google Scholar 

  29. Farooq H, Deckard ER, Ziemba-Davis M, Madsen A, Meneghini RM. Predictors of patient satisfaction following primary total knee arthroplasty: results from a traditional statistical model and a machine learning algorithm. J Arthroplasty. 2020;35(11):3123–30.

    Article  PubMed  Google Scholar 

  30. Pettit MH, Hickman SHM, Malviya A, Khanduja V. Development of machine-learning algorithms to predict attainment of minimal clinically important difference after hip arthroscopy for femoroacetabular impingement yield fair performance and limited clinical utility. Arthroscopy. 2023;40(4):1153–63.

  31. Ramkumar PN, Karnuta JM, Haeberle HS, Sullivan SW, Nawabi DH, Ranawat AS, et al. Radiographic indices are not predictive of clinical outcomes among 1735 patients indicated for hip arthroscopic surgery: a machine learning analysis. Am J Sports Med. 2020;48(12):2910–8.

    Article  PubMed  Google Scholar 

  32. Kunze KN, Polce EM, Rasio J, Nho SJ. Machine learning algorithms predict clinically significant improvements in satisfaction after hip arthroscopy. Arthroscopy. 2021;37(4):1143–51.

    Article  PubMed  Google Scholar 

  33. Kunze KN, Polce EM, Nwachukwu BU, Chahla J, Nho SJ. Development and internal validation of supervised machine learning algorithms for predicting clinically significant functional improvement in a mixed population of primary hip arthroscopy. Arthroscopy. 2021;37(5):1488–97.

    Article  PubMed  Google Scholar 

  34. •Kunze KN, Polce EM, Clapp IM, Alter T, Nho SJ. Association between preoperative patient factors and clinically meaningful outcomes after hip arthroscopy for femoroacetabular impingement syndrome: a machine learning analysis. Am J Sports Med. 2022;50(3):746–56. This study is of importance as it is one of the few in sports medicine which attempts to predict outcomes beyond the MCID and also investigated prediction of the PASS and SCB.

    Article  PubMed  Google Scholar 

  35. Kunze KN, Polce EM, Clapp I, Nwachukwu BU, Chahla J, Nho SJ. Machine learning algorithms predict functional improvement after hip arthroscopy for femoroacetabular impingement syndrome in athletes. J Bone Joint Surg Am. 2021;103(12):1055–62.

    Article  PubMed  Google Scholar 

  36. Kunze KN, Polce EM, Ranawat AS, Randsborg PH, Williams RJ 3rd, Allen AA, et al. Application of machine learning algorithms to predict clinically meaningful improvement after arthroscopic anterior cruciate ligament reconstruction. Orthop J Sports Med. 2021;9(10):23259671211046576.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Ye Z, Zhang T, Wu C, Qiao Y, Su W, Chen J, et al. Predicting the objective and subjective clinical outcomes of anterior cruciate ligament reconstruction: a machine learning analysis of 432 patients. Am J Sports Med. 2022;50(14):3786–95.

    Article  PubMed  Google Scholar 

  38. Martin RK, Wastvedt S, Pareek A, Persson A, Visnes H, Fenstad AM, et al. Predicting subjective failure of ACL reconstruction: a machine learning analysis of the Norwegian Knee Ligament Register and patient reported outcomes. J ISAKOS. 2022;7(3):1–9.

    Article  PubMed  Google Scholar 

  39. Kumar V, Roche C, Overman S, Simovitch R, Flurin PH, Wright T, et al. Using machine learning to predict clinical outcomes after shoulder arthroplasty with a minimal feature set. J Shoulder Elbow Surg. 2021;30(5):e225–36.

    Article  PubMed  Google Scholar 

  40. Kumar V, Roche C, Overman S, Simovitch R, Flurin PH, Wright T, et al. What is the accuracy of three different machine learning techniques to predict clinical outcomes after shoulder arthroplasty? Clin Orthop Relat Res. 2020;478(10):2351–63.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Ramkumar PN, Karnuta JM, Haeberle HS, Owusu-Akyaw KA, Warner TS, Rodeo SA, et al. Association between preoperative mental health and clinically meaningful outcomes after osteochondral allograft for cartilage defects of the knee: a machine learning analysis. Am J Sports Med. 2021;49(4):948–57.

    Article  PubMed  Google Scholar 

  42. Ramkumar PN, Karnuta JM, Haeberle HS, Rodeo SA, Nwachukwu BU, Williams RJ. Effect of preoperative imaging and patient factors on clinically meaningful outcomes and quality of life after osteochondral allograft transplantation: a machine learning analysis of cartilage defects of the knee. Am J Sports Med. 2021;49(8):2177–86.

    Article  PubMed  Google Scholar 

  43. Alaiti RK, Vallio CS, Assunção JH, Andrade e Silva FBd, Gracitelli MEC, Neto AAF, et al. Using machine learning to predict nonachievement of clinically significant outcomes after rotator cuff repair. Orthop J Sports Med. 2023;11(10):23259671231206180.

  44. Potty AG, Potty ASR, Maffulli N, Blumenschein LA, Ganta D, Mistovich RJ, et al. Approaching artificial intelligence in orthopaedics: predictive analytics and machine learning to prognosticate arthroscopic rotator cuff surgical outcomes. J Clin Med. 2023;12(6).

  45. ••Kunze KN, Kaidi A, Madjarova S, Polce EM, Ranawat AS, Nawabi DH, et al. External validation of a machine learning algorithm for predicting clinically meaningful functional improvement after arthroscopic hip preservation surgery. Am J Sports Med. 2022;50(13):3593–9. This study is of great importance as it is the only external validation study in sports medicine to date with a primary outcome of clinically meaningful outcome achievement.

    Article  PubMed  Google Scholar 

  46. Merali ZG, Witiw CD, Badhiwala JH, Wilson JR, Fehlings MG. Using a machine learning approach to predict outcome after surgery for degenerative cervical myelopathy. PLoS ONE. 2019;14(4): e0215133.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Khan O, Badhiwala JH, Witiw CD, Wilson JR, Fehlings MG. Machine learning algorithms for prediction of health-related quality-of-life after surgery for mild degenerative cervical myelopathy. Spine J. 2021;21(10):1659–69.

    Article  PubMed  Google Scholar 

  48. Zhang JK, Jayasekera D, Javeed S, Greenberg JK, Blum J, Dibble CF, et al. Diffusion basis spectrum imaging predicts long-term clinical outcomes following surgery in cervical spondylotic myelopathy. Spine J. 2023;23(4):504–12.

    Article  PubMed  Google Scholar 

  49. Park C, Mummaneni PV, Gottfried ON, Shaffrey CI, Tang AJ, Bisson EF, et al. Which supervised machine learning algorithm can best predict achievement of minimum clinically important difference in neck pain after surgery in patients with cervical myelopathy? A QOD study. Neurosurg Focus. 2023;54(6):E5.

    Article  PubMed  Google Scholar 

  50. Staartjes VE, de Wispelaere MP, Vandertop WP, Schroder ML. Deep learning-based preoperative predictive analytics for patient-reported outcomes following lumbar discectomy: feasibility of center-specific modeling. Spine J. 2019;19(5):853–61.

    Article  PubMed  Google Scholar 

  51. Pedersen CF, Andersen MO, Carreon LY, Eiskjaer S. Applied machine learning for spine surgeons: predicting outcome for patients undergoing treatment for lumbar disc herniation using PRO data. Global Spine J. 2022;12(5):866–76.

    Article  PubMed  Google Scholar 

  52. Berjano P, Langella F, Ventriglia L, Compagnone D, Barletta P, Huber D, et al. The influence of baseline clinical status and surgical strategy on early good to excellent result in spinal lumbar arthrodesis: a machine learning approach. J Pers Med. 2021;11(12).

  53. Staartjes VE, Stumpo V, Ricciardi L, Maldaner N, Eversdijk HAJ, Vieli M, et al. FUSE-ML: development and external validation of a clinical prediction model for mid-term outcomes after lumbar spinal fusion for degenerative disease. Eur Spine J. 2022;31(10):2629–38.

    Article  PubMed  Google Scholar 

  54. Karhade AV, Fogel HA, Cha TD, Hershman SH, Doorly TP, Kang JD, et al. Development of prediction models for clinically meaningful improvement in PROMIS scores after lumbar decompression. Spine J. 2021;21(3):397–404.

    Article  PubMed  Google Scholar 

  55. •Halicka M, Wilby M, Duarte R, Brown C. Predicting patient-reported outcomes following lumbar spine surgery: development and external validation of multivariable prediction models. BMC Musculoskelet Disord. 2023;24(1):333. This study is of importance given its attempt to externally validate their study findings in a unique population of patients.

    Article  PubMed  PubMed Central  Google Scholar 

  56. Siccoli A, de Wispelaere MP, Schroder ML, Staartjes VE. Machine learning-based preoperative predictive analytics for lumbar spinal stenosis. Neurosurg Focus. 2019;46(5):E5.

    Article  PubMed  Google Scholar 

  57. Brinkman N, Shah R, Doornberg J, Ring D, Gwilym S, Jayakumar P. Artificial neural networks outperform linear regression in estimating 9-month patient-reported outcomes after upper extremity fractures with increasing number of variables. OTA Int. 2023;6(5 Suppl): e284.

    PubMed  Google Scholar 

  58. Loos NL, Hoogendam L, Souer JS, Slijper HP, Andrinopoulou ER, Coppieters MW, et al. Machine learning can be used to predict function but not pain after surgery for thumb carpometacarpal osteoarthritis. Clin Orthop Relat Res. 2022;480(7):1271–84.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Harrison CJ, Geoghegan L, Sidey-Gibbons CJ, Stirling PHC, McEachan JE, Rodrigues JN. Developing machine learning algorithms to support patient-centered, value-based carpal tunnel decompression surgery. Plast Reconstr Surg Glob Open. 2022;10(4): e4279.

    Article  PubMed  PubMed Central  Google Scholar 

  60. •Oeding JF, Krych AJ, Pearle AD, Kelly BT, Kunze KN. Medical imaging applications developed using artificial intelligence demonstrate high internal validity yet are limited in scope and lack external validation. Arthroscopy. 2024. This study is of importance to this topic as it parellels the themes identified in this review pertaining to clinically significant outcome achievement - repetitive use cases of current statistical methods and methodological shortcomings are currently impeding progress in this domain.

  61. Rossi MJ, Brand JC, Lubowitz JH. Minimally clinically important difference (MCID) is a low bar. Arthroscopy. 2023;39(2):139–41.

    Article  PubMed  Google Scholar 

  62. Kunze KN, Madjarova S, Jayakumar P, Nwachukwu BU. Challenges and opportunities for the use of patient-reported outcome measures in orthopaedic pediatric and sports medicine surgery. J Am Acad Orthop Surg. 2023;31(20):e898–905.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Orthopedic Surgery, Hospital for Special Surgery, 535 East 70Th Street, New York, NY, 10021, USA

    Seong Jun Jang & Kyle N. Kunze

  2. Georgetown University School of Medicine, Washington, DC, USA

    Jake Rosenstadt

  3. Weill Cornell College of Medicine, New York, NY, USA

    Eugenia Lee

Authors
  1. Seong Jun Jang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jake Rosenstadt
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eugenia Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kyle N. Kunze
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

KNK: Supervision, methodology, writing of initial manuscript, revision of initial manuscript SJ: Writing of initial manuscript, revision of initial manuscript JR: Writing of initial manuscript, revision of initial manuscript EL: Writing of initial manuscript, revision of initial manuscript

Corresponding author

Correspondence to Kyle N. Kunze.

Ethics declarations

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jang, S.J., Rosenstadt, J., Lee, E. et al. Artificial Intelligence for Clinically Meaningful Outcome Prediction in Orthopedic Research: Current Applications and Limitations. Curr Rev Musculoskelet Med (2024). https://doi.org/10.1007/s12178-024-09893-z

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12178-024-09893-z

Keywords

Search

Navigation