Competency based clinical shoulder examination training improves physical exam, confidence, and knowledge in common shoulder conditions
Deficiencies in musculoskeletal knowledge are reported at every stage of learning. Medical programs are looking for effective ways to incorporate competency-based training into musculoskeletal education.
To evaluate the impact of bedside feedback on learner’s shoulder examination skills, confidence, and knowledge of common shoulder conditions.
Four-week musculoskeletal clinic rotation.
UCSD third year medical students and internal medicine residents.
Learners completed three baseline evaluations: videotaped shoulder examination, attitude survey, and knowledge test. During the 4-week intervention learners received bedside observation and feedback from musculoskeletal experts while evaluating patients with shoulder conditions. Post-intervention learners repeated the three assessments.
Eighty-nine learners participated. In the primary outcome measure evaluating the pre/post videotaped shoulder examination, significant improvement was seen in 21 of 23 shoulder examination maneuvers. Secondary outcomes include changes in learner confidence and knowledge. Greatest gains in learner confidence were seen in performing the shoulder examination (61.5% improvement) and performing injections (97.1% improvement). Knowledge improved significantly in all categories including anatomy/examination interpretation, diagnosis, and procedures.
Direct observation and feedback during clinical evaluation of patients with shoulder pain improves shoulder examination competency, provider confidence, and knowledge of common shoulder conditions.
KeywordsMusculoskeletal Shoulder examination Direct observation Competency-based medical education Primary care
Musculoskeletal complaints are a leading cause of chronic pain and disability, accounting for approximately 20% of primary care visits1 , 2 and 60% of American veteran health care utilization.3 – 5 In 2011, the estimated cost from musculoskeletal complaints was $874 billion in the US.6 The burden of musculoskeletal disease is projected to continue to rise sharply with the increased life expectancy of the general population.1 , 7
Primary care providers assess up to 70% of initial musculoskeletal complaints.8 , 9 However, many primary care providers feel inadequately trained to diagnose and treat musculoskeletal disorders.10 , 11 Objectively, insufficient musculoskeletal knowledge has been reported at every stage of learning, from medical students to residents to primary care providers.10 – 17
Mastery of the shoulder examination is recommended in the Association of American Medical Colleges musculoskeletal content report.18 With a prevalence of 16–21% in the general population, shoulder pain is the third most common site of musculoskeletal pain, after back and knee pain.19 – 21 However, recent analysis of the 2011 Step 2 medical board examinations documents deficiencies in musculoskeletal physical examination skills, including the shoulder examination.22 Even within an orthopedic specialty, a recent evaluation of residents reports deficiency in shoulder examination skills.23 In the study survey, 100% of the orthopedic residents reported that physical examination skills were observed only occasionally/rarely, and only 5% felt enough clinic time was devoted to teaching the physical examination.24 These studies demonstrate not only a need to improve teaching the shoulder examination, but also a glaring lack of point-of-care teaching in the clinical setting.
Competency-based medical education has recently emerged as a popular theoretical strategy among medical training programs. However, agreement on what constitutes clinical competency and how to measure it varies widely in the literature. Some of the fundamental characteristics of competency-based medical education include defining the desired competency from patient needs and then incorporating direct observation and feedback into learning experiences, with systematic evaluation before and after training.32 , 33 With the understanding that the desired clinical competency is accurate management of shoulder pain, we propose that the first step toward clinical competence is successfully performing a consistent shoulder physical examination, which can increase provider confidence and improve diagnostic knowledge. We describe an observational study on whether bedside feedback on shoulder cases in a clinical setting improves learners’ physical examination skills, confidence in diagnostic ability, and medical knowledge of shoulder pain.
Setting and participants
This study was approved by the San Diego Veterans Administration (VA) and University of California, San Diego (UCSD) Institutional Review Boards in accordance with the requirements of the Code of Federal Regulations on the Protection of Human Subjects.
In July 2003, the San Diego VA primary care musculoskeletal clinic was created to provide non-surgical treatment options, including joint injections, for common musculoskeletal complaints. The clinic is staffed by two primary care musculoskeletal experts with national experience in musculoskeletal teaching. UCSD medical students and internal medicine residents rotate through the musculoskeletal clinic once weekly for a month. In 2007 and 2008, each UCSD learner rotating through the musculoskeletal clinic was invited to participate in this study. After discussion of the risks and benefits of participation, learners signed an informed consent.
In response to Project 100, an educational initiative challenging 100% of medical schools to implement musculoskeletal curricula, 83% of schools had complied by 2010.34 , 35 But the vast majority of musculoskeletal curricula remained preclinical. Despite the fact that medical educators describe coaching and direct observation as "fundamentally essential to training qualified physicians,”33 , 36 only 24% of programs required a clinical musculoskeletal clerkship.35 Since primary care musculoskeletal clinics have previously been reported as effective sites for evaluating common musculoskeletal complaints,37 – 39 we decided to use our musculoskeletal clinic to evaluate a teaching intervention for common shoulder conditions.
The three study outcomes for our learners were improvement in performance of the shoulder examination, provider confidence, and knowledge of managing common shoulder conditions. Before starting the musculoskeletal rotation, learners completed three baseline evaluations: a videotaped shoulder examination on a standardized patient (Online Appendix A), an attitude survey of confidence regarding shoulder examination and management (Online Appendix B), and a knowledge test on management of common shoulder conditions (Online Appendix C). The learners did not receive the results of their pre-test evaluations or keep the test questions for reference. The shoulder physical examination checklist and knowledge test were developed in consultation with seven local sports medicine, medical education, and primary care experts to make it clinically relevant to primary care shoulder conditions. These providers span a spectrum of clinical settings: academic, active-duty military, and community/private. Every knowledge question and physical examination component was rated by each provider for clinical relevance.
After completing the baseline study evaluations, learners experienced the teaching innovation: repeated directly observed evaluations of patients with shoulder pain, with point-of-care feedback. On average, each learner was assigned five to six shoulder patients over the musculoskeletal rotation. Internal medicine attendings trained in musculoskeletal medicine observed the clinic visit and gave trainees immediate feedback on the physical examination, standardized according to a written checklist (Online Appendix A). Diagnosis, assessment, and treatment, including subacromial bursa injections, were also performed bedside under attending physician guidance.
At the end of the 4-week musculoskeletal rotation, each learner repeated the three assessments: a post-intervention videotaped shoulder examination, attitude survey, and knowledge test. The videotaped shoulder examinations were graded by two independent raters blinded to the pre- or post-intervention status of the learner.
Data analysis followed the standard Cross Industry Standard Process to assure data integrity and to declare each data element type, distribution family type, and appropriate analytic methods for the study question(s). Descriptive statistics including frequencies, percentages, means and standard deviations were calculated for each item. Comparison of pre-post change in pre and post items was conducted using Student’s t-test for continuous variables (knowledge test), McNemar test for categorical data (video), and Mann-Whitney test for ordinal variables (attitude test). A complete audit trail of all changes has been maintained for post-review purposes. Analyses were performed with Statistica.40
A total of 89 out of 89 learners consented to participate. Sixty-two percent were male; 24% were third year medical students; 76% were internal medicine residents. Approximately 75% of participants self-reported prior musculoskeletal training.
Shoulder physical examination maneuvers were graded using a locally validated standardized examination checklist (Online Appendix A) according to the following three-point scoring system: zero for maneuvers not attempted, one for maneuvers incorrectly performed, and two for maneuvers correctly performed. From pre to post assessment, learners significantly improved in 21 out of 23 shoulder physical examination maneuvers (Online Appendix D), and the average score for combined shoulder examination maneuvers improved from 0.65 to 1.4 out of 2, a 115% improvement.
Pre/post shoulder confidence survey results
Importance of shoulder exam
Confidence in shoulder history
Confidence in shoulder exam
Confidence in shoulder pain diagnosis
Confidence in shoulder injections
Knowledge examination scores were analyzed in three question categories: shoulder anatomy, diagnosis, and procedural knowledge. Post intervention, all three knowledge categories showed statistically significant improvement (p < 0.05) (Online Appendix E). Learners showed a 27.5% improvement in shoulder examination interpretation, an 11.7% improvement in accurate diagnosis of shoulder clinical cases, and a 17.2% improvement in procedural knowledge.
Although a growing body of literature calls for musculoskeletal curricular reform,25 – 31 competency-based musculoskeletal training has not been widely incorporated, measured, or reported in clinical settings. Our study demonstrates that direct observation and feedback in a primary care musculoskeletal clinic significantly improves shoulder examination competency, provider confidence, and knowledge in shoulder pain management.
Several limitations exist in this study. First, no validated definition or measurable outcomes for shoulder pain clinical competency exist. Currently, musculoskeletal competency in the literature is most often defined by scoring higher than 70% on the Freedman/Bernstein test, which has never been correlated with improved clinical outcomes.12 , 14 – 17 Thus, we used three measurable surrogate outcomes for clinical competency: shoulder examination scoring, confidence surveys, and knowledge tests on common shoulder conditions. Arguably, these outcomes are necessary precursors for the elusive concept of clinical competency that result in improved diagnosis and ultimately less shoulder pain for patients. In our experience, when learners develop a reliable shoulder physical examination, their confidence and interpretation of the history and physical examination result in increasingly accurate diagnoses and management. For instance, students might report “gerber lift-off is normal, so no subscapularis weakness is noted,” or “cross-arm test is negative, so the acromioclavicular arthritis noted on x-ray is not clinically significant,” or “the patient is limited in all directions of range of motion, consistent with adhesive capsulitis.” These observed connections are difficult to quantify as clinical competence, but the measurable outcomes of physical examination accuracy, confidence, and knowledge testing can demonstrate the effective impact of direct observation and feedback, which is a necessary foundation for a competency-based musculoskeletal training program. Second, no validated shoulder examination or knowledge test currently exists. Since the Freedman/Bernstein test includes only two shoulder knowledge questions, we created and validated a shoulder examination checklist and shoulder knowledge test with local musculoskeletal experts (see Program Description). Our shoulder examination checklist contains all of the same physical examination points included on a recent shoulder examination OSCE used to evaluate orthopedic residents.23 Third, shoulder examination scoring was not done live at point of care, but through evaluation of videotaped examinations. However, evaluation of videotaped examinations with an objective examination checklist has been shown to be as effective as live examination scoring.41 Although we did not measure inter-rater variability between the two independent video reviewers, the use of a simple three-point scale for the examination made scoring more objective and less prone to subjective interpretation.
To support progress toward competency-based education, which is often resource and time intensive, medical educators need to begin the difficult task of defining clinical competencies and observing the impact of training within clinical settings. Once specialty-specific musculoskeletal competencies have been validated, objective checklists may improve teaching consistency and define learning goals.42 , 43 Training faculty to implement a competency-based curriculum is another crucial step, as clinicians otherwise tend to assess learners subjectively.44 – 46 Faculty trained in direct observation of residents’ clinical competence had more stringent grading of residents’ histories and physical examinations and provided more feedback than untrained faculty.44 , 46 Although the transition seems daunting, once modeled, competency-based learning tends to propagate throughout clinical training as learners recognize opportunities to request increased observation, coaching, and point-of-care feedback.47
We demonstrate that direct observation and feedback during clinical evaluation of patients in a primary care musculoskeletal clinic improves shoulder examination competency, provider confidence, and knowledge of common shoulder conditions. In our experience, these three surrogate outcomes anecdotally lead learners toward improved shoulder pain diagnosis and management. Programs seeking to develop competency-based musculoskeletal curriculums may consider providing their faculty with the protected time, training, and resources needed for observed clinical teaching opportunities. Future studies may try to quantify the optimal frequency and number of observed shoulder cases required to perform shoulder examinations and injections independently. Increasing the number of providers with musculoskeletal competency is absolutely essential to manage the growing burden of musculoskeletal pain in an aging population.
The authors would like to offer appreciation to Leonie Heyworth in the General Internal Medicine Department of the San Diego VA Medical Center for her editorial assistance. Funding for the standardized patients was supported by the Center for Applied Research in Education through the UCSD Department of Medicine, Division of General Internal Medicine. The authors received no funding for this study and report no declarations of interest. A poster of this article was presented at the Society of General Internal Medicine California-Hawaii Regional meeting on 1/21/17, where it received first place in the innovations category.
Compliance with ethical standards
Conflict of interest
The authors declare no conflicts of interest.
- 1.United States Bone and Joint Initiative. Why Is the United States Bone and Joint Initiative Important? Available at http://ww.usbji.org/about/impact. Accessed June 23, 2017.
- 5.US Department of Veteran Affairs. 2015. VA Health Care Utilization by Recent Veterans. Available at http://www.publichealth.va.gov/epidemiology/reports/oefoifond/health-care- utilization/. Accessed June 23, 2017.
- 6.United States Bone and Joint Initiative. The Burden of Musculoskeletal Diseases in the United States. The Big Picture: Health Care Utilization and Economic Cost. Available at http://www.boneandjointburden.org/2014-report/if0/health-care-utilization-and-economic-cost. Accessed June 23, 2017.
- 18.Association of American Medical Colleges. Report VII contemporary issues in medicine: musculoskeletal medicine education. Medical School Objectives Project. Available at: https://members.aamc.org/eweb/upload/Contemporary%20Issues%20in%20Med%20Musculoskeletal%20Med%20Report%20VII%20.pdf . Accessed June 23, 2017.
- 34.United States Bone and Joint Initiative. Project 100: undergraduate musculoskeletal education. Available at http://www.usbji.org/programs/project-100. Accessed June 23, 2017.
- 40.StatSoft, Inc. Electronic Statistics Textbook. Tulsa: StatSoft; 2013.Google Scholar
- 44.Halman S, Dudek N, Wood T et al. Direct observation of clinical skills feedback scale: development and validity evidence. Teach Learn Med 2016; 10:1-10.Google Scholar