Advertisement

Journal of General Internal Medicine

, Volume 32, Issue 11, pp 1261–1265 | Cite as

Competency based clinical shoulder examination training improves physical exam, confidence, and knowledge in common shoulder conditions

  • Michal Kalli Hose
  • John Fontanesi
  • Manjulika Woytowitz
  • Diego Jarrin
  • Anna Quan
Innovation and Improvement: Innovations in Medical Education

Abstract

Background

Deficiencies in musculoskeletal knowledge are reported at every stage of learning. Medical programs are looking for effective ways to incorporate competency-based training into musculoskeletal education.

Aim

To evaluate the impact of bedside feedback on learner’s shoulder examination skills, confidence, and knowledge of common shoulder conditions.

Setting

Four-week musculoskeletal clinic rotation.

Participants

UCSD third year medical students and internal medicine residents.

Program description

Learners completed three baseline evaluations: videotaped shoulder examination, attitude survey, and knowledge test. During the 4-week intervention learners received bedside observation and feedback from musculoskeletal experts while evaluating patients with shoulder conditions. Post-intervention learners repeated the three assessments.

Program evaluation

Eighty-nine learners participated. In the primary outcome measure evaluating the pre/post videotaped shoulder examination, significant improvement was seen in 21 of 23 shoulder examination maneuvers. Secondary outcomes include changes in learner confidence and knowledge. Greatest gains in learner confidence were seen in performing the shoulder examination (61.5% improvement) and performing injections (97.1% improvement). Knowledge improved significantly in all categories including anatomy/examination interpretation, diagnosis, and procedures.

Discussion

Direct observation and feedback during clinical evaluation of patients with shoulder pain improves shoulder examination competency, provider confidence, and knowledge of common shoulder conditions.

Keywords

Musculoskeletal Shoulder examination Direct observation Competency-based medical education Primary care 

Introduction

Musculoskeletal complaints are a leading cause of chronic pain and disability, accounting for approximately 20% of primary care visits1 , 2 and 60% of American veteran health care utilization.3 5 In 2011, the estimated cost from musculoskeletal complaints was $874 billion in the US.6 The burden of musculoskeletal disease is projected to continue to rise sharply with the increased life expectancy of the general population.1 , 7

Primary care providers assess up to 70% of initial musculoskeletal complaints.8 , 9 However, many primary care providers feel inadequately trained to diagnose and treat musculoskeletal disorders.10 , 11 Objectively, insufficient musculoskeletal knowledge has been reported at every stage of learning, from medical students to residents to primary care providers.10 17

Mastery of the shoulder examination is recommended in the Association of American Medical Colleges musculoskeletal content report.18 With a prevalence of 16–21% in the general population, shoulder pain is the third most common site of musculoskeletal pain, after back and knee pain.19 21 However, recent analysis of the 2011 Step 2 medical board examinations documents deficiencies in musculoskeletal physical examination skills, including the shoulder examination.22 Even within an orthopedic specialty, a recent evaluation of residents reports deficiency in shoulder examination skills.23 In the study survey, 100% of the orthopedic residents reported that physical examination skills were observed only occasionally/rarely, and only 5% felt enough clinic time was devoted to teaching the physical examination.24 These studies demonstrate not only a need to improve teaching the shoulder examination, but also a glaring lack of point-of-care teaching in the clinical setting.

Competency-based medical education has recently emerged as a popular theoretical strategy among medical training programs. However, agreement on what constitutes clinical competency and how to measure it varies widely in the literature. Some of the fundamental characteristics of competency-based medical education include defining the desired competency from patient needs and then incorporating direct observation and feedback into learning experiences, with systematic evaluation before and after training.32 , 33 With the understanding that the desired clinical competency is accurate management of shoulder pain, we propose that the first step toward clinical competence is successfully performing a consistent shoulder physical examination, which can increase provider confidence and improve diagnostic knowledge. We describe an observational study on whether bedside feedback on shoulder cases in a clinical setting improves learners’ physical examination skills, confidence in diagnostic ability, and medical knowledge of shoulder pain.

Setting and participants

This study was approved by the San Diego Veterans Administration (VA) and University of California, San Diego (UCSD) Institutional Review Boards in accordance with the requirements of the Code of Federal Regulations on the Protection of Human Subjects.

In July 2003, the San Diego VA primary care musculoskeletal clinic was created to provide non-surgical treatment options, including joint injections, for common musculoskeletal complaints. The clinic is staffed by two primary care musculoskeletal experts with national experience in musculoskeletal teaching. UCSD medical students and internal medicine residents rotate through the musculoskeletal clinic once weekly for a month. In 2007 and 2008, each UCSD learner rotating through the musculoskeletal clinic was invited to participate in this study. After discussion of the risks and benefits of participation, learners signed an informed consent.

Program description

In response to Project 100, an educational initiative challenging 100% of medical schools to implement musculoskeletal curricula, 83% of schools had complied by 2010.34 , 35 But the vast majority of musculoskeletal curricula remained preclinical. Despite the fact that medical educators describe coaching and direct observation as "fundamentally essential to training qualified physicians,”33 , 36 only 24% of programs required a clinical musculoskeletal clerkship.35 Since primary care musculoskeletal clinics have previously been reported as effective sites for evaluating common musculoskeletal complaints,37 39 we decided to use our musculoskeletal clinic to evaluate a teaching intervention for common shoulder conditions.

The three study outcomes for our learners were improvement in performance of the shoulder examination, provider confidence, and knowledge of managing common shoulder conditions. Before starting the musculoskeletal rotation, learners completed three baseline evaluations: a videotaped shoulder examination on a standardized patient (Online Appendix A), an attitude survey of confidence regarding shoulder examination and management (Online Appendix B), and a knowledge test on management of common shoulder conditions (Online Appendix C). The learners did not receive the results of their pre-test evaluations or keep the test questions for reference. The shoulder physical examination checklist and knowledge test were developed in consultation with seven local sports medicine, medical education, and primary care experts to make it clinically relevant to primary care shoulder conditions. These providers span a spectrum of clinical settings: academic, active-duty military, and community/private. Every knowledge question and physical examination component was rated by each provider for clinical relevance.

After completing the baseline study evaluations, learners experienced the teaching innovation: repeated directly observed evaluations of patients with shoulder pain, with point-of-care feedback. On average, each learner was assigned five to six shoulder patients over the musculoskeletal rotation. Internal medicine attendings trained in musculoskeletal medicine observed the clinic visit and gave trainees immediate feedback on the physical examination, standardized according to a written checklist (Online Appendix A). Diagnosis, assessment, and treatment, including subacromial bursa injections, were also performed bedside under attending physician guidance.

At the end of the 4-week musculoskeletal rotation, each learner repeated the three assessments: a post-intervention videotaped shoulder examination, attitude survey, and knowledge test. The videotaped shoulder examinations were graded by two independent raters blinded to the pre- or post-intervention status of the learner.

Statistical methods

Data analysis followed the standard Cross Industry Standard Process to assure data integrity and to declare each data element type, distribution family type, and appropriate analytic methods for the study question(s). Descriptive statistics including frequencies, percentages, means and standard deviations were calculated for each item. Comparison of pre-post change in pre and post items was conducted using Student’s t-test for continuous variables (knowledge test), McNemar test for categorical data (video), and Mann-Whitney test for ordinal variables (attitude test). A complete audit trail of all changes has been maintained for post-review purposes. Analyses were performed with Statistica.40

Program evaluation

A total of 89 out of 89 learners consented to participate. Sixty-two percent were male; 24% were third year medical students; 76% were internal medicine residents. Approximately 75% of participants self-reported prior musculoskeletal training.

Physical examination

Shoulder physical examination maneuvers were graded using a locally validated standardized examination checklist (Online Appendix A) according to the following three-point scoring system: zero for maneuvers not attempted, one for maneuvers incorrectly performed, and two for maneuvers correctly performed. From pre to post assessment, learners significantly improved in 21 out of 23 shoulder physical examination maneuvers (Online Appendix D), and the average score for combined shoulder examination maneuvers improved from 0.65 to 1.4 out of 2, a 115% improvement.

Figure 1 displays components of the videotaped shoulder examination aggregated into five examination categories: observation, palpation, range of motion, strength and provocative tests. The pre-intervention baseline score for each examination category was <1, indicating the shoulder examination maneuvers were either not attempted or performed incorrectly. Post-intervention, scoring in all five examination categories showed 50–100% improvement over initial testing scores. The largest improvement is seen in shoulder provocative maneuvers, with scores rising from 0.21 to 1.18 on a two-point scale (P value <0.05).
Fig. 1

Pre/post change in average score for video-taped shoulder examination by examination category. Physical examination scoring system: 0 = maneuver was not tried; 1 = maneuver was tried but done incorrectly; 2 = maneuver was performed correctly. Light gray represents pre-test results; dark gray represent post-test results. All comparisons are statistically significant with p value <0.05

Confidence

From pre to post intervention, a statistically significant increase in confidence scores is seen in all learner levels, from third year medical students to senior medical residents (Table 1). Learners felt more confident in all aspects of shoulder management, with the greatest gains seen in performing the shoulder physical examination and subacromial bursa injections.
Table 1

Pre/post shoulder confidence survey results

Variable

T-tests

Mean

Pre (SD)

Mean

Post (SD)

t-value

p

Importance of shoulder exam

4.31

(± 0.77)

4.29

(± 0.78)

0.19

0.84

Confidence in shoulder history

3.37

(± 0.89)

4.29

(± 0.53)

−8.35

0.00

Confidence in shoulder exam

2.57

(± 0.83)

4.15

(±− 0.64)

−14.16

0.00

Confidence in shoulder pain diagnosis

2.65

(± 0.77)

3.89

(± 0.59)

−11.91

0.00

Confidence in shoulder injections

2.06

(± 0.96)

4.06

(± 0.68)

−15.96

0.00

Scale range from 1 to 5, with 5 high. Bold are statistically significant at 0.05 level

Knowledge

Knowledge examination scores were analyzed in three question categories: shoulder anatomy, diagnosis, and procedural knowledge. Post intervention, all three knowledge categories showed statistically significant improvement (p < 0.05) (Online Appendix E). Learners showed a 27.5% improvement in shoulder examination interpretation, an 11.7% improvement in accurate diagnosis of shoulder clinical cases, and a 17.2% improvement in procedural knowledge.

Discussion

Although a growing body of literature calls for musculoskeletal curricular reform,25 31 competency-based musculoskeletal training has not been widely incorporated, measured, or reported in clinical settings. Our study demonstrates that direct observation and feedback in a primary care musculoskeletal clinic significantly improves shoulder examination competency, provider confidence, and knowledge in shoulder pain management.

Several limitations exist in this study. First, no validated definition or measurable outcomes for shoulder pain clinical competency exist. Currently, musculoskeletal competency in the literature is most often defined by scoring higher than 70% on the Freedman/Bernstein test, which has never been correlated with improved clinical outcomes.12 , 14 17 Thus, we used three measurable surrogate outcomes for clinical competency: shoulder examination scoring, confidence surveys, and knowledge tests on common shoulder conditions. Arguably, these outcomes are necessary precursors for the elusive concept of clinical competency that result in improved diagnosis and ultimately less shoulder pain for patients. In our experience, when learners develop a reliable shoulder physical examination, their confidence and interpretation of the history and physical examination result in increasingly accurate diagnoses and management. For instance, students might report “gerber lift-off is normal, so no subscapularis weakness is noted,” or “cross-arm test is negative, so the acromioclavicular arthritis noted on x-ray is not clinically significant,” or “the patient is limited in all directions of range of motion, consistent with adhesive capsulitis.” These observed connections are difficult to quantify as clinical competence, but the measurable outcomes of physical examination accuracy, confidence, and knowledge testing can demonstrate the effective impact of direct observation and feedback, which is a necessary foundation for a competency-based musculoskeletal training program. Second, no validated shoulder examination or knowledge test currently exists. Since the Freedman/Bernstein test includes only two shoulder knowledge questions, we created and validated a shoulder examination checklist and shoulder knowledge test with local musculoskeletal experts (see Program Description). Our shoulder examination checklist contains all of the same physical examination points included on a recent shoulder examination OSCE used to evaluate orthopedic residents.23 Third, shoulder examination scoring was not done live at point of care, but through evaluation of videotaped examinations. However, evaluation of videotaped examinations with an objective examination checklist has been shown to be as effective as live examination scoring.41 Although we did not measure inter-rater variability between the two independent video reviewers, the use of a simple three-point scale for the examination made scoring more objective and less prone to subjective interpretation.

To support progress toward competency-based education, which is often resource and time intensive, medical educators need to begin the difficult task of defining clinical competencies and observing the impact of training within clinical settings. Once specialty-specific musculoskeletal competencies have been validated, objective checklists may improve teaching consistency and define learning goals.42 , 43 Training faculty to implement a competency-based curriculum is another crucial step, as clinicians otherwise tend to assess learners subjectively.44 46 Faculty trained in direct observation of residents’ clinical competence had more stringent grading of residents’ histories and physical examinations and provided more feedback than untrained faculty.44 , 46 Although the transition seems daunting, once modeled, competency-based learning tends to propagate throughout clinical training as learners recognize opportunities to request increased observation, coaching, and point-of-care feedback.47

We demonstrate that direct observation and feedback during clinical evaluation of patients in a primary care musculoskeletal clinic improves shoulder examination competency, provider confidence, and knowledge of common shoulder conditions. In our experience, these three surrogate outcomes anecdotally lead learners toward improved shoulder pain diagnosis and management. Programs seeking to develop competency-based musculoskeletal curriculums may consider providing their faculty with the protected time, training, and resources needed for observed clinical teaching opportunities. Future studies may try to quantify the optimal frequency and number of observed shoulder cases required to perform shoulder examinations and injections independently. Increasing the number of providers with musculoskeletal competency is absolutely essential to manage the growing burden of musculoskeletal pain in an aging population.

Notes

Acknowledgements

The authors would like to offer appreciation to Leonie Heyworth in the General Internal Medicine Department of the San Diego VA Medical Center for her editorial assistance. Funding for the standardized patients was supported by the Center for Applied Research in Education through the UCSD Department of Medicine, Division of General Internal Medicine. The authors received no funding for this study and report no declarations of interest. A poster of this article was presented at the Society of General Internal Medicine California-Hawaii Regional meeting on 1/21/17, where it received first place in the innovations category.

Compliance with ethical standards

Conflict of interest

The authors declare no conflicts of interest.

Supplementary material

11606_2017_4143_MOESM1_ESM.docx (16 kb)
Online Appendix A (DOCX 15 kb)
11606_2017_4143_MOESM2_ESM.docx (14 kb)
Online Appendix B (DOCX 13 kb)
11606_2017_4143_MOESM3_ESM.docx (15 kb)
Online Appendix C (DOCX 15 kb)
11606_2017_4143_MOESM4_ESM.docx (15 kb)
Online Appendix D (DOCX 15 kb)
11606_2017_4143_MOESM5_ESM.docx (73 kb)
Online Appendix E (DOCX 72 kb)

References

  1. 1.
    United States Bone and Joint Initiative. Why Is the United States Bone and Joint Initiative Important? Available at http://ww.usbji.org/about/impact. Accessed June 23, 2017.
  2. 2.
    Woolf AD, Pfleger B. Special theme – bone and joint decade 2000 –2010: burden of major musculoskeletal conditions. Bull World Health Organ 2003;81:646-656.PubMedPubMedCentralGoogle Scholar
  3. 3.
    Beehler GP, Rodrigues AE, Mercurio-Riley D, Dunn AS. Primary care utilization among veterans with chronic musculoskeletal pain: a retrospective chart review. Pain Med 2013;14:1021-1031.CrossRefPubMedGoogle Scholar
  4. 4.
    Cameron KL, Hsiao MS, Owens BD, Burks R, Svoboda, SJ. Incidence of physician-diagnosed osteoarthritis among active duty United States military service members. Arthritis Rheum 2011;63:2974–2982.CrossRefPubMedGoogle Scholar
  5. 5.
    US Department of Veteran Affairs. 2015. VA Health Care Utilization by Recent Veterans. Available at http://www.publichealth.va.gov/epidemiology/reports/oefoifond/health-care- utilization/. Accessed June 23, 2017.
  6. 6.
    United States Bone and Joint Initiative. The Burden of Musculoskeletal Diseases in the United States. The Big Picture: Health Care Utilization and Economic Cost. Available at http://www.boneandjointburden.org/2014-report/if0/health-care-utilization-and-economic-cost. Accessed June 23, 2017.
  7. 7.
    March L, Smith EU, Hoy DG et al. Burden of disability due to musculoskeletal (MSK) disorders. Best Pract Res Clin Rheumatol 2014;28:353-66.CrossRefPubMedGoogle Scholar
  8. 8.
    Karpman, RR. Musculoskeletal disease in the United States: who provides the care? Clin Orthop Relat Res 2001;385:52-56.CrossRefGoogle Scholar
  9. 9.
    Uhlig T, Hagen KB, Kvien TK. Why do patients with chronic musculoskeletal disorders consult their primary care physicians? Curr Opin Rheumatol 2002;14:104-108.CrossRefPubMedGoogle Scholar
  10. 10.
    Matheny JM, Brinker MR, Elliott MN, Blake R, Rowane MP. Confidence of graduating family practice residents in their management of musculoskeletal conditions. Am J Orthop 2000;29:945-951.PubMedGoogle Scholar
  11. 11.
    Roberts C, Adebajo AO, Long S. Improving the quality of care of musculoskeletal conditions in primary care. Rheumatology 2002;41:503-508.CrossRefPubMedGoogle Scholar
  12. 12.
    Day CS, Yeh AC, Franko O, Ramirez M, Krupat E. Musculoskeletal medicine: an assessment of the attitudes and knowledge of medical students at Harvard Medical School. Acad Med 2007;82:452-457.CrossRefPubMedGoogle Scholar
  13. 13.
    Day CS, Yeh AC. Evidence of educational inadequacies in region-specific musculoskeletal medicine. Clin Orthop Relat Res 2008;466:2542-2547.CrossRefPubMedPubMedCentralGoogle Scholar
  14. 14.
    Freedman KB, Bernstein J. The adequacy of medical school education in musculoskeletal medicine. J Bone Joint Surg Am 1998;80:1421-1427.CrossRefPubMedGoogle Scholar
  15. 15.
    Freedman KB, Bernstein J. Educational deficiencies in musculoskeletal medicine. J Bone Joint Surg Am 2002;84:604-608.CrossRefPubMedGoogle Scholar
  16. 16.
    Matzkin E, Smith E, Freccero D, Richardson A. Adequacy of education in musculoskeletal medicine. J Bone Joint Surg Am 2005;87:310-314.CrossRefPubMedGoogle Scholar
  17. 17.
    Skelley NW, Tanaka MJ, Skelley LM, LaPorte DM. Medical student musculoskeletal education: an institutional survey. J Bone Joint Surg Am 2012; 94:1-7.CrossRefGoogle Scholar
  18. 18.
    Association of American Medical Colleges. Report VII contemporary issues in medicine: musculoskeletal medicine education. Medical School Objectives Project. Available at: https://members.aamc.org/eweb/upload/Contemporary%20Issues%20in%20Med%20Musculoskeletal%20Med%20Report%20VII%20.pdf . Accessed June 23, 2017.
  19. 19.
    Chard MD, Hazleman R, Hazleman BL, King RH, Reiss BB. Shoulder disorders in the elderly: a community survey. Arthritis Rheum 1991;34:766-769.CrossRefPubMedGoogle Scholar
  20. 20.
    Solomon DH, Bates DW, Schaffer JL, Horsky J, Burdick E, Katz JN. Referrals for musculoskeletal disorders: patterns, predictors, and outcomes. J Rheumatol 2001;28:2090-2095.PubMedGoogle Scholar
  21. 21.
    Urwin M, Symmons D, Allison T et al. Estimating the burden of musculoskeletal disorders in the community: the comparative prevalence of symptoms at different anatomical sites, and the relation to social deprivation. Ann Rheum Dis 1998;57:649-655.CrossRefPubMedPubMedCentralGoogle Scholar
  22. 22.
    Peitzman SJ, Cuddy MM. Performance in physical examination on the USMLE Step 2 clinical skills examination. Acad Med 2015;90:209–213.CrossRefPubMedGoogle Scholar
  23. 23.
    Griesser MJ, Beran MC, Flanigan DC, Quackenbush M, Van Hoff C, Bishop JY. Implementation of an Objective Structured Clinical Exam (OSCE) into orthopedic surgery residency training. J Surg Educ 2012;69:180–189.CrossRefPubMedGoogle Scholar
  24. 24.
    Beran MC; Awan H; Rowley D; Samora JB; Griesser MJ; Bishop JY. Assessment of musculoskeletal physical examination skills and attitudes of orthopaedic residents. J Bone Joint Surg Am 2012;94: e36.CrossRefPubMedGoogle Scholar
  25. 25.
    Akesson K, Dreinhofer KE, Woolf AD. Improved education in musculoskeletal conditions is necessary for all doctors. Bull World Health Organ 2003; 81:677-683.PubMedPubMedCentralGoogle Scholar
  26. 26.
    Bernstein J, Alonso DR, DiCaprio M, Friedlaender GE, Heckman JD, Ludmerer KM. Curricular reform in musculoskeletal medicine: needs, opportunities, and solutions. Clin Orthop Relat Res 2003; 415:302-308.CrossRefGoogle Scholar
  27. 27.
    Clawson DK, Jackson DW, Ostergaard DJ. It’s past time to reform the musculoskeletal curriculum. Acad Med 2001;76:709-710.CrossRefPubMedGoogle Scholar
  28. 28.
    DiCaprio MR, Covey A, Bernstein J. Curricular requirements for musculoskeletal medicine in American medical schools. J Bone Joint Surg Am 2003;85:565-577.CrossRefPubMedGoogle Scholar
  29. 29.
    Monrad SU, Zeller J, Craig C, DiPonio L. Musculoskeletal education in US medical schools: lessons from the past and suggestions for the future. Curr Rev Musculoskelet Med 2011;4:91–98.CrossRefPubMedPubMedCentralGoogle Scholar
  30. 30.
    Murphy RF, LaPorte DM, Wadey VM, American Academy of Orthopaedic Surgeons Orthopaedic Education Study Group. Musculoskeletal education in medical school: deficits in knowledge and strategies for improvement. J Bone Joint Surg Am 2014;96:2009-2014.CrossRefPubMedGoogle Scholar
  31. 31.
    Woolf AD, Walsh NE, and Akesson K. Global core recommendations for a musculoskeletal undergraduate curriculum. Ann Rheum Dis 2004;63:517-524.CrossRefPubMedPubMedCentralGoogle Scholar
  32. 32.
    Frank JR, Mungroo R, Ahmad Y, Wang M, De Rossi S, Horsley T. Toward a definition of competency-based education in medicine: a systematic review of published definitions. Med Teach 2010;32:631-637.CrossRefPubMedGoogle Scholar
  33. 33.
    Frank JR, Snell LS, Ten Cate O et al. Competency-based medical education: theory to practice. Med Teach 2010;32:638–645.CrossRefPubMedGoogle Scholar
  34. 34.
    United States Bone and Joint Initiative. Project 100: undergraduate musculoskeletal education. Available at http://www.usbji.org/programs/project-100. Accessed June 23, 2017.
  35. 35.
    Bernstein J, Garcia GH, Guevara JL, Mitchell GW. Progress report: the prevalence of required medical school instruction in musculoskeletal medicine at decade’s end. Clin Orthop Relat Res 2011; 469:895–897.CrossRefPubMedGoogle Scholar
  36. 36.
    Fromme HB, Karani R, Downing SM. Direct observation in medical education: a review of the literature and evidence for validity. Mt Sinai J Med 2009;76:365-71.CrossRefPubMedGoogle Scholar
  37. 37.
    Houston TK, Connors RL, Cutler N, Nidiry MA. A primary care musculoskeletal clinic for residents—success and sustainability. J Gen Intern Med 2004;19:524–529.CrossRefPubMedPubMedCentralGoogle Scholar
  38. 38.
    Vernec, A, Shrier, I. A teaching unit in primary care sports medicine for family medicine residents. Acad Med 2001;76;293-296.CrossRefPubMedGoogle Scholar
  39. 39.
    Wilcox T, Oyler J, Harada C, Utset T. Musculoskeletal exam and joint injection training for internal medicine residents. J Gen Intern Med 2006;21:521–523.CrossRefPubMedPubMedCentralGoogle Scholar
  40. 40.
    StatSoft, Inc. Electronic Statistics Textbook. Tulsa: StatSoft; 2013.Google Scholar
  41. 41.
    Vivekananda-Schmidt P, Lewis M, Coady D, Morley C, Kay L, Walker D. Exploring the use of videotaped objective structured clinical examination in the assessment of joint examination skills of medical students. Arthritis Rheum 2007;57:869-876.CrossRefPubMedGoogle Scholar
  42. 42.
    Altschuler EL, Cruz E, Salim SZ et al. Efficacy of a checklist as part of a physical medicine and rehabilitation clerkship to teach medical students musculoskeletal physical examination skills: a prospective study. Am J Phys Med Rehab 2014;93:82-89.CrossRefGoogle Scholar
  43. 43.
    Shayne P, Gallahue F, Rinnert S et al. Reliability of a core competency checklist assessment in the emergency department: the standardized direct observation assessment tool. Acad Emerg Med 2006;13:727-32.CrossRefPubMedGoogle Scholar
  44. 44.
    Halman S, Dudek N, Wood T et al. Direct observation of clinical skills feedback scale: development and validity evidence. Teach Learn Med 2016; 10:1-10.Google Scholar
  45. 45.
    Hauer KE, Holmboe ES, Kogan JR. Twelve tips for implementing tools for direct observation of medical trainees' clinical skills during patient encounters. Med Teach 2011;33:27-33.CrossRefPubMedGoogle Scholar
  46. 46.
    Holmboe ES, Hawkins RE, Huot SJ. Effects of training in direct observation of medical residents' clinical competence: a randomized trial. Ann Intern Med 2004;140:874-881.CrossRefPubMedGoogle Scholar
  47. 47.
    Kogan JR, Holmboe ES, Hauer KE. Tools for direct observation and assessment of clinical skills of medical trainees. a systematic review. JAMA. 2009;302:1316-1326.CrossRefPubMedGoogle Scholar

Copyright information

© Society of General Internal Medicine (outside the USA) 2017

Authors and Affiliations

  • Michal Kalli Hose
    • 1
    • 2
  • John Fontanesi
    • 3
  • Manjulika Woytowitz
    • 1
    • 2
  • Diego Jarrin
    • 1
    • 2
  • Anna Quan
    • 1
    • 2
  1. 1.VA San Diego Healthcare SystemSan DiegoUSA
  2. 2.University of California San DiegoLa JollaUSA
  3. 3.Family and Preventive MedicineUniversity of California San DiegoLa JollaUSA

Personalised recommendations