INTRODUCTION

Musculoskeletal (MSK) diseases are common, and many patients seek care for these problems in primary care clinics.1 – 3 In the past decade, MSK conditions were identified as the most frequent diagnoses in primary care clinic encounters in the US.4 Despite this prevalence, physicians and other health care providers in primary care settings are often not well prepared to effectively address common MSK problems.5 – 11 A summary statement from The US Bone and Joint Initiative’s 2011 Summit on The Value in Musculoskeletal Care includes the following recommendation:

“Training programs for all health care providers should improve the knowledge, skills, and attitudes of all professionals in the diagnosis and management of musculoskeletal conditions. At present, many graduates report a deficit of knowledge of musculoskeletal conditions and competence in patient evaluation and treatment, including performance of the musculoskeletal physical examination.”12

Previously, we projected the potential cost-effectiveness of a week-long continuing professional development (CPD) program developed to train practicing physicians and other health care providers to more effectively evaluate and manage common MSK problems.13 , 14 A pilot project supported by Specialty Care Services/Specialty Care Transformation (SCS/SCT), Office of Patient Care Services, Veterans Affairs (VA) Central Office, confirmed that such a MSK “mini-residency” program was feasible at the local level and cost effective.15 Although this projection was based on the change in the frequency of joint injections performed by providers after the program, course participants reported that one of the most valuable outcomes was their improved ability to perform, report, and interpret the MSK physical examination. These enhanced skills allowed primary care providers to develop more appropriate management plans, regardless of whether these plans included procedures that they personally performed. Thus, it became clear that while not all primary care providers were interested in developing proficiency in performing joint injections, all of these trainees were eager to develop physical examination and clinical evaluation skills for the treatment of MSK complaints. A challenge with this program as initially designed was the very limited capacity for participation relative to the numbers of applicants—limited in part by the required clinical training in joint injection.

We saw a critical need to determine the generalizability of our experience to other sites and the scalability of this program to a national dimension.15 The objectives of this phase of the project were threefold: (1) to increase the capacity of this program so that more providers could be trained, (2) to strengthen participants’ confidence in examining the shoulder and knee, and (3) to document their competence in these examinations, demonstrating sufficient preparation for subsequent supervised clinical experiences including joint injections. These enhancements of the program were developed as the central theme of the mini-residency with the simultaneous development of separate training opportunities for primary care providers who desired to develop joint injection proficiencies. We now describe the initial 2 years of experience in disseminating this national MSK mini-residency program.

METHODS

Site Selection

Sites were selected in response to a request for proposals (RFP) from the Office of Specialty Care Services. The proposal supported the travel of a team of five Salt Lake City faculty to each location to present the program in partnership with local faculty. Although these applications were critically reviewed as discussed below, the goal of the initiative was to disseminate the program as broadly as possible, while preserving consistency of instruction and assessment. In the proposal, each site was required to (1) identify two local leaders—one generalist and one rheumatologist (or other MSK specialist)—to work with the Salt Lake City faculty in presenting the mini-residency at their local site; (2) negotiate and secure letters from their local leadership and administration in support of this program, including the plan for subsequent supervised clinical experiences; (3) propose a budget, outlining anticipated costs and expenditures for up to $30,000 for each site; this did not include costs for the Salt Lake City faculty, which were supported separately by funding from VA SCS/SCT for this program.

In addition to an administrative review by VA SCS/SCT project managers, each proposal was reviewed independently by three Salt Lake City faculty (MJB, AMB, and GWC). This review focused on (1) the description of the need for this type of educational program at the potential site and how this need would be met by participation in the mini-residency, (2) the willingness of clinical leaders and educators to work with Salt Lake City faculty in co-presenting the educational program, particularly in facilitating the participation of local PCPs and helping to arrange event logistics, and (3) the support of local administrative leadership to make space and time available for the conference and subsequent clinical activities. Following the initial separate review of the submitted proposals, Salt Lake City faculty discussed their impressions together and with VA SCS/SCT project managers to recommend decisions on funding. Finally, Salt Lake City faculty worked with the local teams through conference calls and email to further refine their proposed projects and budgets to ensure feasibility and enhance efficiency and return on investment.

Leaders from sites that were approved for funding to participate then traveled to Salt Lake City for a 3-day “train-the-trainer” meeting to become familiar with the key elements of the curriculum and to begin logistical planning for hosting a mini-residency event at their site. Following this training meeting, periodic conference calls were held with individual sites to facilitate the detailed planning required to coordinate a successful event.

In the second year, the Salt Lake City faculty presented this program at their home facility in addition to the off-site mini-residencies in order to serve the needs of Salt Lake City-based primary care providers. No additional funds were needed to support this program.

Three-day Intensive Training

Course Structure

The curriculum and schedule of the 5-day Salt Lake City-based MSK mini-residency program have been described previously; this served as the initial framework.15 The national version was condensed to 3 days (Fig. 1), retaining the emphasis on the instruction and assessment of physical examination skills of the shoulder and the knee. Although additional joints relevant to other regional pain syndromes were recognized (such as hip and elbow), the course focused on the shoulder and knee because the goal of the initiative was to disseminate the curriculum that had been developed and refined in the local pilot experience. Individual clinical experiences, including opportunities to practice joint injection techniques with real patients under the supervision of the local faculty, were planned as follow-on experiences to occur after the conclusion of the 3-day group program. Competence in performing the physical examination of the shoulder and knee was identified as the indicator of adequate preparation for possible subsequent (post-course) supervised experiences in procedural clinics. The primary objective of this 3-day program was to prepare participants for success in the assessment of this competence and to document their completion of this milestone. The course was accredited through the VA Employee Education System (EES), and participants received 24 h of continuing professional education credit for each week.

Figure 1
figure 1

Three-day Schedule for the National Musculoskeletal Mini-Residency Program

Full size image

Program Evaluation

Course assessment was performed through the VA EES, using a standard four-level process for evaluating and measuring learning solutions and activities. To work towards an expanded outcomes framework, we developed two additional measures to assess this activity:

Self-Assessments

We considered self-assessment to be a complex cognitive task, grounded in context and dependent upon experience.16 – 18 We followed the approach of Nagler and administered three self-assessment surveys: (1) at the beginning of the program (prospective pre-course), (2) at the end of the program (post-course), and (3) at the end of the program, with questions specifically addressing pre-course proficiency (retrospective pre-course).19 At each site, participants used Likert scales anchored at five points ranging from 1 (strongly disagree) to 5 (strongly agree), rating proficiency relative to course goals (pre-course). After the 3-day program, the same dimensions were used to self-rate post-course proficiency and to retrospectively rate their pre-course proficiency—in effect, capturing information that participants “didn’t know they didn’t know” before the training experience. The domains of the self-assessment instruments map to the goals of the course: (1) to evaluate and manage common MSK conditions, (2) to develop the procedural skills of joint aspiration and injection, and (3) to screen patients for osteopenia and osteoporosis.

Observed Structured Clinical Examinations

Participants’ skills in physical examination of the shoulder and knee were assessed in a two-station observed structured clinical examination (OSCE). Details of these examinations have been presented previously, including evidence supporting the use of these exercises as capstone elements in structured educational programs.20 , 21 Two faculty were involved in both stations, one as the simulated patient and the other as rater. The OSCE was scored using the same shoulder and knee checklists used for instruction (21 elements for the shoulder and 25 for the knee).

This project was reviewed by the Institutional Review Board (IRB) of the University of Utah and Salt Lake City VA and was determined to be exempt from further IRB review because the work did not meet the definition of research with human subjects and was considered a quality improvement study.

RESULTS

Six proposals were submitted in each of the 2 years of this initiative, and all of them were funded. The program was presented at 13 sites (including Salt Lake City). Two hundred twenty-seven health care providers were trained, including 135 physicians. Numbers of participants at each site and their respective credentials are presented in Table 1.

Table 1 Participants in the National Musculoskeletal Mini-Residency Program over 2 Years
Full size table

Two hundred twenty-five prospective pre-course, 213 post-course, and 207 retrospective pre-course surveys were collected (response rates of 99 %, 94 %, and 91 %, respectively); mean self-assessment ratings are shown in Table 2. When pre-course mean ratings of confidence were calculated for items relating to the evaluation and management of shoulder, knee, and back problems, participants were significantly less confident with both shoulder and knee complaints than they were with problems involving the back (p = 0.01 comparing mean ratings for the back to both shoulder and knee). The difference in mean ratings of confidence was not statistically significant (p = 0.4) between the shoulder and the knee.

Table 2 Pre- and Post-course (Including Retrospective Pre-course) Self-Assessment Ratings
Full size table

Post-course self-assessments were higher than pre-course ratings in all tested domains. Mean retrospective ratings of pre-course ability to manage shoulder and knee complaints were significantly lower than prospective pre-course ratings. In other words, after the course was completed, participants were, in retrospect, more critical of their pre-course proficiency than they had been when the course began.

Two hundred twelve participants (93 %) completed the OSCE skills assessment. Figure 2 shows the frequency distributions of OSCE scores by percentage decile for both the shoulder and knee. The proportion of participants who scored at the 81–100 deciles was 79 % for the shoulder and 65 % for the knee. Mean scores for both stations were high (90 % for shoulder, 86 % for knee, p < 0.0001), indicating the success of the program at teaching proficiency in these physical examination techniques.

Figure 2
figure 2

Distribution of OSCE scores (percent of items performed) for 212 participants

Full size image

DISCUSSION

We have achieved the goal of implementing and disseminating a national, multidisciplinary, mixed-method educational initiative to strengthen the skills of practicing PCPs in evaluating and managing patients with MSK complaints. Participants’ self-assessments demonstrate significant changes—from pre-course perceptions of inadequacy to post-course confidence. These changes were greater for items specific to the shoulder and knee than for those relating to the back. Accordingly, retrospective ratings of pre-course ability to manage shoulder and knee complaints were lower than prospective pre-course ratings. In contrast, this pattern was not seen for the self-assessment items relevant to back pain. In the pilot phase of the MSK mini-residency we found that primary care providers were relatively confident in addressing common back complaints, so this topic was not emphasized in the national course. This observation was confirmed in the self-assessment reported by our mini-residency participants.

Although reports describing MSK educational programs and curricula have increased in recent years, these have typically focused on the undergraduate and graduate levels of medical education. In 2012, O’Dunn-Orto and colleagues presented a Best Evidence in Medical Education systematic review seeking to determine interventions that were most effective in promoting transfer of MSK clinical skills to medical trainees and physicians.11 Of the 24 controlled studies they identified, only 1 involved practicing physicians.22 Several survey studies and reports describing initiatives for practitioners have been published, presenting experiences and perspectives from Canadian centers. In 2007, Bellamy et al. described the Canadian Viscosupplement Injector Preceptor experience in which 474 primary care physicians participated in a 4-h training session that culminated in their performance of a joint injection with viscosupplementation under expert supervision. Although this study was significant in that it incorporated patient outcomes into their assessment model, assessments of learner competence in this program were not captured.23 In 2006, Petrella and Davis published a review of the Joint Adventures Program, a Canadian national initiative in which 650 family physicians across Canada participated in workshops in which they analyzed and discussed written clinical scenarios depicting common MSK disorders using script concordance methods to address knowledge gaps.24 Although this program involved a relatively large number of providers across a broad geographic range, neither knowledge nor physical examination skills were assessed.

To our knowledge, this is the first description of a nationally consistent CPD program focused on teaching and assessing competence in evaluating MSK problems in primary care across the US. The success of this initiative highlights one of the unique strengths of the VA: it is a national health care system and provides a mechanism of developing innovative educational programs and extending them to a national audience. Several additional factors may further explain why this project of implementation and dissemination was successful. First, participation in the MSK mini-residency was driven by provider interest. This was not a required program, and some initiative on the part of individual providers was required in order to participate. This likely produced a group primed for learning because they were interested in this subject and were motivated to acquire new skills in this area. This view is informed by the Chain of Response model of learning, which emphasizes the importance of active participation in projects. In this model, three internal and inter-related motivating factors are identified: (1) self-evaluation, (2) attitude of the learner towards educational experiences, and (3) goals and expectations specific to each individual learner.25 , 26 Second, the immersive, intensive, mixed-methods approach of this course is consistent with teaching strategies that have been recommended in a recent systematic review of MSK education techniques.11 Although some features of our Salt Lake City MSK mini-residency were adapted for the national program, we retained one of the key strengths—the element of a stable group of learners and faculty who worked together across a range of educational experiences, creating a cohesive, integrated program and a powerful element of continuity. Third, this program provided the opportunity for educational leaders at individual sites to work in partnership, collaborating with us in an initiative that had already been considered to be successful on a smaller scale. This approach has created a national network of training centers, “educational nodes” that may facilitate rapid dissemination of future innovations that are nationally consistent and locally coherent, adaptable to the specific and unique needs and resources of individual clinics and care centers.

Recently, Moore and others proposed an expanded outcomes model as a framework for planning and assessing CPD.27 This seven-stage model identifies competence—defined as the ability to demonstrate how to do something in an educational setting—as being the “extraordinarily important” link between knowledge and performance. From this perspective, incorporating the assessment of competence in CPD programs is critical. Using Moore’s expanded outcomes framework, we see evidence of the successful national dissemination of the MSK mini-residency. The number of participants, as well as the number of participating sites, shows that the construction of educational partnerships is both feasible and efficient, providing opportunities to train many more providers over much shorter time, without requiring them to travel to one location to attend the course. Declarative knowledge, represented in our program by self-assessment ratings, increased over the course. Furthermore, after the program, participants retrospectively rated their pre-course knowledge lower than they had rated it when the course began. This change indicated that participants recognized learning needs that had emerged in the context of their training. The opportunity for providers to practice what they were learning, while receiving immediate feedback grounded in direct observations by course faculty, facilitated the development of competence; this was also systematically assessed in our two-station OSCE. Our experience may be helpful to educators who have an interest in other clinical problems that overlap primary care and specialty care and who would like to develop educational innovations and programs to address needs in these areas. This may be particularly valuable for those concerns that involve an aspect of technical or procedural expertise that is within the scope of practice for generalists (e.g., dermatology and skin biopsies), of which they lack confidence or competence. If such future innovations are considered, we suggest that a systematic approach to curricular planning should include projections of cost effectiveness, followed by implementation of a pilot project to confirm these projections and inform steps for dissemination. Finally, we have found the model of educational partnerships between faculty at the pilot center and faculty at local sites to be extremely valuable as it allows for very efficient division of labor in the complex efforts required to present a high-quality CPD event. To our thinking this is the best way to ensure that the consistency of a proven program is being preserved across multiple sites, while at the same time essential adaptations and modifications are being made to ensure that program is effective at the local level.

Further development of this educational initiative will include exploration of additional methods to deliver the same content effectively and efficiently. These additional techniques could include the use of online resources, “virtual” experiences, and simulation techniques. We will also work to extend this curriculum to other levels across the continuum of medical and health professions education. Though this training program was initially developed as a CPD program for physicians and other primary care providers who are already in practice, elements of this curriculum can be adapted in order to address the educational needs of students and trainees in undergraduate and graduate medical education programs.

We acknowledge several challenges and limitations to this initiative. Throughout all stages of this project, a significant concern has been the time that providers needed to be absent from their clinics in order to participate in the mini-residency. Although we condensed the program substantially from our initial course, a 3-day event still has an impact on the widespread problem of access to primary care. Furthermore, training 10–20 providers at the same time may have even intensified the problem of access by removing a large number of clinicians from the same region at the same time, increasing the workload for those providers who remained to staff the clinics. Additionally, we recognize that the 3-day program only laid the foundation for a critically important element of the training—supervised clinical experiences with real patients, and the opportunity to perform the number of joint injections needed to support clinical privileges. These types of activities require a much higher faculty-to-learner ratio than we provided in the portion of the program that we have described and were not possible to accomplish in 3 days with these numbers of participants. We developed new self-assessment instruments for this initiative and have not presented evidence for their validity. In addition, there are several disadvantages of using Likert scales for self-assessment; these provide limited categories for response, are unidimensional, and the distance between each option, even with individual anchors, may not be equidistant. Finally, it is important to note that this study does not address the question of generalizability of this venture to academic and community outpatient centers, in which the significant cost could be prohibitive. Further work is needed to explore the feasibility of further extending this program through a version adapted to non-VA health care systems.

In summary, the MSK “mini-residency” is an effective and well-received mixed-method educational initiative to strengthen the skills of practicing PCPs in evaluating and managing patients with MSK complaints. We have demonstrated that this innovation has been successfully disseminated to multiple centers across the nation in the VA health care system. We now have a greater understanding of the factors and systems needed for a successful program and have developed an effective curriculum to support these efforts. Individual facilities and institutions can employ these methods and resources, adapting them as needed in order to meet the specific educational and training needs of the local setting. The prevalence of MSK disease throughout the US and globally suggests that this is an important priority.