The Psychiatry Milestones: New Developments and Challenges
- First Online:
- Cite this article as:
- Beresin, E.V., Balon, R. & Coverdale, J. Acad Psychiatry (2014) 38: 249. doi:10.1007/s40596-014-0119-6
- 490 Downloads
The special cluster of articles in this issue of Academic Psychiatry [1, 2, 3, 4, 5, 6, 7, 8, 9] on the Psychiatry Milestones introduces and overviews the accomplishments of the Psychiatry Milestones Working Group and allows us to appreciate the trajectory of the Next Accreditation System (NAS) of the Accreditation Council for Graduate Medical Education (ACGME) , as it applies to general psychiatry.
Psychiatric education is evolving. In 1997, the ACGME initiated the Outcome Project for Graduate Medical Education (GME) following the Institute of Medicine’s attention to safety and quality. The Outcome Project began a developmental approach to residency and fellowship training, with emphasis on trainee, and program performance . The ACGME established a competency-based education and required assessment of individual proficiency within each of the six core competencies .
The competency movement began as a developmental, conceptual framework largely grounded on the Dreyfus and Dreyfus model (from novice to beginner, competent, proficient, and expert) [11, 12]. In itself, the competency movement was a landmark in medical education, in its appreciation that knowledge, skills, and attitudes are acquired by physicians developmentally through lifelong learning. In the area of ethics and professionalism, for example, this developmental insight was particularly significant. Instead of assuming that professional “character” was defined before medical training, this model posited that physicians could learn and be taught about ethics and professional expectations [13, 14, 15]. The competency movement thus brought new obligations to both learners and teachers to be accountable for the outcomes of the educational process.
This innovative educational enterprise for GME was nevertheless found insufficient on a number of levels. First, the achievements of individuals in their training and education within all medical and surgical specialties were left to assessment of the programs themselves, and many were not considered valid . Second, although there were specialty-specific requirements and common program requirements of the ACGME, there was no demand for a high degree of specificity of measurable, individual benchmarks for evaluation. Most programs used global assessments, such as 360° evaluations, and the requirements for construct validity and current milestone-like developmental assessment were largely neglected .
In sum, the initial competency movement brought advances in medical education despite concerns and challenges. This movement initiated a revolution in GME and its principles—a developmental model and requirement for evidence-based assessment of outcomes transformed residency training. It was the beginning of a new era in medical education.
In the series of papers that follow in this issue of the journal [1, 2, 3, 4, 5, 6, 7, 8, 9], the authors explain the new Psychiatry Milestones and the ways in which they were developed. We owe a great debt to the Working Group in its attempt to apply input from all the major stakeholders in psychiatric GME and its use of workshops, surveys, and 19 pilot projects to assess the feasibility of the Milestones. Moreover, the Working Group’s emphasis on making the Milestones “meaningful, measurable, and manageable,” preserving the six core competencies, and paying attention to how the competencies may be transformed into Milestones demonstrates the group’s efforts to capture the best of what psychiatric educators are already doing and ensures a degree of continuity from the current system to the NAS.
The Working Group chose to break down the six core competencies into 22 sub-competencies and to construct “threads” that articulate developmental progress in each of the sub-competencies. Many will view these sub-competencies as a genuine strength of the new approach. The 66 threads delineate core, observable benchmarks that articulate a clear developmental process, such that Clinical Competency Committees may capture a developmental snapshot of a resident or fellow at any particular time of training.
Increased specificity with behavioral knowledge, skills, and attitudes anchors that could lend themselves to improved feedback in the moment and overall assessment. Such benchmarks were often lacking in evaluation of the competencies and demonstrate advancement in the evolution of psychiatric GME.
Clear specialty-specific sub-competencies and threads in patient care and medical knowledge.
Alignment with other medical specialties in the “generic” competencies of professionalism, systems-based practice, practice-based learning and improvement, and interpersonal communication skills, with specialty-specific additions in psychiatry, including community-based care and consultation, as central to the identity and practice of psychiatrists. These psychiatry-specific additions will be particularly important within the establishment of medical homes, accountable care organizations, and population management in the future implementation of the Affordable Care Act.
Allowances for flexibility in the assessment process and, in particular, giving considerable decision-making authority to the program director and the clinical competency committee in the summative evaluation of residents and their ability to progress.
Including elements of training that are already in practice but preparing educators for the future, as demonstrated by the new knowledge, skills, and attitudes, and incorporating novel concepts such as “human factors engineering” in systems-based practice, as more social science concepts are included in training.
Providing a systemic framework of knowledge, skills, and attitudes that allows for feedback and assessment by global ratings and, most important, by direct observation, emphasized extensively in all the papers in this cluster of the journal [1, 2, 3, 4, 5, 6, 7, 8, 9]. The authors are to be commended for clearly delineating anchors (through the threads) for formative assessment (feedback) and distinguishing it from summative assessment (evaluation) and the use of both by the clinical competency committees.
Although the NAS and the Milestones support a more sophisticated developmental model for the assessment of the growth of trainees, in general, they are largely idealistic and have paid little attention to practical issues of implementation and GME funding. The leadership of the ACGME indicated that it cannot “compensate for the overuse of resources, inefficiencies, and disparities that characterize aspects of the nation’s health care system”  in response to the “limitations” in rolling out the NAS.
It is our view that no matter how elegant the model and no matter how developmentally sound and specific the new benchmarks, psychiatry and all other medical specialties are faced with new tremendous burdens and unfunded mandates in a time when most academic medical centers and teaching hospitals are in serious financial hardship.
Infrastructure: The NAS requires increasing direct observation and reporting individual and program achievements every 6 months. At this point, however, there is no uniform information technology system for documenting observations. The evaluation systems we currently use, such as New Innovations, e-Value, and the ACGME Web Ads system, are disconnected and user-unfriendly. In order for on-the-spot and biannual reports to be generated, a single electronic platform that can collect, collate, and store data would be immensely valuable. Hospital electronic medical records are a great example of how systems can be built to align multiple reporting agencies. The lack of a shared infrastructure will make implementation of the Milestones especially challenging.
Faculty development: The current developments in medical education, including the Milestones Project, underscore the need for enhanced professional development activities at every academic institution nationally. The Psychiatry Milestones are relatively simple in comparison to those of some other specialties. However, 6 competencies with 22 sub-competencies and 66 developmental threads organized in a 5-tiered system are hardly comprehensible for the seasoned program director, much less a faculty member who is not well versed in the new educational literature. The time, energy, and funding of faculty development is implicitly “assigned” to the academic system and teaching hospital, at a time when physician reimbursements are likely to drop and clinical expectations of faculty members are escalating. Moreover, some faculty members will require additional training in evidence-based psychiatry and rapidly advancing or hard-to-learn topic areas, including the neurosciences, in order to evaluate residents’ competencies validly in those same areas.
Training vs. service dynamic: In the current ACGME resident survey, a question is asked about whether service demands interfere with the process of learning. We have long worried about the potential conflict that an institution’s clinical demands may be at odds with the educational needs of residents. We know that our best teachers are our patients, and there is no better learning experience than caring for patients. Yet, in upcoming years, as the reimbursements decrease for services, many faculty will be burdened as they are now with increased service demands. This could lead to clinical care that is more geared to the needs of the system and less faculty oversight than is appropriate for education. The Milestones, for example, require knowledge, skills, and attitudes in a range of psychotherapies. How may the psychotherapy developmental threads be accomplished when psychiatrists/residents are providing mostly pharmacotherapy while our colleagues in social work and psychology provide the psychotherapy for resident/psychiatrist patients? In many teaching institutions now, there is increasing pressure for psychiatric trainees to perform diagnostic evaluations and medication management at the expense of psychosocial treatments. This pressure is not likely to diminish under healthcare reform. Yet, psychiatrists are ultimately responsible for a particular patient care, are to understand what is going on in therapy of their patient with social worker/psychologist colleague, and are to supervise the case.
New assessment instruments: More sophisticated and more highly specific measures of assessment are necessary to ensure the success of the Milestones movement. Some instruments may well be produced at a national level, for example, by the Working Group or a designated subcommittee, but it is clear that much of the work will need to be developed in the local ecology of individual academic programs. Faculty will need to learn how to use these new assessment measures and to attain appropriate levels of inter-rater reliability. For instance, Swing et al.  have Milestones designated as threads as “met, not met or partially met,” but the definitions of these terms remain unclear at this time.
Cultural shift in grading and remediation: As we move from “grades,” such as excellent, satisfactory, unsatisfactory, or even letter grades, to subtler means of meeting benchmarks, residents, clearly used to being evaluated categorically, will now be measured in developmental terms. Growth will be viewed as a process, not as a static result of a rotation, for example, and grades will be replaced by clearly observable benchmarks of performance. For some residents, this may be felt as a sort of demotion. After all, we have all grown up in a culture of grades and grade inflation. However, we consider this to be an advance in the field of graduate medical education. In the new Milestones era, there will be need for clear knowledge, skills, and attitudes that always require either “improvement” or some level of remediation. As Wilk and Schultz  note, this new approach may be hard for trainees to accept, particularly if the length of residency training is at stake.
The Psychiatry Milestones remain untested in practice but are based on good ideas and good work. As educators, we will need to modify and refine the Milestones as they are implemented. The Milestones and the knowledge, attitudes, and skills that we aim to formally evaluate should reflect a very thoughtful and well-defined approach to our goals and objectives for teaching. Similarly, we will need to evaluate our ability to efficiently document residents’ knowledge, attitudes, and skills, and to manage the wide range of complex data that will result from this initiative. We will also need to learn more about residents’ and faculty members’ satisfaction with implementing the Milestones and modify the assessments according to this feedback.
Solutions for Consideration
The Milestones are indeed an advance in GME and in medical education in general. Given some of the challenges noted above—all of which represent major unfunded mandates—what may the field consider as possible solutions?
If we are considering medical education as a true developmental continuum from medical school through residency, fellowship, and postgraduate, lifelong education, we need common systems and common funding to teach, assess, and improve our practitioners. Currently, the educational and credentialing system is an array of silos that function truly or nearly completely independently. There is limited connection between academic and hospital systems and credentialing bodies. Yet, all share the common goal of sound education and a social contract to ensure the highest quality of professional care of patients and the advancement of knowledge.
The cornerstone for solving many of the challenges is the need for national collaboration—pooling of resources, eliminating the silos, conserving the workforce, and improving efficiency while maintaining quality and overseeing the developmental process of practitioners. One example of this may be seen if we use the very same maintenance of certification examinations for Board re-certification for hospital credentialing and faculty development in teaching Milestones. Surely this “triple dipping” might be feasible in a number of areas of professional assessment and will have the benefit of satisfying multiple requirements. In addition, it would help save costs for our institutions. But to accomplish such a task, the credentialing agencies would have to collaborate in sharing goals and objectives and in test creation.
A current example of breaking down silos is the recent national uniform accreditation system for osteopathic programs in the USA. In February 2014, the ACGME, the American Osteopathic Association (AOA), and the American Association of Colleges of Osteopathic Medicine (AACOM) agreed on a single, national GME accreditation system for AACOM graduates. In an ACGME press release, Thomas Nasca noted, “The commitment to a single accreditation system comes at a watershed moment for medical education in the US… A single system standardizes the approach to GME accreditation, and ensures that all physicians have access to the primary and sub-specialty training necessary to serve patients” . Although this national unified process only pertains to GME and not for a consortium through the continuum of medical education, it is clearly a small step in the right direction.
This move toward alignment may seem revolutionary, yet is not. We are facing a fairly radical transformation in our healthcare delivery and educational systems, and our professional and social requirements for quality and improvement may well demand a radical shift in our current method of training and education.
Nevertheless, we should not let the many challenges for implementation obscure the educational advances in the Milestones framework. We applaud the diligence and clarity of the Psychiatry Work Group. This initial product is a remarkable achievement, and of necessity, will require a slow, deliberate process to help bring it to fruition in our daily work.
The authors thank Dr. Laura Weiss Roberts for her review of earlier drafts of this editorial and invaluable comments.
On behalf of all the authors, the corresponding author states that there are no conflicts of interest.