Abstract
BACKGROUND
The ACCME is phasing in new criteria for accreditation from 2008 to 2012. These criteria require CME providers to assess the impact of their interventions.
OBJECTIVES
To assess the feasibility of measuring outcomes at a national meeting, the SGIM evaluation committee conducted a pilot assessment of two workshops and one precourse.
DESIGN AND PARTICIPANTS
Session coordinators prepared a five-item questionnaire to assess the knowledge and confidence of participants. The questionnaire was administered pre, immediately post, and 9 months after the educational sessions.
MEASUREMENTS
Changes in performance were calculated as a standardized difference, or effect size.
RESULTS
All three sessions demonstrated initial knowledge acquisition with effect sizes ranging from 0.39 (small) to 0.99 (large) immediately after the sessions. One session demonstrated sustainment of knowledge over the subsequent 9 months while the other two demonstrated decay. Confidence levels decreased following one of the sessions with an effect size of −0.72 (modest effect).
CONCLUSIONS
Effect size measurement of sessions provides quantitative information about their impact on learning and is one way to achieve ACCME compliance. The method, however, poses methodological and logistical challenges that raise questions about the feasibility of tracking learning and retention following a national meeting.
This is a preview of subscription content, log in via an institution to check access.
References
See “CME as a Bridge to Quality: Updated Accreditation Criteria,” Sept 2006, available at http://www.cmanet.org/upload/CME%20Expectation%20Criteria.pdf. Last accessed August 2008
Regnier K, Kopelow M, Lane D, Alden E. Accreditation for learning and change: quality and improvement as the outcome. J Contin Educ Health Prof. 2005;25(3):174–182.
Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Med Care. 1989;27:S178–S189.
Kopelow M. Letter from the CEO of ACCME. Accreditation for Learning and Changes. http://www.accme.org/dir_docs/whats_new/aa64ed0e-fbbc-44fd-abad-550a6da7edef_uploadfile.pdf. Last accessed August 2008.
Moore DE. A framework for outcomes evaluation in the continuing professional development of physicians. In: Davis D, Barnes BE, Fox R, eds. The Continuing Professional Development of Physicians: From Research to Practice. Chicago, IL: AMA Press; 2003:249–74.
Acknowledgement
This research is supported by the Veterans Administration HSR&D Service, Center for Management of Complex Chronic Care, Center of Excellence, Illinois.
Conflict of Interest Statement
None disclosed.
Author information
Authors and Affiliations
Corresponding author
Additional information
The opinions or assertions contained herein are the private views of the author and should not be construed as official or as necessarily reflecting the views of the United States Army or the Department of Defense.
Rights and permissions
About this article
Cite this article
Weiner, S.J., Jackson, J.L. & Garten, S. Measuring Continuing Medical Education Outcomes: A Pilot Study of Effect Size of Three CME Interventions at an SGIM Annual Meeting. J GEN INTERN MED 24, 626–629 (2009). https://doi.org/10.1007/s11606-009-0902-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-009-0902-3