Knowledge Retention after an Online Tutorial: A Randomized Educational Experiment among Resident Physicians
The time course of physicians’ knowledge retention after learning activities has not been well characterized. Understanding the time course of retention is critical to optimizing the reinforcement of knowledge.
Educational follow-up experiment with knowledge retention measured at 1 of 6 randomly assigned time intervals (0–55 days) after an online tutorial covering 2 American Diabetes Association guidelines.
Internal and family medicine residents.
Multiple-choice knowledge tests, subject characteristics including critical appraisal skills, and learner satisfaction.
Of 197 residents invited, 91 (46%) completed the tutorial and were randomized; of these, 87 (96%) provided complete follow-up data. Ninety-two percent of the subjects rated the tutorial as “very good” or “excellent.” Mean knowledge scores increased from 50% before the tutorial to 76% among those tested immediately afterward. Score gains were only half as great at 3–8 days and no significant retention was measurable at 55 days. The shape of the retention curve corresponded with a 1/4-power transformation of the delay interval. In multivariate analyses, critical appraisal skills and participant age were associated with greater initial learning, but no participant characteristic significantly modified the rate of decline in retention.
Education that appears successful from immediate posttests and learner evaluations can result in knowledge that is mostly lost to recall over the ensuing days and weeks. To achieve longer-term retention, physicians should review or otherwise reinforce new learning after as little as 1 week.
KEY WORDSknowledge retention online tutorial randomized educational experiment resident physicians educational technology learning theory
- 1.Davis D, O’Brien MA, Freemantle N, Wolf FM, Mazmanian P,Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. 1999;282(9):867–74.PubMedCrossRefGoogle Scholar
- 5.Marinopoulos SS, Dorman T, Ratanawongsa N, Wilson LM, Ashar BH, Magaziner JL, et al. Effectiveness of Continuing Medical Education. Rockville, MD: Agency for Healthcare Research and Quality; 2007. Evidence Report AHRQ Publication No. 07-E006, January.Google Scholar
- 6.Bjork EL, Bjork RA. On the adaptive aspects of retrieval failure in autobiographical memory. In: Grueneberg MM, Morris PE, Sykes RN, eds. Practical Aspects of Memory: Current Research and Issues: Vol 1 Memory in Everyday Life. New York: Wiley; 1988.Google Scholar
- 7.Bjork RA, Bjork EL. A new theory of disuse and an old theory of stimulus fluctuation. In: Healy A, Kosslyn S, Shiffrin R, eds. From Learning Processes to Cognitive Processes: Essays in honor of William K Estes. Hillsdale, NJ: Lawrence; 1992.Google Scholar
- 9.Anderson JR, editor. Cognitive psychology and its implications. New York: Freeman; 1995.Google Scholar
- 17.Accreditation Council for Continuing Medical Education. ACCME Annual Report Data 2004; 2005.Google Scholar
- 23.Kern DE, Thomas PA, Howard DM, Bass EB. Goals and Objectives. Curriculum Development for Medical Education: A Six Step Approach. Baltimore: Johns Hopkins University Press; 1998.Google Scholar
- 28.Bjork RA. Memory and metamemory considerations in the training of human beings. In: Metcalfe J, Shimamura A, eds. Metacognition: Knowing about Knowing. Cambridge, MA: MIT; 1994:185–205.Google Scholar
- 34.Institute of Medicine Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.Google Scholar