Knowledge Retention after an Online Tutorial: A Randomized Educational Experiment among Resident Physicians
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.Get Access
The time course of physicians’ knowledge retention after learning activities has not been well characterized. Understanding the time course of retention is critical to optimizing the reinforcement of knowledge.
Educational follow-up experiment with knowledge retention measured at 1 of 6 randomly assigned time intervals (0–55 days) after an online tutorial covering 2 American Diabetes Association guidelines.
Internal and family medicine residents.
Multiple-choice knowledge tests, subject characteristics including critical appraisal skills, and learner satisfaction.
Of 197 residents invited, 91 (46%) completed the tutorial and were randomized; of these, 87 (96%) provided complete follow-up data. Ninety-two percent of the subjects rated the tutorial as “very good” or “excellent.” Mean knowledge scores increased from 50% before the tutorial to 76% among those tested immediately afterward. Score gains were only half as great at 3–8 days and no significant retention was measurable at 55 days. The shape of the retention curve corresponded with a 1/4-power transformation of the delay interval. In multivariate analyses, critical appraisal skills and participant age were associated with greater initial learning, but no participant characteristic significantly modified the rate of decline in retention.
Education that appears successful from immediate posttests and learner evaluations can result in knowledge that is mostly lost to recall over the ensuing days and weeks. To achieve longer-term retention, physicians should review or otherwise reinforce new learning after as little as 1 week.
- Davis D, O’Brien MA, Freemantle N, Wolf FM, Mazmanian P,Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. 1999;282(9):867–74. CrossRef
- Choudhry NK, Fletcher RH, Soumerai SB. Systematic review: the relationship between clinical experience and quality of health care. Ann Intern Med. 2005;142(4):260–73.
- Eisenberg JM. An educational program to modify laboratory use by house staff. J Med Educ. 1977;52(7):578–81.
- Evans CE, Haynes RB, Birkett NJ, Gilbert JR, Taylor DW, Sackett DL, et al. Does a mailed continuing education program improve physician performance? Results of a randomized trial in antihypertensive care. JAMA. 1986;255(4):501–4. CrossRef
- Marinopoulos SS, Dorman T, Ratanawongsa N, Wilson LM, Ashar BH, Magaziner JL, et al. Effectiveness of Continuing Medical Education. Rockville, MD: Agency for Healthcare Research and Quality; 2007. Evidence Report AHRQ Publication No. 07-E006, January.
- Bjork EL, Bjork RA. On the adaptive aspects of retrieval failure in autobiographical memory. In: Grueneberg MM, Morris PE, Sykes RN, eds. Practical Aspects of Memory: Current Research and Issues: Vol 1 Memory in Everyday Life. New York: Wiley; 1988.
- Bjork RA, Bjork EL. A new theory of disuse and an old theory of stimulus fluctuation. In: Healy A, Kosslyn S, Shiffrin R, eds. From Learning Processes to Cognitive Processes: Essays in honor of William K Estes. Hillsdale, NJ: Lawrence; 1992.
- Schmidt RA, Bjork RA. New conceptualizations of practice: common principles in three paradigms suggest new concepts for training. Psychol Sci. 1992;3(4):207–17. CrossRef
- Anderson JR, editor. Cognitive psychology and its implications. New York: Freeman; 1995.
- Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977;84(2):191–215. CrossRef
- Chumley-Jones HS, Dobbie A, Alford CL. Web-based learning: sound educational method or hype? A review of the evaluation literature. Acad Med. 2002;77(10 Suppl):S86–93. CrossRef
- Cook DA, Dupras DM. A practical guide to developing effective web-based learning. J Gen Intern Med. 2004;19(6):698–707. CrossRef
- Embi PJ, Bowen JL, Singer E. A Web-based curriculum to improve residents’ education in outpatient medicine. Acad Med. 2001;76(5):545. CrossRef
- Sisson SD, Hughes MT, Levine D, Brancati FL. Effect of an internet-based curriculum on postgraduate education. A multicenter intervention. J Gen Intern Med. 2004;19(5 Pt 2):505–9. CrossRef
- Cook DA, Dupras DM, Thompson WG, Pankratz VS. Web-based learning in residents’ continuity clinics: a randomized, controlled trial. Acad Med. 2005;80(1):90–7. CrossRef
- Bennett NL, Casebeer LL, Kristofco RE, Strasser SM. Physicians’ internet information-seeking behaviors. J Contin Educ Health Prof. 2004;24(1):31–8. CrossRef
- Accreditation Council for Continuing Medical Education. ACCME Annual Report Data 2004; 2005.
- van Braak JP, Goeman K. Differences between general computer attitudes and perceived computer attributes: development and validation of a scale. Psychol Rep. 2003;92(2):655–60. CrossRef
- Chin MH, Cook S, Jin L, Drum ML, Harrison JF, Koppert J, et al. Barriers to providing diabetes care in community health centers. Diabetes Care. 2001;24(2):268–74. CrossRef
- Cabana MD, Rand C, Slish K, Nan B, Davis MM, Clark N. Pediatrician self-efficacy for counseling parents of asthmatic children to quit smoking. Pediatrics. 2004;113(1 Pt 1):78–81. CrossRef
- Arauz-Pacheco C, Parrott MA, Raskin P. Hypertension management in adults with diabetes. Diabetes Care. 2004;27(Suppl 1):S65–7.
- Haffner SM. Dyslipidemia management in adults with diabetes. Diabetes Care. 2004;27(Suppl 1):S68–71.
- Kern DE, Thomas PA, Howard DM, Bass EB. Goals and Objectives. Curriculum Development for Medical Education: A Six Step Approach. Baltimore: Johns Hopkins University Press; 1998.
- Bell DS, Fonarow GC, Hays RD, Mangione CM. Self-study from web-based and printed guideline materials. A randomized, controlled trial among resident physicians. Ann Intern Med. 2000;132(12):938–46.
- Matts JP, Lachin JM. Properties of permuted-block randomization in clinical trials. Control Clin Trials. 1988;9(4):327–44. CrossRef
- Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297. CrossRef
- Rubin DC, Wenzel AE. One hundred years of forgetting: a quantitative description of retention. Psychol Rev. 1996;103(4):734–60. CrossRef
- Bjork RA. Memory and metamemory considerations in the training of human beings. In: Metcalfe J, Shimamura A, eds. Metacognition: Knowing about Knowing. Cambridge, MA: MIT; 1994:185–205.
- Chung S, Mandl KD, Shannon M, Fleisher GR. Efficacy of an educational Web site for educating physicians about bioterrorism. Acad Emerg Med. 2004;11(2):143–8.
- Beal T, Kemper KJ, Gardiner P, Woods C. Long-term impact of four different strategies for delivering an on-line curriculum about herbs and other dietary supplements. BMC Med Educ. 2006;6:39. CrossRef
- Fordis M, King JE, Ballantyne CM, Jones PH, Schneider KH, Spann SJ, et al. Comparison of the instructional efficacy of internet-based CME with live interactive CME workshops: a randomized controlled trial. JAMA. 2005;294(9):1043–51. CrossRef
- Gais S, Born J. Declarative memory consolidation: mechanisms acting during human sleep. Learn Mem. 2004;11(6):679–85. CrossRef
- Chen FM, Bauchner H, Burstin H. A call for outcomes research in medical education. Acad Med. 2004;79(10):955–60. CrossRef
- Institute of Medicine Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.
- Knowledge Retention after an Online Tutorial: A Randomized Educational Experiment among Resident Physicians
Journal of General Internal Medicine
Volume 23, Issue 8 , pp 1164-1171
- Cover Date
- Print ISSN
- Online ISSN
- Additional Links
- knowledge retention
- online tutorial
- randomized educational experiment
- resident physicians
- educational technology
- learning theory
- Industry Sectors
- Author Affiliations
- 1. Division of General Internal Medicine and Health Services Research, Department of Medicine, David Geffen School of Medicine at UCLA, Los Angeles, CA, USA
- 2. Department of Psychology, University of California, Los Angeles, CA, USA
- 3. Department of Family Medicine, Charles R. Drew University, Los Angeles, CA, USA
- 4. Department of Health Services, UCLA School of Public Health, Los Angeles, CA, USA