Authors have claimed that Internet-based instruction promotes greater learning efficiency than non-computer methods. Objectives Determine, through a systematic synthesis of evidence in health professions education, how Internet-based instruction compares with non-computer instruction in time spent learning, and what features of Internet-based instruction are associated with improved learning efficiency. Data sources We searched databases including MEDLINE, CINAHL, EMBASE, and ERIC from 1990 through November 2008. Study selection and data abstraction We included all studies quantifying learning time for Internet-based instruction for health professionals, compared with other instruction. Reviewers worked independently, in duplicate, to abstract information on interventions, outcomes, and study design. Results We identified 20 eligible studies. Random effects meta-analysis of 8 studies comparing Internet-based with non-Internet instruction (positive numbers indicating Internet longer) revealed pooled effect size (ES) for time −0.10 (p = 0.63). Among comparisons of two Internet-based interventions, providing feedback adds time (ES 0.67, p = 0.003, two studies), and greater interactivity generally takes longer (ES 0.25, p = 0.089, five studies). One study demonstrated that adapting to learner prior knowledge saves time without significantly affecting knowledge scores. Other studies revealed that audio narration, video clips, interactive models, and animations increase learning time but also facilitate higher knowledge and/or satisfaction. Across all studies, time correlated positively with knowledge outcomes (r = 0.53, p = 0.021). Conclusions On average, Internet-based instruction and non-computer instruction require similar time. Instructional strategies to enhance feedback and interactivity typically prolong learning time, but in many cases also enhance learning outcomes. Isolated examples suggest potential for improving efficiency in Internet-based instruction.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Bell, D. S., Fonarow, G. C., Hays, R. D., & Mangione, C. M. (2000). Self-study from web-based and printed guideline materials: A randomized, controlled trial among resident physicians. Annals of Internal Medicine, 132, 938–946.
Berman, N. B., Fall, L. H., Maloney, C. G., & Levine, D. A. (2008). Computer-assisted instruction in clinical education: A roadmap to increasing CAI implementation. Advances in Health Sciences Education, 13, 373–383.
Bernard, R. M., Abrami, P. C., Borokhovski, E., Wade, C. A., Tamim, R. M., Surkes, M. A., et al. (2009). A meta-analysis of three types of interaction treatments in distance education. Review of Educational Research, 79, 1243–1289.
Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., et al. (2004). How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Review of Educational Research, 74, 379–439.
Blackmore, C., Tantam, D., & Van Deurzen, E. (2006). The role of the eTutor—evaluating tutor input in a virtual learning community for psychotherapists and psychologists across Europe. International Journal of Psychotherapy, 10(2), 35–46.
Chumley-Jones, H. S., Dobbie, A., & Alford, C. L. (2002). Web-based learning: Sound educational method or hype? A review of the evaluation literature. Academic Medicine, 77(10 suppl), S86–S93.
Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53, 445–459.
Clark, D. (2002). Psychological myths in e-learning. Medical Teacher, 24(6), 598–604.
Clark, R. C., & Mayer, R. E. (2007). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning (2nd ed.). San Francisco: Pfeiffer.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
Cohen, P. A., & Dacanay, L. D. (1992). Computer-based instruction and health professions education: A meta-analysis of outcomes. Evaluation and the Health Professions, 15, 259–281.
Cohen, P. A., & Dacanay, L. D. (1994). A meta-analysis of computer-based instruction in nursing education. Computers in Nursing, 12(2), 89–97.
Cook, D. A. (2005). The research we still are not doing: An agenda for the study of computer-based learning. Academic Medicine, 80, 541–548.
Cook, D. A. (2009). The failure of e-learning research to inform educational practice, and what we can do about it. Medical Teacher, 31, 158–162.
Cook, D. A., Beckman, T. J., Thomas, K. G., & Thompson, W. G. (2008a). Adapting Web-based instruction to residents’ knowledge improves learning efficiency: A randomized controlled trial. Journal of General Internal Medicine, 23, 985–990.
Cook, D. A., Beckman, T. J., Thomas, K. G., & Thompson, W. G. (2008b). Introducing resident physicians to complexity in ambulatory medicine. Medical Education, 42, 838–848.
Cook, D. A., & Dupras, D. M. (2004). A practical guide to developing effective web-based learning. Journal of General Internal Medicine, 19, 698–707.
Cook, D. A., Dupras, D. M., Thompson, W. G., & Pankratz, V. S. (2005). Web-based learning in residents’ continuity clinics: A randomized, controlled trial. Academic Medicine, 80, 90–97.
Cook, D. A., Levinson, A. J., Garside, S., Dupras, D. M., Erwin, P. J., & Montori, V. M. (2008c). Internet-based learning in the health professions: A meta-analysis. JAMA, 300, 1181–1196.
Cook, D. A., Levinson, A. J., Garside, S., Dupras, D. M., Erwin, P. J., & Montori, V. M. (2010). Instructional design variations in internet-based learning for health professions education: A systematic review and meta-analysis. Academic Medicine, 85, 909–922.
Cook, D. A., Thompson, W. G., & Thomas, K. G. (2009a). Case-based or non-case-based questions for teaching postgraduate physicians: A randomized crossover trial. Academic Medicine, 84, 1419–1425.
Cook, D. A., Thompson, W. G., Thomas, K. G., & Thomas, M. R. (2009b). Lack of interaction between sensing–intuitive learning styles and problem-first versus information-first instruction: A randomized crossover trial. Advances in Health Sciences Education, 14, 79–90.
Cook, D. A., Thompson, W. G., Thomas, K. G., Thomas, M. R., & Pankratz, V. S. (2006). Impact of self-assessment questions and learning styles in web-based learning: A randomized, controlled, crossover trial. Academic Medicine, 81, 231–238.
Cooper, E. H., & Pantle, A. J. (1967). The total-time hypothesis in verbal learning. Psychological Bulletin, 68, 221–234.
Craik, F. I. M., & Tulving, E. (1975). Depth of processing and the retention of words in episodic memory. Journal ol Experimental Psychology: General, 104, 268–294.
Cronbach, L. J., & Snow, R. E. (1977). Aptitudes and instructional methods: A handbook for research on interactions. New York: Irvington Publishers.
Curtin, F., Altman, D. G., & Elbourne, D. (2002). Meta-analysis combining parallel and cross-over clinical trials. I: Continuous outcomes. Statistics in Medicine, 21, 2131–2144.
Dennis, J. K. (2003). Problem-based learning in online vs. face-to-face environments. Education for Health, 16(2), 198–209.
Effective Use of Educational Technology in Medical Education: Summary Report of the 2006 AAMC Colloquium on Educational Technology. (2007). Washington, DC: Association of American Medical Colleges.
Fletcher, J. D. (2009). Education and training technology in the military. Science, 323, 72–75.
Fordis, M., King, J. E., Ballantyne, C. M., Jones, P. H., Schneider, K. H., Spann, S. J., et al. (2005). Comparison of the instructional efficacy of internet-based CME with live interactive CME workshops: A randomized controlled trial. JAMA, 294, 1043–1051.
Fredrick, W. C., & Walberg, H. J. (1980). Learning as a function of time. Journal of Educational Research, 73, 183–194.
Friedl, R., Hoppler, H., Ecard, K., Scholz, W., Hannekum, A., Ochsner, W., et al. (2006a). Multimedia-driven teaching significantly improves students’ performance when compared with a print medium. Annals of Thoracic Surgery, 81, 1760–1766.
Friedl, R., Hoppler, H., Ecard, K., Scholz, W., Hannekum, A., Oechsner, W., et al. (2006b). Comparative evaluation of multimedia driven, interactive, and case-based teaching in heart surgery. Annals of Thoracic Surgery, 82, 1790–1795.
Gijselaers, W. H., & Schmidt, H. G. (1995). Effects of quantity of instruction on time spent on learning and achievement. Educational Research and Evaluation, 1(2), 183–201.
Goitein, L., Shanafelt, T. D., Wipf, J. E., Slatore, C. G., & Back, A. L. (2005). The effects of work-hour limitations on resident well-being, patient care, and education in an internal medicine residency program. Archives of Internal Medicine, 165, 2601–2606.
Greenhalgh, T. (2001). Computer assisted learning in undergraduate medical education. BMJ, 322, 40–44.
Grundman, J., Wigton, R., & Nickol, D. (2000). A controlled trial of an interactive, web-based virtual reality program for teaching physical diagnosis skills to medical students. Academic Medicine, 75(10 Suppl), S47–S49.
Higgins, J. P. T., & Green, S. (2008). Cochrane handbook for systematic reviews of interventions 5.0.1 [updated September 2008]. Retrieved May 11, 2009, from http://www.cochrane.org/resources/handbook/index.htm.
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. BMJ, 327, 557–560.
Kopp, V., Stark, R., & Fischer, M. R. (2008). Fostering diagnostic knowledge through computer-supported, case-based worked examples: Effects of erroneous examples and feedback. Medical Education, 42, 823–829.
Kulik, C.-L. C., Kulik, J. A., & Shwalb, B. J. (1986). The effectiveness of computer-based adult education: A meta-analysis. Journal of Educational Computing Research, 2, 235–252.
Larreamendy-Joerns, J., & Leinhardt, G. (2006). Going the distance with online education. Review of Educational Research, 76(4), 567–605.
Leong, S. L., Baldwin, C. D., & Adelman, A. M. (2003). Integrating web-based computer cases into a required clerkship: Development and evaluation. Academic Medicine, 78, 295–301.
Lewis, M. J., Davies, R., Jenkins, D., & Tait, M. I. (2001). A review of evaluative studies of computer-based learning in nursing education. Nurse Education Today, 21, 26–37.
Mattheos, N., Nattestad, A., Christersson, C., Jansson, H., & Attstrom, R. (2004). The effects of an interactive software application on the self-assessment ability of dental students. European Journal of Dental Education, 8(3), 97–104.
McKimm, J., Jollie, C., & Cantillon, P. (2003). ABC of learning and teaching in medicine: Web based learning. BMJ, 326(7394), 870–873.
Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999). Improving the quality of reports of meta-analyses of randomised controlled trials: The QUOROM statement. Quality of reporting of meta-analyses. Lancet, 354, 1896–1900.
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA Statement. Annals of Internal Medicine, 151, 264–269.
Nicholson, D. T., Chalk, C., Funnell, W. R. J., & Daniel, S. J. (2006). Can virtual reality improve anatomy education? A randomised controlled study of a computer-generated three-dimensional anatomical ear model. Medical Education, 40(1), 1081–1087.
Papa, F. J., Aldrich, D., & Schumacker, R. E. (1999). The effects of immediate online feedback upon diagnostic performance. Academic Medicine, 74(10 suppl), S16–S18.
Papa, F. J., Oglesby, M. W., Aldrich, D. G., Schaller, F., & Cipher, D. J. (2007). Improving diagnostic capabilities of medical students via application of cognitive sciences-derived learning principles. Medical Education, 41, 419–425.
Rosenthal, R. (1994). Parametric measures of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 231–244). New York: Russell Sage Foundation.
Ruiz, J. G., Mintzer, M. J., & Leipzig, R. M. (2006). The impact of e-learning in medical education. Academic Medicine, 81, 207–212.
Ryan, J. (2005). Unintended consequences: The accreditation council for graduate medical education work-hour rules in practice. Annals of Internal Medicine, 143, 82–83.
Schittek Janda, M., Tani Botticelli, A., Mattheos, N., Nebel, D., Wagner, A., Nattestad, A., et al. (2005). Computer-mediated instructional video: A randomised controlled trial comparing a sequential and a segmented instructional video in surgical hand wash. European Journal of Dental Education, 9(2), 53–58.
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.
Spickard, A. III, Alrajeh, N., Cordray, D., & Gigante, J. (2002). Learning about screening using an online or live lecture: Does it matter? Journal of General Internal Medicine, 17, 540–545.
Spickard, A. III., Smithers, J., Cordray, D., Gigante, J., & Wofford, J. L. (2004). A randomised trial of an online lecture with and without audio. Medical Education, 38, 787–790.
Stroup, D. F., Berlin, J. A., Morton, S. C., Olkin, I., Williamson, G. D., Rennie, D., et al. (2000). Meta-analysis of observational studies in epidemiology: A proposal for reporting. JAMA, 283, 2008–2012.
Tallent-Runnels, M. K., Thomas, J. A., Lan, W. Y., Cooper, S., Ahern, T. C., Shaw, S. M., et al. (2006). Teaching courses online: A review of the research. Review of Educational Research, 76(1), 93–135.
Tunuguntla, R., Rodriguez, O., Ruiz, J. G., Qadri, S. S., Mintzer, M. J., Van Zuilen, M. H., et al. (2008). Computer-based animations and static graphics as medical student aids in learning home safety assessment: A randomized controlled trial. Medical Teacher, 30, 815–817.
Wells, G. A., Shea, B., O’Connell, D., Peterson, J., Welch, V., Losos, M., et al. (2007). The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Retrieved March 20, 2010, from http://www.ohri.ca/programs/clinical_epidemiology/oxford.htm.
Wutoh, R., Boren, S. A., & Balas, E. A. (2004). eLearning: A review of internet-based continuing medical education. Journal of Continuing Education in the Health Professions, 24, 20–30.
The authors thank Denise M. Dupras, MD, PhD, Patricia J. Erwin, MLS, and Victor M. Montori, MD, MSc for their role in study identification and data abstraction. This work was supported by intramural funds and a Mayo Foundation Education Innovation award. Dr. Levinson is supported in part through the John R. Evans Chair in Health Sciences Education Research. Dr. Cook had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
About this article
Cite this article
Cook, D.A., Levinson, A.J. & Garside, S. Time and learning efficiency in Internet-based learning: a systematic review and meta-analysis. Adv in Health Sci Educ 15, 755–770 (2010). https://doi.org/10.1007/s10459-010-9231-x
- Educational technology
- Education, professional
- Computer-assisted instruction
- Distance education
- Learning efficiency