Time and learning efficiency in Internet-based learning: a systematic review and meta-analysis

Abstract

Authors have claimed that Internet-based instruction promotes greater learning efficiency than non-computer methods. Objectives Determine, through a systematic synthesis of evidence in health professions education, how Internet-based instruction compares with non-computer instruction in time spent learning, and what features of Internet-based instruction are associated with improved learning efficiency. Data sources We searched databases including MEDLINE, CINAHL, EMBASE, and ERIC from 1990 through November 2008. Study selection and data abstraction We included all studies quantifying learning time for Internet-based instruction for health professionals, compared with other instruction. Reviewers worked independently, in duplicate, to abstract information on interventions, outcomes, and study design. Results We identified 20 eligible studies. Random effects meta-analysis of 8 studies comparing Internet-based with non-Internet instruction (positive numbers indicating Internet longer) revealed pooled effect size (ES) for time −0.10 (p = 0.63). Among comparisons of two Internet-based interventions, providing feedback adds time (ES 0.67, p = 0.003, two studies), and greater interactivity generally takes longer (ES 0.25, p = 0.089, five studies). One study demonstrated that adapting to learner prior knowledge saves time without significantly affecting knowledge scores. Other studies revealed that audio narration, video clips, interactive models, and animations increase learning time but also facilitate higher knowledge and/or satisfaction. Across all studies, time correlated positively with knowledge outcomes (r = 0.53, p = 0.021). Conclusions On average, Internet-based instruction and non-computer instruction require similar time. Instructional strategies to enhance feedback and interactivity typically prolong learning time, but in many cases also enhance learning outcomes. Isolated examples suggest potential for improving efficiency in Internet-based instruction.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

References

  1. Bell, D. S., Fonarow, G. C., Hays, R. D., & Mangione, C. M. (2000). Self-study from web-based and printed guideline materials: A randomized, controlled trial among resident physicians. Annals of Internal Medicine, 132, 938–946.

    Google Scholar 

  2. Berman, N. B., Fall, L. H., Maloney, C. G., & Levine, D. A. (2008). Computer-assisted instruction in clinical education: A roadmap to increasing CAI implementation. Advances in Health Sciences Education, 13, 373–383.

    Article  Google Scholar 

  3. Bernard, R. M., Abrami, P. C., Borokhovski, E., Wade, C. A., Tamim, R. M., Surkes, M. A., et al. (2009). A meta-analysis of three types of interaction treatments in distance education. Review of Educational Research, 79, 1243–1289.

    Article  Google Scholar 

  4. Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., et al. (2004). How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Review of Educational Research, 74, 379–439.

    Article  Google Scholar 

  5. Blackmore, C., Tantam, D., & Van Deurzen, E. (2006). The role of the eTutor—evaluating tutor input in a virtual learning community for psychotherapists and psychologists across Europe. International Journal of Psychotherapy, 10(2), 35–46.

    Google Scholar 

  6. Chumley-Jones, H. S., Dobbie, A., & Alford, C. L. (2002). Web-based learning: Sound educational method or hype? A review of the evaluation literature. Academic Medicine, 77(10 suppl), S86–S93.

    Article  Google Scholar 

  7. Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53, 445–459.

    Google Scholar 

  8. Clark, D. (2002). Psychological myths in e-learning. Medical Teacher, 24(6), 598–604.

    Article  Google Scholar 

  9. Clark, R. C., & Mayer, R. E. (2007). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning (2nd ed.). San Francisco: Pfeiffer.

    Google Scholar 

  10. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  11. Cohen, P. A., & Dacanay, L. D. (1992). Computer-based instruction and health professions education: A meta-analysis of outcomes. Evaluation and the Health Professions, 15, 259–281.

    Article  Google Scholar 

  12. Cohen, P. A., & Dacanay, L. D. (1994). A meta-analysis of computer-based instruction in nursing education. Computers in Nursing, 12(2), 89–97.

    Google Scholar 

  13. Cook, D. A. (2005). The research we still are not doing: An agenda for the study of computer-based learning. Academic Medicine, 80, 541–548.

    Article  Google Scholar 

  14. Cook, D. A. (2009). The failure of e-learning research to inform educational practice, and what we can do about it. Medical Teacher, 31, 158–162.

    Article  Google Scholar 

  15. Cook, D. A., Beckman, T. J., Thomas, K. G., & Thompson, W. G. (2008a). Adapting Web-based instruction to residents’ knowledge improves learning efficiency: A randomized controlled trial. Journal of General Internal Medicine, 23, 985–990.

    Article  Google Scholar 

  16. Cook, D. A., Beckman, T. J., Thomas, K. G., & Thompson, W. G. (2008b). Introducing resident physicians to complexity in ambulatory medicine. Medical Education, 42, 838–848.

    Article  Google Scholar 

  17. Cook, D. A., & Dupras, D. M. (2004). A practical guide to developing effective web-based learning. Journal of General Internal Medicine, 19, 698–707.

    Article  Google Scholar 

  18. Cook, D. A., Dupras, D. M., Thompson, W. G., & Pankratz, V. S. (2005). Web-based learning in residents’ continuity clinics: A randomized, controlled trial. Academic Medicine, 80, 90–97.

    Article  Google Scholar 

  19. Cook, D. A., Levinson, A. J., Garside, S., Dupras, D. M., Erwin, P. J., & Montori, V. M. (2008c). Internet-based learning in the health professions: A meta-analysis. JAMA, 300, 1181–1196.

    Article  Google Scholar 

  20. Cook, D. A., Levinson, A. J., Garside, S., Dupras, D. M., Erwin, P. J., & Montori, V. M. (2010). Instructional design variations in internet-based learning for health professions education: A systematic review and meta-analysis. Academic Medicine, 85, 909–922.

    Google Scholar 

  21. Cook, D. A., Thompson, W. G., & Thomas, K. G. (2009a). Case-based or non-case-based questions for teaching postgraduate physicians: A randomized crossover trial. Academic Medicine, 84, 1419–1425.

    Article  Google Scholar 

  22. Cook, D. A., Thompson, W. G., Thomas, K. G., & Thomas, M. R. (2009b). Lack of interaction between sensing–intuitive learning styles and problem-first versus information-first instruction: A randomized crossover trial. Advances in Health Sciences Education, 14, 79–90.

    Article  Google Scholar 

  23. Cook, D. A., Thompson, W. G., Thomas, K. G., Thomas, M. R., & Pankratz, V. S. (2006). Impact of self-assessment questions and learning styles in web-based learning: A randomized, controlled, crossover trial. Academic Medicine, 81, 231–238.

    Article  Google Scholar 

  24. Cooper, E. H., & Pantle, A. J. (1967). The total-time hypothesis in verbal learning. Psychological Bulletin, 68, 221–234.

    Article  Google Scholar 

  25. Craik, F. I. M., & Tulving, E. (1975). Depth of processing and the retention of words in episodic memory. Journal ol Experimental Psychology: General, 104, 268–294.

    Article  Google Scholar 

  26. Cronbach, L. J., & Snow, R. E. (1977). Aptitudes and instructional methods: A handbook for research on interactions. New York: Irvington Publishers.

    Google Scholar 

  27. Curtin, F., Altman, D. G., & Elbourne, D. (2002). Meta-analysis combining parallel and cross-over clinical trials. I: Continuous outcomes. Statistics in Medicine, 21, 2131–2144.

    Article  Google Scholar 

  28. Dennis, J. K. (2003). Problem-based learning in online vs. face-to-face environments. Education for Health, 16(2), 198–209.

    Article  Google Scholar 

  29. Effective Use of Educational Technology in Medical Education: Summary Report of the 2006 AAMC Colloquium on Educational Technology. (2007). Washington, DC: Association of American Medical Colleges.

  30. Fletcher, J. D. (2009). Education and training technology in the military. Science, 323, 72–75.

    Article  Google Scholar 

  31. Fordis, M., King, J. E., Ballantyne, C. M., Jones, P. H., Schneider, K. H., Spann, S. J., et al. (2005). Comparison of the instructional efficacy of internet-based CME with live interactive CME workshops: A randomized controlled trial. JAMA, 294, 1043–1051.

    Article  Google Scholar 

  32. Fredrick, W. C., & Walberg, H. J. (1980). Learning as a function of time. Journal of Educational Research, 73, 183–194.

    Google Scholar 

  33. Friedl, R., Hoppler, H., Ecard, K., Scholz, W., Hannekum, A., Ochsner, W., et al. (2006a). Multimedia-driven teaching significantly improves students’ performance when compared with a print medium. Annals of Thoracic Surgery, 81, 1760–1766.

    Article  Google Scholar 

  34. Friedl, R., Hoppler, H., Ecard, K., Scholz, W., Hannekum, A., Oechsner, W., et al. (2006b). Comparative evaluation of multimedia driven, interactive, and case-based teaching in heart surgery. Annals of Thoracic Surgery, 82, 1790–1795.

    Article  Google Scholar 

  35. Gijselaers, W. H., & Schmidt, H. G. (1995). Effects of quantity of instruction on time spent on learning and achievement. Educational Research and Evaluation, 1(2), 183–201.

    Article  Google Scholar 

  36. Goitein, L., Shanafelt, T. D., Wipf, J. E., Slatore, C. G., & Back, A. L. (2005). The effects of work-hour limitations on resident well-being, patient care, and education in an internal medicine residency program. Archives of Internal Medicine, 165, 2601–2606.

    Article  Google Scholar 

  37. Greenhalgh, T. (2001). Computer assisted learning in undergraduate medical education. BMJ, 322, 40–44.

    Article  Google Scholar 

  38. Grundman, J., Wigton, R., & Nickol, D. (2000). A controlled trial of an interactive, web-based virtual reality program for teaching physical diagnosis skills to medical students. Academic Medicine, 75(10 Suppl), S47–S49.

    Article  Google Scholar 

  39. Higgins, J. P. T., & Green, S. (2008). Cochrane handbook for systematic reviews of interventions 5.0.1 [updated September 2008]. Retrieved May 11, 2009, from http://www.cochrane.org/resources/handbook/index.htm.

  40. Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. BMJ, 327, 557–560.

    Article  Google Scholar 

  41. Kopp, V., Stark, R., & Fischer, M. R. (2008). Fostering diagnostic knowledge through computer-supported, case-based worked examples: Effects of erroneous examples and feedback. Medical Education, 42, 823–829.

    Article  Google Scholar 

  42. Kulik, C.-L. C., Kulik, J. A., & Shwalb, B. J. (1986). The effectiveness of computer-based adult education: A meta-analysis. Journal of Educational Computing Research, 2, 235–252.

    Article  Google Scholar 

  43. Larreamendy-Joerns, J., & Leinhardt, G. (2006). Going the distance with online education. Review of Educational Research, 76(4), 567–605.

    Article  Google Scholar 

  44. Leong, S. L., Baldwin, C. D., & Adelman, A. M. (2003). Integrating web-based computer cases into a required clerkship: Development and evaluation. Academic Medicine, 78, 295–301.

    Article  Google Scholar 

  45. Lewis, M. J., Davies, R., Jenkins, D., & Tait, M. I. (2001). A review of evaluative studies of computer-based learning in nursing education. Nurse Education Today, 21, 26–37.

    Article  Google Scholar 

  46. Mattheos, N., Nattestad, A., Christersson, C., Jansson, H., & Attstrom, R. (2004). The effects of an interactive software application on the self-assessment ability of dental students. European Journal of Dental Education, 8(3), 97–104.

    Google Scholar 

  47. McKimm, J., Jollie, C., & Cantillon, P. (2003). ABC of learning and teaching in medicine: Web based learning. BMJ, 326(7394), 870–873.

    Article  Google Scholar 

  48. Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999). Improving the quality of reports of meta-analyses of randomised controlled trials: The QUOROM statement. Quality of reporting of meta-analyses. Lancet, 354, 1896–1900.

    Article  Google Scholar 

  49. Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA Statement. Annals of Internal Medicine, 151, 264–269.

    Google Scholar 

  50. Nicholson, D. T., Chalk, C., Funnell, W. R. J., & Daniel, S. J. (2006). Can virtual reality improve anatomy education? A randomised controlled study of a computer-generated three-dimensional anatomical ear model. Medical Education, 40(1), 1081–1087.

    Article  Google Scholar 

  51. Papa, F. J., Aldrich, D., & Schumacker, R. E. (1999). The effects of immediate online feedback upon diagnostic performance. Academic Medicine, 74(10 suppl), S16–S18.

    Article  Google Scholar 

  52. Papa, F. J., Oglesby, M. W., Aldrich, D. G., Schaller, F., & Cipher, D. J. (2007). Improving diagnostic capabilities of medical students via application of cognitive sciences-derived learning principles. Medical Education, 41, 419–425.

    Article  Google Scholar 

  53. Rosenthal, R. (1994). Parametric measures of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 231–244). New York: Russell Sage Foundation.

    Google Scholar 

  54. Ruiz, J. G., Mintzer, M. J., & Leipzig, R. M. (2006). The impact of e-learning in medical education. Academic Medicine, 81, 207–212.

    Article  Google Scholar 

  55. Ryan, J. (2005). Unintended consequences: The accreditation council for graduate medical education work-hour rules in practice. Annals of Internal Medicine, 143, 82–83.

    Google Scholar 

  56. Schittek Janda, M., Tani Botticelli, A., Mattheos, N., Nebel, D., Wagner, A., Nattestad, A., et al. (2005). Computer-mediated instructional video: A randomised controlled trial comparing a sequential and a segmented instructional video in surgical hand wash. European Journal of Dental Education, 9(2), 53–58.

    Article  Google Scholar 

  57. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.

    Article  Google Scholar 

  58. Spickard, A. III, Alrajeh, N., Cordray, D., & Gigante, J. (2002). Learning about screening using an online or live lecture: Does it matter? Journal of General Internal Medicine, 17, 540–545.

    Article  Google Scholar 

  59. Spickard, A. III., Smithers, J., Cordray, D., Gigante, J., & Wofford, J. L. (2004). A randomised trial of an online lecture with and without audio. Medical Education, 38, 787–790.

    Article  Google Scholar 

  60. Stroup, D. F., Berlin, J. A., Morton, S. C., Olkin, I., Williamson, G. D., Rennie, D., et al. (2000). Meta-analysis of observational studies in epidemiology: A proposal for reporting. JAMA, 283, 2008–2012.

    Article  Google Scholar 

  61. Tallent-Runnels, M. K., Thomas, J. A., Lan, W. Y., Cooper, S., Ahern, T. C., Shaw, S. M., et al. (2006). Teaching courses online: A review of the research. Review of Educational Research, 76(1), 93–135.

    Article  Google Scholar 

  62. Tunuguntla, R., Rodriguez, O., Ruiz, J. G., Qadri, S. S., Mintzer, M. J., Van Zuilen, M. H., et al. (2008). Computer-based animations and static graphics as medical student aids in learning home safety assessment: A randomized controlled trial. Medical Teacher, 30, 815–817.

    Article  Google Scholar 

  63. Wells, G. A., Shea, B., O’Connell, D., Peterson, J., Welch, V., Losos, M., et al. (2007). The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Retrieved March 20, 2010, from http://www.ohri.ca/programs/clinical_epidemiology/oxford.htm.

  64. Wutoh, R., Boren, S. A., & Balas, E. A. (2004). eLearning: A review of internet-based continuing medical education. Journal of Continuing Education in the Health Professions, 24, 20–30.

    Article  Google Scholar 

Download references

Acknowledgments

The authors thank Denise M. Dupras, MD, PhD, Patricia J. Erwin, MLS, and Victor M. Montori, MD, MSc for their role in study identification and data abstraction. This work was supported by intramural funds and a Mayo Foundation Education Innovation award. Dr. Levinson is supported in part through the John R. Evans Chair in Health Sciences Education Research. Dr. Cook had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Author information

Affiliations

Authors

Corresponding author

Correspondence to David A. Cook.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Cook, D.A., Levinson, A.J. & Garside, S. Time and learning efficiency in Internet-based learning: a systematic review and meta-analysis. Adv in Health Sci Educ 15, 755–770 (2010). https://doi.org/10.1007/s10459-010-9231-x

Download citation

Keywords

  • Internet
  • Educational technology
  • Education, professional
  • Computer-assisted instruction
  • Distance education
  • Learning efficiency