Skip to main content

Advertisement

SpringerLink
Log in

Temporal patterns and dynamics of e-learning usage in medical education

  • Research Article
  • Published:
Educational Technology Research and Development Aims and scope Submit manuscript

Abstract

Despite the increasing popularity of e-learning systems across a variety of educational programmes, there is relatively little understanding of how students and trainees distribute their learning efforts over time. This study aimed to analyse the usage patterns of an e-learning resource designed to support specialty training. Data were collected from e-learning Anaesthesia, a web-based training programme offered by the Royal College of Anaesthetists in partnership with e-Learning for Healthcare. We constructed the time series of 45,020 records of knowledge and self-assessment sessions accessed by 2491 trainees between August 2008 and February 2010. Analysis of the time series suggested that e-learning usage was characterised by concentrations of rapidly occurring sessions within short time frames of intense activity, separated by disproportionally long periods of reduced activity. Non-uniform temporal fluctuations of usage were pronounced especially for self-assessment sessions, the timing of which was highly correlated with examination dates. While on average trainees’ involvement in knowledge sessions was larger than in self-assessment sessions, for both sessions average hourly activity and length remained stable between 9:00 am and 10:00 pm during weekdays. Average daily activity decayed as the weekend approached, but average session length did not vary significantly across the week. Combined with previous research on distributed practice, learning time distribution and test-enhanced learning, our study has implications for the improvement of long-term retention through the redistribution of knowledge sessions uniformly over time and the sustainment of frequent information retrieval and repeated testing. Findings on hourly and daily periodicities also suggest how new learning materials could be broken down into suitable components that fit learners’ time constraints.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Andrews, R., & Haythornthwaite, C. (Eds.). (2007). The SAGE Handbook of E-learning Research. London: Sage.

    Google Scholar 

  • Arnott, E., & Dust, M. (2012). Combating unintended consequences of in-class revision using study skills training. Psychology Learning and Teaching, 11, 99–104.

    Article  Google Scholar 

  • Avouris, N., Komis, V., Fiotakis, G., Margaritis, M., & Voyiatzaki, G. (2005). Logging of fingertip actions is not enough for analysis of learning activities. In Proceedings of Workshop Usage Analysis in Learning Systems (AIED’05), Amsterdam.

  • Bangert-Drowns, R. L., Kulik, C. L. C., Kulik, J. A., & Morgan, M. T. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61, 213–238.

    Article  Google Scholar 

  • Barab, S. A., Kling, R., & Gray, J. H. (Eds.). (2004). Designing for Virtual Communities in the Service of Learning. New York: Cambridge University Press.

    Google Scholar 

  • Barabási, A. L. (2005). The origin of bursts and heavy tails in human dynamics. Nature, 435, 207–211.

    Article  Google Scholar 

  • Bata-Jones, B., & Avery, M. D. (2004). Teaching pharmacology to graduate nursing students: Evaluation and comparison of Web-based and face-to-face methods. Journal of Nursing Education, 43(4), 185–189.

    Google Scholar 

  • Bonk, C. J., & Graham, C. R. (Eds.). (2006). Handbook of Blended Learning: Global Perspectives, Local Designs. San Francisco: Pfeiffer Publishing.

    Google Scholar 

  • Butler, A. C., & Roediger, H. L. (2007). Testing improves long-term retention in a simulated classroom setting. European Journal of Cognitive Psychology, 19, 514–527.

    Article  Google Scholar 

  • Carpenter, S. K. (2009). Cue strength as a moderator of the testing effect: The benefits of elaborative retrieval. Journal of Experimental Psychology. Learning, Memory, and Cognition, 35, 1563–1569.

    Article  Google Scholar 

  • Carpenter, S. K., & DeLosh, E. L. (2005). Application of the testing and spacing effects to name learning. Applied Cognitive Psychology, 19, 619–636.

    Article  Google Scholar 

  • Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132, 354–380.

    Article  Google Scholar 

  • Cepeda, N. J., Vul, E., Rohrer, D., Wixted, J. T., & Pashler, H. (2008). Spacing effects in learning: A temporal ridgeline of optimal retention. Psychological Science, 19, 1095–1102.

    Article  Google Scholar 

  • Childs, S., Blenkinsopp, E., Hall, A., & Walton, G. (2005). Effective e-learning for health professionals and students–barriers and their solutions. A systematic review of the literature. Findings from the HeXL project. Health Information & Libraries Journal, 22, 20–32.

    Article  Google Scholar 

  • Choules, A. P. (2007). The use of elearning in medical education: A review of the current situation. Postgraduate Medical Journal, 83(978), 212–216.

    Article  Google Scholar 

  • Cocea, M., & Weibelzahl, S. (2009). Log file analysis for disengagement detection in e-learning environments. User Modeling and User-Adapted Interaction, 19(4), 341–385.

    Article  Google Scholar 

  • Conrad, R. M., & Donaldson, J. A. (2004). Engaging the Online Learner: Activities and Resources for Creative Instruction. San Francisco: Jossey-Bass.

    Google Scholar 

  • Cook, D. A., Levinson, A., Dupras, D., Garside, S., Erwin, P., & Montori, V. M. (2008). Internet-based learning in the health professions: A meta-analysis. Journal of the American Medical Association (JAMA), 300(10), 1181–1196.

    Article  Google Scholar 

  • Cull, W. L. (2000). Untangling the benefits of multiple study opportunities and repeated testing for cued recall. Applied Cognitive Psychology, 14, 215–235.

    Article  Google Scholar 

  • Curran, V. R., & Fleet, L. (2005). A review of evaluation outcomes of web-based continuing medical education. Medical Education, 39(6), 561–567.

    Article  Google Scholar 

  • De Jonge, M., Tabbers, H. K., Pecher, D., & Zeelenberg, R. (2012). The effect of study time distribution on learning and retention: A Goldilocks principle for presentation rate. Journal of Experimental Psychology, 38(2), 405–412.

    Google Scholar 

  • Delaney, P. F., Verkoeijen, P. P. J. L., & Spirgel, A. (2010). Spacing and testing effects: A deeply critical, lengthy, and at times discursive review of the literature. Psychology of Learning and Motivation, 53, 63–147.

    Article  Google Scholar 

  • Devitt, P., & Palmer, E. (1999). Computer-aided learning: An overvalued educational resource? Medical Education, 33, 136–139.

    Article  Google Scholar 

  • Dezsö, Z., Almaas, E., Lukács, A., Rácz, B., Szakadát, I., & Barabási, A. L. (2006). Dynamics of information access on the web. Physical Review E, 73, 066132.

    Article  Google Scholar 

  • Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4–58.

    Article  Google Scholar 

  • Eckmann, J. P., Moses, E., & Sergi, D. (2004). Entropy of dialogues creates coherent structures in e-mail traffic. Proceedings of the National Academy of Sciences of the USA, 101(40), 14333–14337.

    Article  Google Scholar 

  • Ellaway, R. (2010). eMedical teacher. Medical Teacher, 32, 705–707.

    Article  Google Scholar 

  • Ellaway, R. (2011). E-learning: Is the revolution over? Medical Teacher, 33, 297–302.

    Article  Google Scholar 

  • Ganger, A. C., & Jackson, M. (2003). Wireless handheld computers in the preclinical undergraduate curriculum. Medical Education Online, 8(3).

  • Goetz, M., Leskovec, J., Mcglohon, M., & Faloutsos, C. (2009). Modeling blog dynamics. In AAAI Conference on Weblogs and Social Media (ICWSM).

  • Goh, K. I., & Barabási, A. L. (2008). Burstiness and memory in complex systems. Europhysics Letters, 81, 48002.

    Article  Google Scholar 

  • Goossens, N. A. M. C., Camp, G., Verkoeijen, P. P. J. L., Tabbers, H. K., & Zwaan, R. A. (2012). Spreading the words: A spacing effect in vocabulary learning. Journal of Cognitive Psychology, 24(8), 965–971.

    Article  Google Scholar 

  • Greenhalgh, T. (2001). Computer assisted learning in undergraduate medical education. British Medical Journal, 322, 40–44.

    Article  Google Scholar 

  • Haight, F. A. (1967). Handbook of the Poisson Distribution. New York: Wiley.

    Google Scholar 

  • Haythornthwaite, C., & Andrews, R. (2011). E-learning Theory & Practice. London: Sage.

    Book  Google Scholar 

  • Henderson, T., & Bhatti, S. (2001). Modelling user behaviour in networked games. Proceedings of ACM Multimedia (pp. 212–220). Ottawa: ACM Press.

    Google Scholar 

  • Hwang, W. Y., & Li, C. C. (2002). What the user log shows based on learning time distribution. Journal of Computer Assisted learning, 18(2), 232–233.

    Article  Google Scholar 

  • Jackson, M., Ganger, A. C., Bridge, P. D., & Ginsburg, K. (2005). Wireless handheld computers in the undergraduate medical curriculum. Medical Education Online, 10(5).

  • Jeske, D., Backhaus, J., & Stamov Rossnagel, C. (2014). Self-regulation during e-learning: Using behavioural evidence from navigation log files. Journal of Computer Assisted learning, 30(3), 272–284.

    Article  Google Scholar 

  • Johnson, N. F. (1964). The functional relationship between amount learned and frequency versus rate versus total time of exposure of verbal materials. Journal of Verbal Learning and Verbal Behavior, 3, 502–504.

    Article  Google Scholar 

  • Jonassen, D. H. (1996). Computers in the Classroom: Mindtools for Critical Thinking. Englewood Cliffs, NJ: Prentice Hall.

    Google Scholar 

  • Jones, T., & Jones, M. (1997). MacSQUEAL: A tool for exploration of hypermedia log file sequences. In Proceedings of Ed-Media 1997 World Conference on Multimedia and Hypermedia in Education. Charlottesville, VA: Association for the Advancement of Computing in Education.

  • Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of retrieval for learning. Science, 15, 966–968.

    Article  Google Scholar 

  • Kleinpell, R., Ely, E. W., Liolios, A., Ward, N., & Tisherman, S. A. (2011). Web-based resources for critical care education. Critical Care Medicine, 39, 2202–2203.

    Article  Google Scholar 

  • Kornell, N. (2009). Optimising learning using flashcards: Spacing is more effective than cramming. Applied Cognitive Psychology, 23(9), 1297–1317.

    Article  Google Scholar 

  • Kujawski, B., Hołyst, J., & Rodgers, G. J. (2007). Growing trees in internet news groups and forums. Physical Review E, 76, 036103.

    Article  Google Scholar 

  • Kumta, S. M., Tsang, P. L., Hung, L. K., & Cheng, J. C. Y. (2003). Fostering critical thinking skills through a web-based tutorial programme for final year medical students—A randomized controlled study. Journal of Educational Multimedia and Hypermedia, 12(3), 267–273.

    Google Scholar 

  • Larsen, D. P., Butler, A. C., & Roediger, H. L. (2009). Repeated testing improves long-term retention relative to repeated study: A randomised controlled trial. Medical Education, 43, 1174–1181.

    Article  Google Scholar 

  • Lau, F., & Bates, J. (2004). A review of e-learning practices for undergraduate medical education. Journal of Medical Systems, 28(1), 71–87.

    Article  Google Scholar 

  • Leeming, F. C. (2002). The exam-a-day procedure improves performance in psychology classes. Teaching of Psychology, 29, 210–212.

    Article  Google Scholar 

  • Li, C., & Irby, B. (2008). An overview of online education: Attractiveness, benefits, challenges, concerns and recommendations. College Student Journal, 42(2), 449–458.

    Google Scholar 

  • Lyle, K. B., & Crawford, N. A. (2011). Retrieving essential material at the end of lectures improves performance on statistics exams. Teaching of Psychology, 38, 94–97.

    Article  Google Scholar 

  • Mawhinney, V. T., Bostow, D. E., Laws, D. R., Blumenfield, G. J., & Hopkins, B. L. (1971). A comparison of students studying behavior produced by daily, weekly, and three-week testing schedules. Journal of Applied Behavior Analysis, 4, 257–264.

    Article  Google Scholar 

  • Michael, J. (1991). A behavioral perspective on college teaching. The Behavior Analyst, 14, 229–239.

    Google Scholar 

  • Mohanna, K. (2007). The use of e-learning in medical education. Postgraduate Medical Journal, 83, 211.

    Article  Google Scholar 

  • Morrison, G. R., Ross, S. M., Kalman, H. K., & Kemp, J. E. (2013). Designing Effective Instruction (7th ed.). Hoboken, NJ: Wiley.

    Google Scholar 

  • Panzarasa, P., Opsahl, T., & Carley, K. M. (2009). Patterns and dynamics of users’ behavior and interaction: Network analysis of an online community. Journal of the American Society for Information Science and Technology, 60(5), 911–932.

    Article  Google Scholar 

  • Pyc, M. A., & Rawson, K. A. (2010). Why testing improves memory: Mediator effectiveness hypothesis. Science, 330, 335.

    Article  Google Scholar 

  • Pyc, M. A., & Rawson, K. A. (2012). Why is test-restudy practice beneficial for memory? An evaluation of the mediator shift hypothesis. Journal of Experimental Psychology, 38, 737–746.

    Google Scholar 

  • Reiser, S. (2007). Technological Medicine: The Changing World of Doctors and Patients. New York: Cambridge University Press.

    Google Scholar 

  • Ritter, F., Baxter, G., Kim, J., & Srinivasmurthy, S. (2012). Learning and retention. In J. Lee & A. Kirlik (Eds.), The Oxford Handbook of Cognitive Engineering. New York: Oxford University Press.

    Google Scholar 

  • Roediger, H. L., & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15, 20–27.

    Article  Google Scholar 

  • Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17, 249–255.

    Article  Google Scholar 

  • Sajeva, M. (2006). E-learning: Web-based education. Current Opinion in Anaesthesiology, 19, 645–649.

    Article  Google Scholar 

  • Sandars, J., & Haythornthwaite, C. (2007). New horizons for e-learning in medical education: Ecological and web 2.0 perspectives. Medical Teacher, 29, 307–310.

    Article  Google Scholar 

  • Sargeant, J., Curran, V., Jarvis-Selinger, S., Ferrier, S., Allen, M., Kirby, F., & Ho, K. (2004). Interactive on-line continuing medical education: Physicians’ perceptions and experiences. Journal of Continuing Education in the Health Professions, 24(4), 227–236.

    Article  Google Scholar 

  • Stubin, E. J., Heimer, W. I., & Tatz, S. J. (1970). Total time and presentation time in paired-associate learning. Journal of Experimental Psychology, 84, 308–310.

    Article  Google Scholar 

  • Thiele, J. (2003). Learning patterns of online students. Nursing Education, 42(8), 364–366.

    Google Scholar 

  • Valsamidis, S., Kontogiannis, S., Kazanidis, I., & Karakos, A. (2011). E-learning platform usage analysis. Interdisciplinary Journal of E-Learning and Learning Objects, 7, 185–204.

    Google Scholar 

  • Vázquez, A., Oliveira, J. G., Dezsö, Z., Goh, K., Kondor, I., & Barabási, A. L. (2006). Modeling bursts and heavy tails in human dynamics. Physical Review E, 73(3), 036127.

    Article  Google Scholar 

  • Wang, M., Chan, N. H., Papadimitriou, S., Faloutsos, C., & Madhyastha, T. M. (2002). Data mining meets performance evaluation: Fast algorithms for modeling bursty traffic. In ICDE ‘02 Proceedings of the 18th International Conference on Data Engineering (pp. 507–516).

  • Willinger, W., Taqqu, M. S., Sherman, R., & Wilson, D. V. (1997). Self-similarity through high-variability: Statistical analysis of ethernet LAN traffic at the source level. IEEE/ACM Transactions on Networking, 5(1), 71–86.

    Article  Google Scholar 

  • Wong, L., & Tatnall, A. (2010). Factors determining the balance between online and face-to-face teaching: An analysis using actor-network theory. Interdisciplinary Journal of Information, Knowledge, and Management, 5, 167–176.

    Google Scholar 

Download references

Acknowledgments

We wish to give special thanks to Moreno Bonaventura and Valerio Ciotti for their help in the preparation of the figures and for their comments on earlier versions of this article.

Author information

Authors and Affiliations

  1. School of Business and Management, Queen Mary University of London, London, UK

    Pietro Panzarasa & Bernard Kujawski

  2. Royal College of Anaesthetists, London, UK

    Edward J. Hammond

  3. Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London, UK

    C. Michael Roberts

Authors
  1. Pietro Panzarasa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bernard Kujawski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Edward J. Hammond
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. C. Michael Roberts
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pietro Panzarasa.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Panzarasa, P., Kujawski, B., Hammond, E.J. et al. Temporal patterns and dynamics of e-learning usage in medical education. Education Tech Research Dev 64, 13–35 (2016). https://doi.org/10.1007/s11423-015-9407-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-015-9407-4

Keywords

Search

Navigation