Usability Evaluation of Virtual Learning Environments: A University Case Study

  • Attila VertesiEmail author
  • Huseyin Dogan
  • Angelos Stefanidis
Part of the Cognition and Exploratory Learning in the Digital Age book series (CELDA)


Virtual learning environments (VLEs) are widely adopted in higher education to facilitate online methods of content delivery for the lecturers, to enable online submission for the students and to provide course management tools for the administration team. There are a variety of software solutions to choose from, modelled according to heterogeneous needs and functionalities. Despite the existence of clear organisational, pedagogical and technological goals, the procurement and implementation of a VLE is a complex task with significant challenges. We present the case study of a university which went through the process of procuring a new VLE. We examine the usability of each VLE utilising the System Usability Scale (SUS) and capture further feedback from the stakeholders by applying the Interactive Management (IM) methodology.

The first part of the research focuses on the three VLEs remaining in contention during the final stages of the procurement process. The results of the usability evaluations are analysed, explained and compared. The second part of the study (Vertesi A, Dogan H, Stefanidis A, Ashton G, Drake W: Usability evaluation of a virtual learning environment: a university case study. In: 15th international Conference on Cognition and Exploratory Learning in Digital Age (CELDA), 21–23 October 2018, Budapest, Hungary, 2018) examines the selected and implemented VLE 6 months after its initiation. A usability test was carried out again on this VLE to examine changes since its launching. Additional feedback was collected from the stakeholders to support the fine-tuning process after the implementation. According to the evaluation of each user group, all the three VLEs performed below the average usability expectation. Generally, students evaluated the usability of the VLEs higher than the academics and administration staff. The usability scores of the students’ evaluation from different courses and years show remarkable differences. The ranked and categorised feedback given by the stakeholders highlights the importance of planning, training and communication prior to and during the implementation process. Usability and learnability play important roles according to the feedback.


Virtual learning environment (VLE) Learning Management System (LMS) Usability evaluation System Usability Scale (SUS) Interactive Management (IM) 



The research team would like to express gratitude to the Centre for Excellence of Learning (CEL) and Visual for Learning (V4L) team of the university, especially to Ms. Wendy Drake and Mr. Giles Ashton for the contribution of the research by organising surveys and providing data for the research.


  1. Ayad, K., & Rigas, D. (2010). Comparing virtual classroom, game-based learning and storytelling teachings in e-learning. International Journal of Education and Information Technologies, 4(1), 15–23.Google Scholar
  2. Babić, S. (2012). Factors that influence academic teacher’s acceptance of e-learning technology in blended learning environment. In E-learning-organizational infrastructure and tools for specific areas (pp. 3–18). Rijeka: InTech.Google Scholar
  3. Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the system usability scale. International Journal of Human-Computer Interaction, 24(6), 574–594.CrossRefGoogle Scholar
  4. Bangor, A., Kortum, P., & Miller, J. (2009). Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of Usability Studies, 4(3), 114–123.Google Scholar
  5. Boyd, K., Bond, R., Vertesi, A., Dogan, H., & Magee, J. (2019). How people judge the usability of a desktop graphic user interface at different time points: Is there evidence for memory decay, recall bias or temporal bias? Interacting with Computers, 31(2), 221–230.CrossRefGoogle Scholar
  6. Brooke, J. (1996). SUS-A quick and dirty usability scale. Usability Evaluation in Industry, 189(194), 4–7.Google Scholar
  7. Broome, B. J., & Keever, D. B. (1986). Facilitating group communication: The interactive management approach.Google Scholar
  8. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.CrossRefGoogle Scholar
  9. Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982–1003.CrossRefGoogle Scholar
  10. Dogan, H., & Henshaw, M. J. D. (2010, April). Transition from soft systems to an enterprise knowledge management architecture. In International conference on Contemporary Ergonomics and Human Factors, pp. 13–15.Google Scholar
  11. Emelyanova, N., & Voronina, E. (2014). Introducing a learning management system at a Russian university: Students’ and teachers’ perceptions. The International Review of Research in Open and Distributed Learning, 15(1).Google Scholar
  12. Granić, A., & Ćukušić, M. (2011). Usability testing and expert inspections complemented by educational evaluation: A case study of an e-learning platform. Journal of Educational Technology & Society, 14(2), 107–123.Google Scholar
  13. ISO. (1998). 9241-11. Ergonomic requirements for office work with visual display terminals (VDTs). The International Organization for Standardization, 45(9).Google Scholar
  14. Luo, G. H., Liu, E. Z. F., Kuo, H. W., & Yuan, S. M. (2014). Design and implementation of a simulation-based learning system for international trade. The International Review of Research in Open and Distributed Learning, 15(1).Google Scholar
  15. Marco, F. A., Penichet, V. M. R., & Gallud, J. A. (2013). Collaborative e-learning through Drag & Share in synchronous shared workspaces. Journal of Universal Computer Science, 19(7), 894–911.Google Scholar
  16. Melis, E., Weber, M., & Andrès, E. (2003). Lessons for (pedagogic) usability of eLearning systems. In E-learn: World conference on E-learning in corporate, government, healthcare, and higher education (pp. 281–284). Norfolk: Association for the Advancement of Computing in Education (AACE).Google Scholar
  17. Onacan, M. B. K., & Erturk, A. (2016, October). Usability evaluation of learning management system in higher education institution: A scale development study. In International conference on Leadership, Technology, Innovation and Business Management (ICLTIBM-2016), Antalya, Turkiye.Google Scholar
  18. Paulsen, M. F. (2002). Online education systems: Discussion and definition of terms. NKI distance education, 202.Google Scholar
  19. Renaut, C., Batier, C., Flory, L., & Heyde, M. (2006). Improving web site usability for a better e-learning experience. Current developments in technology-assisted education, 891–896.Google Scholar
  20. Ryan, S., Scott, B., Freeman, H., & Patel, D. (2013). The virtual university: The internet and resource-based learning. Routledge.Google Scholar
  21. Sauro, J., & Lewis, J. R. (2011, May). When designing usability questionnaires, does it hurt to be positive? In CHI (pp. 2215–2224).Google Scholar
  22. Tullis, T. S., & Stetson, J. N. (2004, June). A comparison of questionnaires for assessing website usability. In Usability Professional Association conference (Vol. 1).Google Scholar
  23. Virzi, R. A. (1992). Refining the test phase of usability evaluation: How many subjects is enough? Human Factors, 34(4), 457–468.CrossRefGoogle Scholar
  24. Vertesi, A., Dogan, H., Stefanidis, A., Ashton, G., & Drake, W. (2018). Usability evaluation of a virtual learning environment: a university case study. In: 15th International Conference on Cognition and Exploratory Learning in Digital Age (CELDA), Budapest, Hungary.Google Scholar
  25. Warfield, J. N., & Cárdenas, A. R. (1994). A handbook of interactive management (p. 338). Ames: Iowa State University Press.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Attila Vertesi
    • 1
    Email author
  • Huseyin Dogan
    • 1
  • Angelos Stefanidis
    • 1
  1. 1.Bournemouth UniversityDorsetUK

Personalised recommendations