Skip to main content
Book cover

Online Teaching and Learning in Higher Education pp 161–183Cite as

Usability Evaluation of Virtual Learning Environments: A University Case Study

Part of the Cognition and Exploratory Learning in the Digital Age book series (CELDA)

Abstract

Virtual learning environments (VLEs) are widely adopted in higher education to facilitate online methods of content delivery for the lecturers, to enable online submission for the students and to provide course management tools for the administration team. There are a variety of software solutions to choose from, modelled according to heterogeneous needs and functionalities. Despite the existence of clear organisational, pedagogical and technological goals, the procurement and implementation of a VLE is a complex task with significant challenges. We present the case study of a university which went through the process of procuring a new VLE. We examine the usability of each VLE utilising the System Usability Scale (SUS) and capture further feedback from the stakeholders by applying the Interactive Management (IM) methodology.

The first part of the research focuses on the three VLEs remaining in contention during the final stages of the procurement process. The results of the usability evaluations are analysed, explained and compared. The second part of the study (Vertesi A, Dogan H, Stefanidis A, Ashton G, Drake W: Usability evaluation of a virtual learning environment: a university case study. In: 15th international Conference on Cognition and Exploratory Learning in Digital Age (CELDA), 21–23 October 2018, Budapest, Hungary, 2018) examines the selected and implemented VLE 6 months after its initiation. A usability test was carried out again on this VLE to examine changes since its launching. Additional feedback was collected from the stakeholders to support the fine-tuning process after the implementation. According to the evaluation of each user group, all the three VLEs performed below the average usability expectation. Generally, students evaluated the usability of the VLEs higher than the academics and administration staff. The usability scores of the students’ evaluation from different courses and years show remarkable differences. The ranked and categorised feedback given by the stakeholders highlights the importance of planning, training and communication prior to and during the implementation process. Usability and learnability play important roles according to the feedback.

Keywords

  • Virtual learning environment (VLE)
  • Learning Management System (LMS)
  • Usability evaluation
  • System Usability Scale (SUS)
  • Interactive Management (IM)

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-48190-2_9
  • Chapter length: 23 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-48190-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)
Hardcover Book
USD   169.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 9.1
Fig. 9.2
Fig. 9.3
Fig. 9.4
Fig. 9.5
Fig. 9.6
Fig. 9.7
Fig. 9.8
Fig. 9.9

References

  • Ayad, K., & Rigas, D. (2010). Comparing virtual classroom, game-based learning and storytelling teachings in e-learning. International Journal of Education and Information Technologies, 4(1), 15–23.

    Google Scholar 

  • Babić, S. (2012). Factors that influence academic teacher’s acceptance of e-learning technology in blended learning environment. In E-learning-organizational infrastructure and tools for specific areas (pp. 3–18). Rijeka: InTech.

    Google Scholar 

  • Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the system usability scale. International Journal of Human-Computer Interaction, 24(6), 574–594.

    CrossRef  Google Scholar 

  • Bangor, A., Kortum, P., & Miller, J. (2009). Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of Usability Studies, 4(3), 114–123.

    Google Scholar 

  • Boyd, K., Bond, R., Vertesi, A., Dogan, H., & Magee, J. (2019). How people judge the usability of a desktop graphic user interface at different time points: Is there evidence for memory decay, recall bias or temporal bias? Interacting with Computers, 31(2), 221–230.

    CrossRef  Google Scholar 

  • Brooke, J. (1996). SUS-A quick and dirty usability scale. Usability Evaluation in Industry, 189(194), 4–7.

    Google Scholar 

  • Broome, B. J., & Keever, D. B. (1986). Facilitating group communication: The interactive management approach.

    Google Scholar 

  • Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.

    CrossRef  Google Scholar 

  • Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982–1003.

    CrossRef  Google Scholar 

  • Dogan, H., & Henshaw, M. J. D. (2010, April). Transition from soft systems to an enterprise knowledge management architecture. In International conference on Contemporary Ergonomics and Human Factors, pp. 13–15.

    Google Scholar 

  • Emelyanova, N., & Voronina, E. (2014). Introducing a learning management system at a Russian university: Students’ and teachers’ perceptions. The International Review of Research in Open and Distributed Learning, 15(1).

    Google Scholar 

  • Granić, A., & Ćukušić, M. (2011). Usability testing and expert inspections complemented by educational evaluation: A case study of an e-learning platform. Journal of Educational Technology & Society, 14(2), 107–123.

    Google Scholar 

  • ISO. (1998). 9241-11. Ergonomic requirements for office work with visual display terminals (VDTs). The International Organization for Standardization, 45(9).

    Google Scholar 

  • Luo, G. H., Liu, E. Z. F., Kuo, H. W., & Yuan, S. M. (2014). Design and implementation of a simulation-based learning system for international trade. The International Review of Research in Open and Distributed Learning, 15(1).

    Google Scholar 

  • Marco, F. A., Penichet, V. M. R., & Gallud, J. A. (2013). Collaborative e-learning through Drag & Share in synchronous shared workspaces. Journal of Universal Computer Science, 19(7), 894–911.

    Google Scholar 

  • Melis, E., Weber, M., & Andrès, E. (2003). Lessons for (pedagogic) usability of eLearning systems. In E-learn: World conference on E-learning in corporate, government, healthcare, and higher education (pp. 281–284). Norfolk: Association for the Advancement of Computing in Education (AACE).

    Google Scholar 

  • Onacan, M. B. K., & Erturk, A. (2016, October). Usability evaluation of learning management system in higher education institution: A scale development study. In International conference on Leadership, Technology, Innovation and Business Management (ICLTIBM-2016), Antalya, Turkiye.

    Google Scholar 

  • Paulsen, M. F. (2002). Online education systems: Discussion and definition of terms. NKI distance education, 202.

    Google Scholar 

  • Renaut, C., Batier, C., Flory, L., & Heyde, M. (2006). Improving web site usability for a better e-learning experience. Current developments in technology-assisted education, 891–896.

    Google Scholar 

  • Ryan, S., Scott, B., Freeman, H., & Patel, D. (2013). The virtual university: The internet and resource-based learning. Routledge.

    Google Scholar 

  • Sauro, J., & Lewis, J. R. (2011, May). When designing usability questionnaires, does it hurt to be positive? In CHI (pp. 2215–2224).

    Google Scholar 

  • Tullis, T. S., & Stetson, J. N. (2004, June). A comparison of questionnaires for assessing website usability. In Usability Professional Association conference (Vol. 1).

    Google Scholar 

  • Virzi, R. A. (1992). Refining the test phase of usability evaluation: How many subjects is enough? Human Factors, 34(4), 457–468.

    CrossRef  Google Scholar 

  • Vertesi, A., Dogan, H., Stefanidis, A., Ashton, G., & Drake, W. (2018). Usability evaluation of a virtual learning environment: a university case study. In: 15th International Conference on Cognition and Exploratory Learning in Digital Age (CELDA), Budapest, Hungary.

    Google Scholar 

  • Warfield, J. N., & Cárdenas, A. R. (1994). A handbook of interactive management (p. 338). Ames: Iowa State University Press.

    Google Scholar 

Download references

Acknowledgements

The research team would like to express gratitude to the Centre for Excellence of Learning (CEL) and Visual for Learning (V4L) team of the university, especially to Ms. Wendy Drake and Mr. Giles Ashton for the contribution of the research by organising surveys and providing data for the research.

Author information

Authors and Affiliations

  1. Bournemouth University, Dorset, UK

    Attila Vertesi, Huseyin Dogan & Angelos Stefanidis

Authors
  1. Attila Vertesi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huseyin Dogan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Angelos Stefanidis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Attila Vertesi .

Editor information

Editors and Affiliations

  1. School of Information Systems & Technology Management, UNSW Business School, Sydney, NSW, Australia

    Pedro Isaias

  2. Department of Digital Systems, University of Piraeus, Piraeus, Greece

    Dr. Demetrios G. Sampson

  3. Learning, Design and Technology, University of Mannheim, Mannheim, Baden-Württemberg, Germany

    Prof. Dr. Dirk Ifenthaler

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Vertesi, A., Dogan, H., Stefanidis, A. (2020). Usability Evaluation of Virtual Learning Environments: A University Case Study. In: Isaias, P., Sampson, D.G., Ifenthaler, D. (eds) Online Teaching and Learning in Higher Education. Cognition and Exploratory Learning in the Digital Age. Springer, Cham. https://doi.org/10.1007/978-3-030-48190-2_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-48190-2_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-48189-6

  • Online ISBN: 978-3-030-48190-2

  • eBook Packages: EducationEducation (R0)