Skip to main content
SpringerLink
Log in

An approach to usability evaluation of e-learning applications

  • LONG PAPER
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

Despite recent advances of electronic technologies in e-learning, a consolidated evaluation methodology for e-learning applications is not available. The evaluation of educational software must consider its usability and more in general its accessibility, as well as its didactic effectiveness. This work is a first step towards the definition of a methodology for evaluating e-learning applications. Specific usability attributes capturing the peculiar features of these applications are identified. A preliminary user study involving a group of e-students, observed during their interaction with an e-learning application in a real situation, is reported. Then, the proposal is put forward to adapt to the e-learning domain a methodology for systematic usability evaluation, called SUE. Specifically, evaluation patterns are proposed that are able to drive the evaluators in the analysis of an e-learning application.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Ardito C, De Marsico M, Lanzilotti R, Levialdi S, Roselli T, Rossano V, Tersigni M (2004) Usability of e-learning tools. In: Proceedings of AVI 2004, May 25–28, 2004, Gallipoli (Italy), pp 80–84

  2. Asdi AK, Daniels BH (2000) Special interest group: “learnability” testing in learner-centered design. Extended abstracts on Human Factors in Computing Systems CHI 00, The Hague, 1–6 April, 2000, pp 309

  3. Baker K, Greenberg S, Gutwin C (2002) Empirical development of a heuristics evaluation methodology for shared workspace groupware. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW 02), New Orleans, 16–20 November 2002, pp 96–105

  4. Costabile MF, Matera M (2001) Guidelines for hypermedia usability inspection. IEEE Multimedia, Vol. January–March, pp 66–69

  5. De Angeli A, Matera M, Costabile MF, Garzotto F, Paolini P (2000) Validating the SUE inspection technique. In: Proceedings of AVI 2000, 23–26 May, 2000, Palermo (Italy), pp 143–150

  6. De Angeli A, Matera M, Costabile MF, Garzotto F, Paolini P (2003) On the Advantages of a Systematic Inspection for Evaluating Hypermedia Usability. Int J Hum-Comput Interact 15(3):315–335

    Article  Google Scholar 

  7. Dringus L (1995) An iterative usability evaluation procedure for interactive online courses. J Int Instr Dev 7(4):10–14

    Google Scholar 

  8. Galtung J (1981) Structure, culture and intellectual style: an essay comparing saxonic, teutonic, gallic and nipponic approaches. In Social Science Formation, SAGE, London and Beverly Hills, pp 817–856

  9. Hofstede G (1997) Cultures and organizations: software of the mind, McGraw-Hill, NY

    Google Scholar 

  10. International Organisation for Standardisation (1998) ISO 9241: Software Ergonomics Requirements for office work with visual display terminal (VDT), Geneva, Switzerland

  11. International Organisation for Standardisation (1999) ISO 13407: Human-centered design process for interactive systems, Geneva, Switzerland

  12. Jeffriess R, Miller J, Wharton C, Uyeda KM (1991) User interface evaluation in the real world: a comparison of four techniques. In: Proceedings of ACM Conference on Human Factors and Computing Systems (CHI’91), New Orleans 1991, ACM Press, pp 119–124

  13. Kamentz E, Schudnagis M, Womser-Hacker C (2002) SELIM: human computer interaction in the context of multimedia learning systems and the aspect of adaptivity on the basis of cultural differences. In: Wagner E, Szucs A (eds) EDEN second research workshop 2002. Research and policy in open distance learning. Research workshop book. Universitat Hildesheim, pp 211–215

  14. Kamentz E, Womser-Hacker C (2003) Using cultural differences in educational program design and approaches to computers for adaptation concepts of multimedia learning. In: Stephanidis (eds) Universal access in HCI: inclusive design in the information society. Proceedings of the tenth International Conference on Human–Computer Interaction, HCI 2003, pp 1128–1132

  15. Kruse, K. (May, 2000). Web rules: effective user interface design http://www.learningcircuits.org/may2000/may2000_webrules.html

  16. Mankoff J, Dey A, Hsieh G, Kients J, Lederer S, Ames M (2003) Heuristic evaluation of ambient display. In: Proceedings of ACM Conference on Human Factors and Computing Systems (CHI ’03), Ft. Lauderdale, 5–10 April, 2003, pp 169–176

  17. Matera M, Costabile MF, Garzotto F, Paolini P (2002) SUE inspection: an effective method for systematic usability evaluation of hypermedia. IEEE Trans Syst, Man and Cybern—Part A 32(1):93–103

    Article  Google Scholar 

  18. Meyhew DJ (1992) Principles and Guidelines in Software User Interface Design. Prentice-Hall, Englewood Cliffs

    Google Scholar 

  19. Nielsen J (1993) Usability Engineering. Academic Press, Cambridge

    MATH  Google Scholar 

  20. Norman D (1993) Things that make us smart: defending human attributes in the age of the machine. Perseus Publishing, Cambridge, MA

    Google Scholar 

  21. Notess M (2001) Usability, user experience, and learner experience, http://www.elearnmag.org

  22. Parlangeli O, Marchigiani E, Bagnara S (1999) Multimedia system in distance education: effects on usability. Interacting Comput 12:37–49

    Article  Google Scholar 

  23. Phillips DC (1995) The Good, the bad, the ugly: the many faces of constructivism. Educ Res 24(7):5–12

    Google Scholar 

  24. Quinn CN, Alem L, Eklund J (1997) A pragmatic evaluation methodology for an assessment of learning effectiveness in instructional systems, http://www.testingcentre.com/jeklund/Interact.htm

  25. Quintana C, Carra A, Krajcik J, Elliot S (2001) Learner-centred design: reflections and new directions.In: Carroll (ed) Human-computer interaction in the new millennium. ACM Press, New York, pp 605–626

    Google Scholar 

  26. Ravden SJ, Johnson GI (1989) Evaluating usability of human-computer interface: a practical method. Wiley, Chichester

    Google Scholar 

  27. Roselli T (1995) Artificial Intelligence Can Improve Hypermedia Instructional Technologies for Learning. ACM Comput Sur 27(4):624–626

    Article  Google Scholar 

  28. Roselli T, Faggiano E, Rossano V (2003) A further investigation on the effectiveness of a WWW hypermedial system for cooperative learning. In: Proceedings of International Conference on Computer in Education (ICCE 2003), Hong Kong, December 2003, pp 440–442

  29. Roselli T, Faggiano E, Grasso A, Rossano V (2003) A platform for building and using personalized courseware. In: Proceedings of E-Learn 2003: World conference on E-Learning in Corporate, Government, Healthcare,& Higher Education, Phoenix, November 2003, pp 141–144

  30. Roselli T, Faggiano E, Grasso A, Pragnell MV (2003) A formal model for developing e-learning software. In: Proceedings of IEEE International Convention (Mipro 2003), Opatija, Croatia, May 2003, pp 70–74

  31. Schwier RA, Misanchunk ER (1993) Interactive multimedia instruction. Educational Technology Publications, Englewood Cliffs

    Google Scholar 

  32. Shneidermann B (2003) Designing the User Interface, Addison-Wesley

  33. Silius K, Tervakari A (2003) The usefulness of web-based learning environments. In: Proceeding of the International Conference on Network Universities and e-learning,Valencia, Spain, 8–9 May 2003

  34. Soloway E, Jackson SL, Kleim J, Quintana C, Reed J, Spitulnik J, Stratford SJ, Studer S, Eng J, Scala N (1996) Learning theory in practice: case studies in learner-centered design. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’96), Vancouver, British Columbia, Canada, April 1996. ACM Press, New York, pp 189–196

  35. Somervell J, Wahid S, McCrickard DS (2003) Usability heuristics for large screen information exhibits. In: Proceedings of Human–Computer Interaction (Interact ’03), Zurigo, September, 2003, pp 904–907

  36. Squire D, Preece J (1996) Usability and learning: evaluating the potential of educational software. Comput Educ 27(1):15–22

    Article  Google Scholar 

  37. Squires D (1999) Usability and educational software design: special issue of interacting with computers. Int Comput 11:463–466

    Article  Google Scholar 

  38. Squires D, Preece J (1999) Predicting quality in educational software: evaluating for learning, usability, and the synergy between them. Int Comput 11(5):467–483

    Article  Google Scholar 

  39. Storey MA, Philipps B, Maczewski M, Wang M (2002) Evaluating the usability of web-based learning tools. Educ Tech Soc 5(3):91–100

    Google Scholar 

  40. Takagi T, Sugeno M (1985) Fuzzy identification of systems and its applications to modelling and control. IEEE Transac Syst, Man Cybern 15:116–132

    MATH  Google Scholar 

  41. Trinchero R (2004) Valutazione della qualità dei sistemi formativi basati sulle TIC, http://puntoeduft.indire.it/160mila/b/moduli/app/corsi_fortic/b6/6.3/pdf/6.3.pdf

  42. Wong B, Nguyen TT, Chang E, and Jayaratna N (2003) Usability metrics for e-learning. workshop on human computer interface for semantic web and web applications, 3–7 November, 2003, Catania, Sicily, Italy, Springer-Verlag, Berlin Heidelberg New York, Germany, LNCS No: 2889, pp 235–252

  43. Zaharias P, Vasslopoulou K, and Poulymenakou A (2002) Designing on-line learning courses: implications for usability, http://www.japit.org/zaharias_etal02.pdf

Download references

Acknowledgments

The financial support of Italian MIUR through VICE and WEB-MINDS projects is acknowledged. We also thank the anonymous reviewers for their useful comments.

Author information

Authors and Affiliations

  1. Dipartimento di Informatica, Università di Bari, Bari, Italy

    C. Ardito, M. F. Costabile, R. Lanzilotti, T. Roselli & V. Rossano

  2. Dipartimento di Informatica, Università di Roma “La Sapienza”, Rome, Italy

    M. De Marsico & S. Levialdi

Authors
  1. C. Ardito
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. F. Costabile
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. De Marsico
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R. Lanzilotti
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. S. Levialdi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. T. Roselli
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. V. Rossano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to C. Ardito.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ardito, C., Costabile, M.F., Marsico, M.D. et al. An approach to usability evaluation of e-learning applications. Univ Access Inf Soc 4, 270–283 (2006). https://doi.org/10.1007/s10209-005-0008-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-005-0008-6

Keywords

Search

Navigation