Abstract
This paper presents the results of an exploratory study whose main aim is to verify if the Pattern-Based (PB) inspection technique enables end users to perform reliable evaluation of e-learning systems in real work-related settings. The study involved 13 Polish and Italian participants, who did not have an HCI background, but used e-learning platforms for didactic and/or administrative purposes. The study revealed that the participants were able to effectively and efficiently apply the PB inspection technique with minimum effort. However, in some cases, participants complained that, in some cases, the technique appeared time demanding. This work provides some valuable suggestions to redesign the evaluation tools of the PB technique, in order to improve the focus on specific elements of the e-learning system and to streamline better the evaluation process.
Chapter PDF
References
Hornbæk, K., Frøkjær, E.: Comparing usability problems and redesign proposals as input to practical systems development. In: Proc. of CHI 2005, pp. 391–400. ACM (2005)
Steves, M.P., Morse, E., Gutwin, C., Greenberg, S.: A comparison of usage evaluation and inspection methods for assessing groupware usability. In: Proc. of ACM SIGGROUP Conference on Supporting Group Work, pp. 125–134. ACM (2001)
Tan, W., Dahai, L., Bishu, R.: Web evaluation: heuristic evaluation vs. usability testing. International Journal of Industrial Ergonomics 39(4), 621–627 (2009)
Gerdt, P., Miraftabi, R., Tukiainen, M.: Evaluating educational software environments. In: Proc. of Computers in Education, pp. 675–676. IEEE Computer Society (2002)
Mendes, E., Hall, W., Harrison, R.: Applying metrics to the evaluation of educational hypermedia application. Journal of Universal Computer Science 4(4), 382–403 (1998)
Quinn, C.N., Alem, L., Eklund, J.A.: Pragmatic evaluation methodology for an assessment of learning effectiveness in instructional systems. In: Bewster, S., Cawsey, A., Cockton, G. (eds.) Human–Computer Interaction, vol. II, pp. 55–56. British Computer Society (1997)
Squires, D., Preece, J.: Predicting quality in educational software: evaluating for learning, usability, and the synergy between them. Interact. with Computers 11(5), 467–483 (1999)
Jeffries, R., Desurvire, H.W.: Usability testing vs heuristic evaluation: was there a context? ACM SIGCHI Bulletin 24(4), 39–41 (1992)
Doubleday, A., Ryan, M., Springett, M., Sutcliffe, A.: A comparison of usability techniques for evaluating design. In: Proc. of DIS 1997, pp. 101–110. Springer, Heidelberg (1997)
Law, E.: Heuristic Evaluation. In: Proceedings of COST294-MAUSE International Workshop “Review, Report and Refine Usability Evaluation Methods (R3-UEM)”, pp. 61–63 (2007)
Lanzilotti, R., De Angeli, A., Ardito, C., Costabile, M.F.: Do patterns help novice evaluators? A comparative study. Intern. Journal of Human-Computer Studies 69, 52-69 (2011)
Matera, M., Costabile, M.F., Garzotto, F., Paolini, P.: SUE Inspection: an effective method for systematic usability evaluation of hypermedia. IEEE Transactions on Systems, Man and Cybernetics – Part A 32(1), 93–103 (2002)
Bias, R.G., Mayhew, D.J.: Cost-justifying usability: An update for the Internet age, 2nd edn. Morgan Kaufmann Publishers, San Francisco (2005)
Hudson, J.: Beyond Usability to User Experience. In: Workshop UXEM at CHI 2008 (2008), http://www.cs.tut.fi/ihte/CHI08_workshop/papers.shtml (last access on January 31, 2012)
Lallemand, C.: Toward a closer integration of usability in software development: a study of usability inputs in a model-driven engineering process. In: Proc. of EICS 2011, pp. 299–302. ACM (2011)
Rosenbaum, S., Rohn, J.A., Humburg, J.: A toolkit for strategic usability: Results from workshops, panels, and surveys. In: Proc. of CHI 2000, pp. 337–344. ACM (2000)
Seffah, A., Donyaee, M., Kline, R.B., Padda, H.K.: Usability measurement and metrics: A consolidated model. Software Quality Journal 14(2), 159–178 (2006)
Venturi, G.: Troost. J. 2004. Survey on the UCD integration in the industry. In: Proc. of NordiCHI 2004, pp. 449–452. ACM (2004)
Höök, K., Löwgren, J.: Strong concepts: Intermediate-level knowledge in interaction design research. ACM TOCHI 19(3), Article 23 (2012)
Ardito, C.: Usability of virtual reality systems. Thesis. University of Bari Aldo Moro, Italy (2002)
Sikorski, M., Garnik, I.: Towards Methodology for User Experience Measurement in on-line Services. In: Korczak, J., Dudycz, H., Dyczkowski, M. (eds.) Advanced Information Technologies for Management - AITM 2010, Research Papers of Wroclaw University of Economics, vol. (147) (2011)
Hertzum, M., Jacobsen, N.E.: The evaluator effect: a chilling fact about usability evaluation methods. Intern. Journal of Human–Computer Interaction 15(1), 183–204 (2003)
Ardito, C., Costabile, M.F., De Angeli, A., Lanzilotti, R.: Systematic evaluation of e-learning systems: an experimental validation. In: Proc. of NordiCHI 2006, pp. 195–202. ACM (2006)
Nielsen, J., Landauer, T.K.: A mathematical model of the finding of usability problems. In: Proc. of INTERCHI 1993, pp. 296–313. ACM (1993)
Vredenburg, K., Mao, J.Y., Smith, P.W., Carey, T.: A survey of user-centered design practice. In: Proc. of CHI 2002, pp. 471–478. ACM (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Ardito, C., Lanzilotti, R., Sikorski, M., Garnik, I. (2014). Can Evaluation Patterns Enable End Users to Evaluate the Quality of an e-learning System? An Exploratory Study. In: Stephanidis, C., Antona, M. (eds) Universal Access in Human-Computer Interaction. Universal Access to Information and Knowledge. UAHCI 2014. Lecture Notes in Computer Science, vol 8514. Springer, Cham. https://doi.org/10.1007/978-3-319-07440-5_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-07440-5_18
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07439-9
Online ISBN: 978-3-319-07440-5
eBook Packages: Computer ScienceComputer Science (R0)