Abstract
The use of heuristics for the evaluation of interfaces is a well studied area. Currently there appear to be two main research areas in relation to heuristics: the analysis of methods to improve the effectiveness of heuristic evaluations; and the development of new heuristic sets for novel and specialised domains. This paper proposes an evidence based design approach to the development of domain specific heuristics and shows how this method was applied within the context of computer assisted assessment. A corpus of usability problems was created through a series of student surveys, heuristic evaluations, and a review of the literature. This corpus was then used to synthesise a set of domain specific heuristics for evaluating CAA applications. The paper describes the process, and presents a new set of heuristics for evaluating CAA applications.
Chapter PDF
Similar content being viewed by others
References
Cockton, G., Woolrych, A., Hindmarch, M.: Reconditioned Merchandise: Extending Structured Report Formats in Usability Inspection. In: CHI 2004, Vienna (2004)
Coyle, C.L., et al.: Heuristic Evaluations at Bell Labs: Analyses of Evaluator Overlap and Group Sessions. In: CHI 2007. ACM Press, San Jose (2007)
Korhonen, H., Koivisto, E.M.: Playability Heuristics for Mobile Games. In: MobileHCI 2006. ACM, Helsinki (2006)
Paddison, C., Englefield, P.: Applying heuristics to accessibility inspections. Interacting with Computers 16(2), 507–521 (2004)
Nielsen, J.: Enhancing the Explanatory Power of Usability Heuristics. In: Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence. ACM, Boston (1994)
Ling, C., Salvendy, G.: Extension of Heuristic evaluation method: a review and reaapraisal. International Journal of Ergonomics and Human Factors 27(3), 179–197 (2005)
Gray, W.D., Salzman, M.C.: Damaged Merchandise? Journal of Human-Computer Interaction 13(4), 203–262 (1998)
Somervell, J., McCrickard, D.: Better discount evaluation: illustrating how critical parameters support heuristic creation. Interacting with Computers 17(5), 592–612 (2005)
Hartson, H.R., Andre, T.S., Williges, R.C.: Criteria for Evaluating Usability Evaluation Methods. International Journal Human Computer Interaction 15(1), 145–181 (2003)
Squires, D., Preece, J.: Predicting quality in educational software: Evaluating for learning, usability and the synergy between them. Interacting with Computers 11, 467–483 (1999)
Bull, J., McKenna, C.: Blueprint for computer-assisted assessment, p. 168. Loughborough University (2001)
Parlangeli, O., Marchigiani, E., Bagnara, S.: Multimedia systems in distance education: effects of usability on learning. Interacting with Computers 12(1), 37–49 (1999)
Piguet, A., Peraya, D.: Creating web-integrated learning environments: An analysis of WebCT authoring tools in respect to usability. Australian Journal of Educational Technology 16(3), 302–314 (2000)
Berg, G.A.: Human-Computer Interaction (HCI) in Educational Environments: Implications of Understanding Computers as Media. Journal of Educational Multimedia and Hypermedia 9(4), 347–368 (2000)
Sim, G., Horton, M., Strong, S.: Interfaces for online assessment: friend or foe? In: 7th HCI Educators Workshop. LTSN, Preston (2004)
Davies, P.: There’s no confidence in multiple-choice testing. In: Sixth International Computer Assisted Assessment Conference, Loughborough (2002)
Nielsen, J., Mack, R.L.: Usability Inspection Methods. John Wiley & Sons, New York (1994)
Desurvire, H., Caplan, M., Toth, J.A.: Using heuristics to Evaluate the Playability of Games. In: CHI 2004. ACM, Vienne (2004)
Jacobsen, N.E., John, B.E.: The evaluator effect in usability studies: problem detection and severity judgements. In: Proceeding of the Human Factors and Ergonomics Society 42nd Annual Meeting. HFES, Chicago (1998)
Nielsen, J.: Usability Engineering. Morgan Kaufmann, San Francisco (1994)
Connell, I.W., Hammond, N.V.: Comparing usability evaluation principles with heuristics: problem instances versus problem types. In: Human-Computer Interaction - INTERACT 1999. IOS Press, Edinburgh (1999)
Embi, Z.C., Hussain, H.: Analysis of local and foreign edutainment products- and effort to implement the design framework for an edutainment environment in Malaysia. Journal of Computers in Mathematics and Science Teaching 24(1), 27–42 (2005)
Fernandes, G., Holmes, C.: Applying HCI to music related hardware. In: CHI 2002. ACM, Minneapolis (2002)
Sim, G., Holifield, P.: Computer Assisted Assessment: All those in favour tick here. In: World Conference on Educational Multimedia, Hypermedia and Telecommunications. AACE, Lugano (2004)
Sim, G., Holifield, P.: Piloting CAA:All aboard. In: 8th International Computer Assisted Assessment Conference, Loughborough (2004)
Sim, G., et al.: Heuristic Evaluations of Computer Assisted Assessment Environments. In: World Conference on Educational Multimedia, Hypermedia and Telecommunications. AACE, Vancouver (2007)
Conole, G., Warburton, B.: A review of computer-assisted assessment. ALT-J 13(1), 19–33 (2005)
Sim, G., Holifield, P., Brown, M.: Implementation of computer assisted assessment: lessons from the literature. ALT-J 12(3), 215–229 (2004)
Jeffries, R., Desurvire, H.: Usability Testing vs Heuristic Evaluation: Was there a contest? SIGCHI Bulletin 24(4), 39–41 (1992)
Lavery, D., Cockton, G.: Representing Predicted and Actual Usability Problems. In: Workshop on Representation in Interactive Software Development. Queen Mary and Westfield College, University of London (1997)
Sim, G., Read, J.C., Holifield, P.: Evaluating the user experience in CAA Environments: What affects user satisfaction? In: 10th International Computer Assisted Assessment Conference, Loughborough (2006)
Nielsen, J.: Finding usability problems through heuristic evaluation. In: Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, Monterey (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 IFIP International Federation for Information Processing
About this paper
Cite this paper
Sim, G., Read, J.C., Cockton, G. (2009). Evidence Based Design of Heuristics for Computer Assisted Assessment. In: Gross, T., et al. Human-Computer Interaction – INTERACT 2009. INTERACT 2009. Lecture Notes in Computer Science, vol 5726. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03655-2_25
Download citation
DOI: https://doi.org/10.1007/978-3-642-03655-2_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03654-5
Online ISBN: 978-3-642-03655-2
eBook Packages: Computer ScienceComputer Science (R0)