Formative Evaluation for Complex Interactive Systems

  • Chris Roast
  • Elizabeth Uruchurtu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8278)


This paper reports upon the design and use of a lightweight evaluation method, especially designed to examine complex interactive systems. The approach is illustrated through a case study involving an interactive tool designed to help enable users examine large scale data arising from authentication activity in higher education institutes. The evaluation approach illustrated is to enable the lightweight assessment of usability issues within complex interactive systems and identifying opportunities for significant design improvements. Specifically we argue that this method benefits from capturing key generic factors that underpin the effectiveness of tools for working with complex data. The paper concludes by reflecting upon the effectiveness of the lightweight structured assessment approach and how it supports to formative evaluation.


Evaluation Cognitive Dimensions Complex Data Information Retrieval Innovation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Blackwell, A.F., Green, T.R.G.: A Cognitive Dimensions questionnaire optimised for users. In: Blackwell, A.F., Bilotta, E. (eds.) Proceedings of the Twelfth Annual Meeting of the Psychology of Programming Interest Group, pp. 137–152 (2000)Google Scholar
  2. 2.
    Blandford, A., Green, T.R.G., Connell, I.: Formalising an understanding of user-system misfits. In: Bastide, R., Palanque, P., Roth, J. (eds.) DSV-IS 2004 and EHCI 2004. LNCS, vol. 3425, pp. 253–270. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  3. 3.
    Dearden, A.M., Finlay, J.: Pattern languages in HCI: A critical review. Human Computer Interaction 21(1), 49–102 (2006), doi:10.1207/s15327051hci2101_3Google Scholar
  4. 4.
    Green, T.R.G., Blackwell, A.F.: Design for usability using Cognitive Dimensions. Tutorial Session at British Computer Society Conference on Human Computer Interaction (HCI 1998) (1998),
  5. 5.
    Green, T.R.G., Blandford, A.E., Church, L., Roast, C.R., Clarke, S.: Cognitive dimensions: Achievements, new directions, and open questions. Journal of Visual Languages and Computing 17(4), 328–365 (2006), doi:10.1016/j.jvlc.2006.04.004Google Scholar
  6. 6.
    Grudin, J.: The case against user interface consistency. CACM 32(10), 1164–1173 (1989), doi:10.1145/67933.67934CrossRefGoogle Scholar
  7. 7.
    Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: Chew, J.C., Whiteside, J. (eds.) Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 1990), pp. 249–256. ACM, New York (1990), doi:10.1145/97243.97281Google Scholar
  8. 8.
    Nielsen, J.: Usability Engineering. Academic Press, Boston (1993)MATHGoogle Scholar
  9. 9.
    Roast, C., Khazaei, B.: An investigation into the validation of formalised cognitive dimensions. In: Doherty, G., Blandford, A. (eds.) DSVIS 2006. LNCS, vol. 4323, pp. 109–122. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  10. 10.
    Roast, C., Dearden, A., Uruchurtu, E.: Using and utilizing an innovative media development tool. In: Proceedings of the 10th Brazilian Symposium on Human Factors in Computing Systems and the 5th Latin American Confernce on Human-Computer Interaction, pp. 145–156 (2011)Google Scholar
  11. 11.
    Roast, C., Uruchurtu, E., Dearden, A.: The programming-like-analysis of an innovative media tool. In: Psychology of Programming Interest Group Annual Conference. University of York (2011)Google Scholar
  12. 12.
    Scapin, L.D., Bastien, C.J.M.: Ergonomic criteria for evaluating the ergonomic quality of interactive systems. Behaviour and Information Technology 16, 220–231 (1997)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  • Chris Roast
    • 1
  • Elizabeth Uruchurtu
    • 1
  1. 1.Culture, Communication and Computing Research InstituteSheffield Hallam UniversitySheffieldUK

Personalised recommendations