A Tool for Design-Time Usability Evaluation of Web User Interfaces

  • Jevgeni MarenkovEmail author
  • Tarmo Robal
  • Ahto Kalja
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10509)


The diversity of smartphones and tablet computers has become intrinsic part of modern life. Following usability guidelines while designing web user interface (UI) is an essential requirement for each web application. Even a minor change in UI could lead to usability problems, e.g. changing background or foreground colour of buttons could cause usability problems especially for people with disabilities. Empirical evaluation methods such as questionnaires and Card Sorting are effective in finding such problems. Nevertheless, these methods cannot be used widely when time, money and evaluators are scarce. The purpose of our work is to deliver a tool for design-time automatic evaluation of UI conformance to category-specific usability guidelines. The main contribution of this solution is enabling immediate cost-efficient and automatic web UI evaluation that conforms to available and set standards. This approach is being integrated into the Estonian eGovernment authority in order to automate usability evaluation of web applications.


Web usability Usability guidelines Web user interface 



This research was supported by the Estonian Ministry of Research and Education institutional research grant no. IUT33-13. Authors are very thankful to Estonian Information Systems Authority team for governmental portal consulting.


  1. 1.
    International Organization for Standardization: ISO 9241-210:2010 Ergonomics of human-system interaction Part 210: human-centred design process for interactive systems (2010)Google Scholar
  2. 2.
    Smith, S.L., Mosier, J.N.: Guidelines for Designing User Interface Software. Bed-ford, Mitre (1986)Google Scholar
  3. 3.
    Borges, J.A., Morales, I., Rodriguez, N.J.: Guidelines for designing usable World Wide Web pages. In: Conference on Human Factors in Computing Systems, pp. 277–278. ACM, New York (1996)Google Scholar
  4. 4.
    Mack, R.L., Nielsen, J.: Usability Inspection Methods. Wiley, New York (1994)Google Scholar
  5. 5.
    Kock, E., Biljon, J., Pretorius, M.: Usability evaluation methods: mind the gaps. In: Proceedings of the 2009 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists, pp. 122–131. ACM, New York (2009)Google Scholar
  6. 6.
    Ivory, M.Y., Hearst, M.: The state of the art in automating usability evaluation of user interfaces. ACM Comput. Surv. 33, 470–516 (2001)CrossRefGoogle Scholar
  7. 7.
    Bak, J.O., Nguyen, K., Risgaard, P, Stage, J.: Obstacles to usability evaluation in practice: a survey of software development organizations. In: Proceedings of the 5th Nordic conference on Human-computer interaction, pp 23–32. ACM, New York (2008)Google Scholar
  8. 8.
    Häkli, A.: Introducing user-centered design in a small-size software development organization. Helsinki University of Technology, Helsinki (2005)Google Scholar
  9. 9.
    Lizano, F., Sandoval, M.M., Bruun, A., Stage, J.: Is usability evaluation important: the perspective of novice software developers. In: The 27th International BCS Human Computer Interaction Conference, Article 31, British Computer Society, Swinton (2013)Google Scholar
  10. 10.
    Web Content Accessibility Guidelines.
  11. 11.
  12. 12.
    User Experience for Mobile Applications and Websites.
  13. 13.
  14. 14.
    Marenkov, J., Robal, T., Kalja, A.: A framework for improving web application user interfaces through immediate evaluation. In: Databases and Information Systems, pp. 283–296. IOS Press, Amsterdam (2016)Google Scholar
  15. 15.
    World Wide Web Consortium W3C, 2010b. Web Accessibility Initiative (WAI).
  16. 16.
    Dingli, A.: USEFul: a framework to mainstream web site usability. Int. J. Hum. Comput. Interact. 2, 10–30 (2011)Google Scholar
  17. 17.
    Dingli, A., Cassar, S.: An intelligent framework for website usability. Adv. Hum Comput Interact. (2014). Article 5Google Scholar
  18. 18.
    Schiavone, A.G., Paterno, F.: An extensible environment for guideline-based accessibility evaluation of dynamic Web applications. J. Univ. Access Inf. Soc. 14, 111–132 (2015)CrossRefGoogle Scholar
  19. 19.
    Leporini, B., Paterno, F., Scorcia, A.: Flexible tool support for accessibility evaluation. Interact. Comput. 18(5), 869–890 (2006)CrossRefGoogle Scholar
  20. 20.
    Davis, P.A., Shipman, F.M.: Learning usability assessment models for web sites. In: Proceedings of the 16th International Conference On Intelligent User Interfaces, pp. 195–204. ACM, New York (2011)Google Scholar
  21. 21.
    Boza, B.C., Schiaffino S., Teyseyre A., Godoy D.: An approach for knowledge discovery in a web usability context. In: Proceedings of the 13th Brazilian Symposium on Human Factors in Computing Systems, pp 393–396, Porto Alegre (2014)Google Scholar
  22. 22.
    Winckler, M.A.A., Freitas, C.M.D.S., De Lima, J.V.: Usability remote evaluation for WWW. In: CHI 2000 Extended Abstracts on Human Factors in Computing Systems (CHI EA 2000), pp. 131–132. ACM, New York (2000)Google Scholar
  23. 23.
    Yusop, N.S.M.Y., Grundy, J., Vasa, R.:Reporting usability defects: do reporters report what software developers need? In: Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering, pp. 1–10. ACM, New York (2016)Google Scholar
  24. 24.
    Davies, S., Roper, M.: What’s in a bug report? In: Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, Article 26. ACM, Torino (2014)Google Scholar
  25. 25.
    Yusop, N.S.M., Grundy, J., Vasa, R.: Reporting Usability Defects: Limitations of open source defect repositories and suggestions for improvement. In: Proceedings of ASWEC Australasian Software Engineering Conference, pp. 38–43. ACM, New York (2015)Google Scholar
  26. 26.
    OWL Web Ontology Language Guide,
  27. 27.
    Robal, T., Marenkov, J., Kalja, A.: Ontology design for automatic evaluation of web user interface usability. In: PICMET 2017 Conference: “Technology Management for Interconnected World”, Portland, USA. 9–13 July 2017 (2017, in press)Google Scholar
  28. 28.
    Horridge, M., Bechhofer, S.: The OWL API: A Java API for OWL ontologies. Semantic Web 2, 11–21 (2011)Google Scholar
  29. 29.
    Tsarkov, D., Horrocks, I.: FaCT ++ description logic reasoner: system description. In: Proceedings of the Third International Joint Conference, pp. 292–297. Springer, Heidelberg (2006)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Tallinn University of TechnologyTallinnEstonia

Personalised recommendations