A Study on Immediate Automatic Usability Evaluation of Web Application User Interfaces

  • Jevgeni MarenkovEmail author
  • Tarmo Robal
  • Ahto Kalja
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 615)


More and more web applications are being migrated from desktop platforms to mobile platforms. User experience is extremely different on desktop and portable devices. Changes in user interfaces (UI) could lead to severe violations of usability rules, e.g. changing the text color could lead to decrease of accessibility for users with low vision or cognitive impairments. Manual usability inspection methods are the approaches that help to verify the usability conformance to guidelines. Nevertheless, there are number of difficulties why the aforementioned approaches could not be always applied. The purpose of our research is to develop a conceptual model and corresponding framework including category specific metrics with methodology for immediate automatic usability evaluation of web application user interfaces during design and implementation phase. We address the gap between usability evaluation and development stage of user interface by providing immediate feedback to UI developers.


Web usability Usability guidelines Web user interface 



This research was supported by the Estonian Ministry of Research and Education institutional research grant no. IUT33-13.


  1. 1.
    Balagtas-Fernandez, F., Hussmann, H.: A methodology and framework to simplify usability analysis of mobile applications. In: 24th IEEE/ACM International Conference on Automated Software Engineering, pp. 520–524. IEEE Press, Washington (2009)Google Scholar
  2. 2.
    Kaasila, J., Ferreira, D., Kostakos, V., Ojala T.: Testdroid: automated remote UI testing on Android. In: Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia, Article No. 28. ACM, New York (2012)Google Scholar
  3. 3.
    Wetzlinger, W., Nedbal, D., Auinger, A., Grossauer, C., Holzmann, C., Lettner, F: Mobile usability testing requirements and their implementation in the automation engineering industry. In: Proceedings of the 12th International Conference on Advances in Mobile Computing and Multimedia (MoMM 2014), pp. 62–71. ACM, New York (2014)Google Scholar
  4. 4.
    Porat, T., Schclar, A., Shapira B.: MATE: a mobile analysis tool for usability experts. In: CHI 2013 Extended Abstracts on Human Factors in Computing Systems, pp. 265–270. ACM, New York (2013)Google Scholar
  5. 5.
    Dingli, A.: USEFul: a framework to mainstream web site usability. Int. J. Hum. Comput. Interact. 10–30 (2011)Google Scholar
  6. 6.
    Hussain, A., Ferneley, E.: Usability metric for mobile application: a goal question metric (GQM) approach. In: Proceedings of the 10th International Conference on Information Integration and Web-Based Applications and Services, pp. 567–570. ACM, New York (2008)Google Scholar
  7. 7.
    van Solingen, R., Berghout, E.: The Goal/Question/Metric Method: A Practical Guide for Quality Improvement of Software Development. McGraw Hill, New York (1999)Google Scholar
  8. 8.
    Moreno, A.M., Seffah, A., Capilla, R.: HCI practices for building usable software. Computer 46, 100–102 (2013)CrossRefGoogle Scholar
  9. 9.
    Borges, J.A., Morales, I., Rodríguez, N.J.: Guidelines for designing usable World Wide Web pages. In: Conference Companion on Human Factors in Computing Systems, pp. 277–278. ACM, New York (1996)Google Scholar
  10. 10.
    Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 249–256. ACM, New York (1990)Google Scholar
  11. 11.
    Yehuda, H., McGinn, J.: Coming to terms: comparing and combining the results of multiple evaluators performing heuristic evaluation. In: CHI 2007 Extended Abstracts on Human Factors in Computing Systems, pp. 1899–1904. ACM, New York (2007)Google Scholar
  12. 12.
    Ekşioğlu, M., Kiris, E., Çapar, B., Selçuk, M.N., Ouzeir, S.: Heuristic evaluation and usability testing: case study. In: Patrick Rau, P.L. (ed.) IDGD 2011. LNCS, vol. 6775, pp. 143–151. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  13. 13.
    Hollingsed, T., Novick, D.G.: Usability inspection methods after 15 years of research and practice. In: Proceedings of the 25th Annual ACM International Conference on Design of Communication, pp. 249–255. ACM, New York (2007)Google Scholar
  14. 14.
    Fernandez, A., Abrahao, S., Insfran, E.: Empirical validation of a usability inspection method for model-driven web development. J. Syst. Softw. 86, 161–186 (2013)CrossRefGoogle Scholar
  15. 15.
    Biasm, R.G.: The pluralistic usability walkthrough: coordinated empathies. In: The Pluralistic Usability Walkthrough, pp. 63–76 (1994)Google Scholar
  16. 16.
    Leporini, B., Paterno, F., Scorcia, A.: Flexible tool support for accessibility evaluation. Interact. Comput. 18, 869–890 (2006)CrossRefGoogle Scholar
  17. 17.
    Vuolle, M., Tiainen, M., Kallio, T., Vainio, T., Kulju, M., Wigelius, H.: Developing a questionnaire for measuring mobile business service experience. In: Proceedings of the 10th International Conference on Human Computer Interaction with Mobile Devices and Services, pp. 53–62. ACM, New York (2008)Google Scholar
  18. 18.
    Kantner, L., Sova, D.H., Rosenbaum, S.: Alternative methods for field usability research. In: Proceedings of the 21st Annual International Conference on Documentation, pp. 68–72. ACM, New York (2003)Google Scholar
  19. 19.
    Bak, J.O., Nguyen, K., Risgaard, P, Stage, J.: Obstacles to usability evaluation in practice: a survey of software development organizations. In: Proceedings of the 5th Nordic Conference on Human-Computer Interaction: Building Bridges, pp. 23–32. ACM, New York (2008)Google Scholar
  20. 20.
    Ivory, M.Y., Hearst, M.: The state of the art in automating usability evaluation of user interfaces. ACM Comput. Surv. 33, 470–516 (2001)CrossRefGoogle Scholar
  21. 21.
    Lecerof, A., Paterno, F.: Automatic support for usability evaluation. IEEE Trans. Softw. Eng. 24, 863–888 (1998)CrossRefGoogle Scholar
  22. 22.
    Web Content Accessibility Guidelines.
  23. 23.
    Nebeling, M., Matulic, F., Norrie, M.C.: Metrics for the evaluation of news site content layout in large-screen contexts. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1511–1520. ACM, New York (2011)Google Scholar
  24. 24.
    Nebeling, M., Speicher, M., Norrie, M.: W3touch: metrics-based web page adaptation for touch. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2311–2320. ACM, New York (2013)Google Scholar
  25. 25.
    Ivory, M.Y., Mankoff, J., Le, A.: Using automated tools to improve web site usage by users with diverse abilities. Inf. Technol. Soc. 1(3), 195–236 (2003)Google Scholar
  26. 26.
    Montero, F., González, P., Lozano, M., Vanderdonckt, J.: Quality modles for automated evaluation of websites usability and accessibility. In: Proceedings of the International COST294 Workshop on User Interface Quality Model, pp. 37–43. Interact, Rome (2005)Google Scholar
  27. 27.
    Aizpurua, A., Arrue, M., Vigo, M., Abascal, J.: Transition of accessibility evaluation tools to new standards. In: Proceedings of the 2009 International Cross-Disciplinary Conference on Web Accessibility, pp. 36–44. ACM, New York (2009)Google Scholar
  28. 28.
    Leporini, B., Paternó, F., Scorcia, A.: An environment for defining and handling guidelines for the web. In: Miesenberger, K., Klaus, J., Zagler, W.L., Karshmer, A.I. (eds.) ICCHP 2006. LNCS, vol. 4061, pp. 176–183. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  29. 29.
    Leporini, B., Paterno, F., Scorcia, A.: Flexible tool support for accessibility evaluation. Interact. Comput. 18(5), 869–890 (2006)CrossRefGoogle Scholar
  30. 30.
    Automatic Web Accessibility Tool.
  31. 31.
    Davis, P.A., Shipman, F.M.: Learning usability assessment models for web sites. In: Proceedings of the 16th International Conference on Intelligent User Interfaces, pp. 195–204. ACM, New York (2011)Google Scholar
  32. 32.
    Dingli, A., Cassar, S.: An intelligent framework for website usability. In: Advances in Human-Computer Interaction, Article No. 5 (2014)Google Scholar
  33. 33.
    Nielsen, J., Loranger, H.: Prioritizing Web Usability. New Riders, Berkeley (2006)Google Scholar
  34. 34.
    Dix, A., Finlay, J.F., Abowd, G.D., Beale, R.: Human-Computer Interaction. Pearson Education, Edinburgh (2004)zbMATHGoogle Scholar
  35. 35.
    Nielsen, J.: User interface directions for the web: user interface directions for the web. Mag. Commun. ACM 42(1), 65–72 (1999)CrossRefGoogle Scholar
  36. 36.
    Nielsen, J., Tahir, M.: Homepage Usability: 50 Websites Deconstructed. New Riders Publishing, Edinburgh (2001)Google Scholar
  37. 37.
  38. 38.
    Nielsen, J.: Usability Engineering. Morgan Kaufmann, London (1993)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Tallinn University of TechnologyTallinnEstonia

Personalised recommendations