Abstract
Human-Computer Interaction sees increased application of AI methods, particularly for testing and assessing the characteristics of graphic user interfaces (UIs). For instance, target users’ subjective perceptions of visual complexity, aesthetic impression, trust, etc. can be predicted to some extend based on UI appearance. Buttons, text blocks, images and other elements in today’s UIs at all platforms are usually aligned to grids – i.e. regular vertical and horizontal lines – in order to decrease visual clutter. However, the grids are not apparent in the UI representations available for analysis (HTML/CSS, etc.), unlike in design mockups, and have to be reverse-engineered for the benefit of further UI assessment. In our paper we propose the algorithm for automated construction of layout grids on top of visual representations (screenshots) of existing UIs and demonstrate its work with various configuration parameters. The algorithm was inspired by the informal Squint Test known from Usability Engineering practice and is based on subsequent application of several computer vision techniques supported by OpenCV library. The main stages are edge detection, image pixelization, grid overlaying, and cell coding. The tuning of the algorithm’s configuration parameters can be performed to match the UI perception by representative users, as demonstrated in the paper. The outcome of the algorithm is a coded representation of graphical UI as a 2D matrix, which is a convenient medium for further processing. The automation of UI layouts coding can allow obtaining large datasets needed by up-to-date user behavior models that predict the quality of interaction with UIs.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Machado, P., et al.: Computerized measures of visual complexity. Acta Physiol (Oxf.) 160, 43–57 (2015)
Miniukovich, A., Marchese, M.: Relationship between visual complexity and aesthetics of webpages. In Proceedings of the 2020 CHI Conference on Human Factors in Computer System, pp. 1–13 (2020)
Stickel, C., Ebner, M., Holzinger, A.: The XAOS metric–understanding visual complexity as measure of usability. In: Symposium of the Austrian HCI and Usability Engineering Group, pp. 278–290 (2010)
King, A.J., Lazard, A.J., White, S.R.: The influence of visual complexity on initial user impressions: testing the persuasive model of web design. Behav. Inf. Technol. 39(5), 497–510 (2020)
Bakaev, M., Avdeenko, T.: A quantitative measure for information transfer in human-machine control systems. In: Proceedings of the IEEE International Siberian Conference on Control and Communications (SIBCON), pp. 1–4 (2015)
Wu, O., Hu, W., Shi, L.: Measuring the visual complexities of web pages. ACM Trans. Web (TWEB) 7(1), 1 (2013)
Michailidou, E., et al.: Automated prediction of visual complexity of web pages: tools and evaluations. Int. J. Hum. Comput. Stud. 145, 102523 (2021)
Donderi, D.C.: Visual complexity: a review. Psychol. Bull. 132(1), 73 (2006)
Bakaev, M. et al.: Data compression algorithms in analysis of UI layouts visual complexity. In: Proceedings of the International Andrei Ershov Memorial Conference on Perspectives of System Informatics, pp. 167–184 (2019)
Thielsch, M.T., Haines, R., Flacke, L.: Experimental investigation on the effects of website aesthetics on user performance in different virtual tasks. PeerJ 7, e6516 (2019)
Alemerien, K., Magel, K. GUIEvaluator: A metric-tool for evaluating the complexity of graphical user interfaces. In: SEKE, pp. 13–18 (2014)
Miniukovich, A., de Angeli, A.: Quantification of interface visual complexity. In: Proceedings of the International Working Conference on Advanced Visual Interfaces, pp. 153–160 (2014)
Oulasvirta, A. et al.: Aalto interface metrics (AIM) a service and codebase for computational GUI evaluation. In: Adjunct Proceedings 31st Annual ACM Symposium on User Interface Software and Technology, pp. 16–19 (2018)
Kim, N.W., et al.: BubbleView: an interface for crowdsourcing image importance maps and tracking visual attention. ACM Trans. Comput.-Hum. Interact. (TOCHI), 24(5), 36 (2017)
Boychuk, E., Bakaev, M.: Entropy and compression based analysis of web user interfaces. In: Proceedings of International Conference on Web Engineering (ICWE), pp. 253–261 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 Springer Nature Switzerland AG
About this paper
Cite this paper
Bakaev, M., Shirokov, M. (2024). Algorithm for Mapping Layout Grids in User Interfaces: Automating the “Squint Test”. In: Yavorskiy, R., Cavalli, A.R., Kalenkova, A. (eds) Tools and Methods of Program Analysis. TMPA 2021. Communications in Computer and Information Science, vol 1559. Springer, Cham. https://doi.org/10.1007/978-3-031-50423-5_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-50423-5_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-50422-8
Online ISBN: 978-3-031-50423-5
eBook Packages: Computer ScienceComputer Science (R0)