A Taxonomy for Website Evaluation Tools Grounded on Semiotic Framework
Abstract
Taxonomies are valuable for providing a standardized way of cataloging elements into categories. In the context of website evaluation tools, providing a structured way for researchers and practitioners to compare and analyze existing solutions is valuable for identifying gaps/trends or to support well-informed decisions during development cycles (from planning to deployment). This paper proposes a taxonomy for classifying website evaluation tools grounded on Semiotic Framework, an artifact from Organizational Semiotics. The taxonomy is structured into 4 main dimensions (i.e., Participant-evaluator interaction; Effort; Automation type; Data source) and considers interaction and efforts involving UI evaluation stakeholders. From the proposed taxonomy, we expect to support consistent characterization of website evaluation tools.
Keywords
User interface evaluation Usability Accessibility Organizational SemioticsReferences
- 1.Abascal, J., Nicolle, C.: Moving towards inclusive design guidelines for socially and ethically aware HCI. Interact. Comput. 17(5), 484–505 (2005)CrossRefGoogle Scholar
- 2.Arroyo, E., Selker, T., Wei, W.: Usability tool for analysis of web designs using mouse tracks. In: Proceedings of ACM CHI 2006 Conference on Human Factors in Computing Systems. Work-in-Progress, vol. 2, pp. 484–489 (2006)Google Scholar
- 3.Balbo, S., Goschnick, S., Tong, D., Paris, C.: Leading web usability evaluations to WAUTER. In: AusWeb05 - Australian World Wide Web Conference 2005 (2005)Google Scholar
- 4.Bigham, J.P., Cavender, A.C., Brudvik, J.T., Wobbrock, J.O., Ladner, R.E.: WebinSitu: a comparative analysis of blind and sighted browsing behavior. In: Pontelli, E., Trewin, S. (eds.) ASSETS, pp. 51–58. ACM (2007)Google Scholar
- 5.Brajnik, G.: Using automatic tools in accessibility and usability assurance processes. In: Stary, C., Stephanidis, C. (eds.) UI4ALL 2004. LNCS, vol. 3196, pp. 219–234. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30111-0_18CrossRefGoogle Scholar
- 6.Cadez, I.V., Heckerman, D., Smyth, P., Meek, C., White, S.: Model-based clustering and visualization of navigation patterns on a web site. Data Mining Knowl. Discov. 7(4), 399–424 (2003)MathSciNetCrossRefGoogle Scholar
- 7.Carta, T., Paternò, F., de Santana, V.F.: Web usability probe: a tool for supporting remote usability evaluation of web sites. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds.) INTERACT 2011. LNCS, vol. 6949, pp. 349–357. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23768-3_29CrossRefGoogle Scholar
- 8.Chi, E.H., Rosien, A., Heer, J.: LumberJack: intelligent discovery and analysis of web user traffic composition. In: Zaïane, O.R., Srivastava, J., Spiliopoulou, M., Masand, B. (eds.) WebKDD 2002. LNCS (LNAI), vol. 2703, pp. 1–16. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-39663-5_1CrossRefGoogle Scholar
- 9.Cooley, R., Tan, P.-N., Srivastava, J.: Discovery of interesting usage patterns from web data. In: Masand, B., Spiliopoulou, M. (eds.) WebKDD 1999. LNCS (LNAI), vol. 1836, pp. 163–182. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-44934-5_10CrossRefGoogle Scholar
- 10.de Santana, V.F., Baranauskas, M.C.C., Henriques, M.A.A.: A framework for web 2.0 secure widgets. In: IADIS, pp. 69– 76 (2011)Google Scholar
- 11.Domenech, J.M., Lorenzo, J.: A tool for web usage mining. In: Yin, H., Tino, P., Corchado, E., Byrne, W., Yao, X. (eds.) IDEAL 2007. LNCS, vol. 4881, pp. 695–704. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-77226-2_70CrossRefGoogle Scholar
- 12.Etgen, M., Cantor, J.: What does getting WET (web event-logging tool) mean for web usability? In: Proceedings of 5th Conference on Human Factors & the Web (1999)Google Scholar
- 13.Fenstermacher, K.D., Ginsburg, M.: Mining client-side activity for personalization. In: WECWIS, pp. 205–212 (2002)Google Scholar
- 14.Google. Google analytics (2018). http://www.google.com/analytics
- 15.Hand, D., Mannila, H., Smyth, P.: Principles of Data Mining. MIT Press, Cambridge (2001)Google Scholar
- 16.Hartson, H.R., Castillo, J.C., Kelso, J., Neale, W.C.: Remote evaluation: the network as an extension of the usability laboratory. In: Proceedings of ACM CHI 96 Conference on Human Factors in Computing Systems. PAPERS: Evaluation, vol. 1, pp. 228–235 (1996)Google Scholar
- 17.Hong, J.I., Heer, J., Waterson, S., Landay, J.A.: WebQuilt: a proxy-based approach to remote web usability testing. ACM Trans. Inf. Syst. 19(3), 263–285 (2001)CrossRefGoogle Scholar
- 18.International Organization for Standardization ISO. Ergonomic requirements for office work with display terminals (VDTs). part 11: Guidance on usability (1998)Google Scholar
- 19.Ivory, M.Y., Hearst, M.A.: The state of the art in automating usability evaluation of user interfaces. ACM Comput. Surv. 33(4), 470–516 (2001)CrossRefGoogle Scholar
- 20.Liu, K.: Semiotics in Information Systems Engineering. Cambridge University Press, Cambridge (2000)CrossRefGoogle Scholar
- 21.National Institute of Standards and Technology NIST. WebVIP (2002). http://zing.ncsl.nist.gov/WebTools/WebVIP/overview.html
- 22.Paganelli, L., Paternò, F.: Intelligent analysis of user interactions with web applications. In: Proceedings of the 7th International Conference on Intelligent User Interfaces, IUI 2002, pp. 111–118, New York, NY, USA (2002)Google Scholar
- 23.Paternò, F., Piruzza, A., Santoro, C.: Remote web usability evaluation exploiting multimodal information on user behavior. In: Calvary, G., Pribeanu, C., Santucci, G., Vanderdonckt, J. (eds.) Computer-Aided Design of User Interfaces V, pp. 287–298. Springer, Dordrecht (2007). https://doi.org/10.1007/978-1-4020-5820-2_24CrossRefGoogle Scholar
- 24.Paternò, F., Santoro, C.: Remote usability evaluation: discussion of a general framework and experiences from research with a specific tool. In: Law, E.L.-C., Hvannberg, E.T., Cockton, G. (eds.) Maturing Usability. HIS, pp. 197–221. Springer, London (2008). https://doi.org/10.1007/978-1-84628-941-5_9CrossRefGoogle Scholar
- 25.Pierrakos, D., Paliouras, G., Papatheodorou, C., Spyropoulos, C.D.: Web usage mining as a tool for personalization: a survey. User Model. User-Adapt. Interact. 13(4), 311–372 (2003)CrossRefGoogle Scholar
- 26.Rubin, J.: Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests, 1st edn. Wiley, Hoboken (1994)Google Scholar
- 27.de Santana, V.F.: Identificação de padrões de utilização da web mediada por tecnologias assistivas. Master’s thesis, Instituto de Computação - Universidade Estadual de Campinas (UNICAMP), Supervisor: Profa. Maria Cecilia Calani Baranauskas (2009)Google Scholar
- 28.de Santana, V.F., Baranauskas, M.C.C.: A prospect of websites evaluation tools based on event logs. In: Forbrig, P., Paternò, F., Pejtersen, A.M. (eds.) HCIS 2008. IIFIP, vol. 272, pp. 99–104. Springer, Boston, MA (2008). https://doi.org/10.1007/978-0-387-09678-0_9CrossRefGoogle Scholar
- 29.de Santana, V.F., Baranauskas, M.C.C.: Bringing users of a digital divide context to website evaluation using welfit. In: Fatores Humanos em Sistemas Computacionais (IHC 2010). SBC (2010)Google Scholar
- 30.de Santana, V.F., Baranauskas, M.C.C.: WELFIT: a remote evaluation tool for identifying Web usage patterns through client-side logging. Int. J. Hum.-Comput. Stud. 76, 40–49 (2015)CrossRefGoogle Scholar
- 31.Shahabi, C., Banaei-Kashani, F.: A framework for efficient and anonymous web usage mining based on client-side tracking. In: Kohavi, R., Masand, B.M., Spiliopoulou, M., Srivastava, J. (eds.) WebKDD 2001. LNCS (LNAI), vol. 2356, pp. 113–144. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45640-6_6CrossRefzbMATHGoogle Scholar
- 32.Spiliopoulou, M., Faulstich, L.C.: WUM: a tool for web utilization analysis. In: Atzeni, P., Mendelzon, A., Mecca, G. (eds.) WebDB 1998. LNCS, vol. 1590, pp. 184–203. Springer, Heidelberg (1999). https://doi.org/10.1007/10704656_12CrossRefGoogle Scholar
- 33.Stamper, R.K.: A semiotic theory of information and information systems/applied semiotics. In: Invited papers for the ICL/University of Newcastle Seminar on ‘Information’, 6–10 September 1993Google Scholar
- 34.Stamper, R.K.: Signs of Work: Semiosis and Information Processing in Organisations. Walter de Gruyter, Berlin (1996)Google Scholar
- 35.Stamper, R.K.: Extending semiotics for the study of organisations. In: Proceedings of Conference on Semiotics and the Information Sciences (1998)Google Scholar
- 36.Tan, P.-N., Steinbach, M., Kumar, V.: Introduction to Data Mining. Person Education, London (2006)Google Scholar