Identifying Relevant Dimensions for the Quality of Web Mashups: An Empirical Study

  • Tihomir OrehovačkiEmail author
  • Cinzia Cappiello
  • Maristella Matera
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9731)


Web mashups available online today are very often characterized by a poor quality. Several researchers justify this aspect by considering the situational, short-living nature of these applications. We instead believe such a low quality is also due to the lack of suitable quality models. This paper presents a quality model that tries to capture the nature of Web mashups by focusing on their component-based nature and the added value that they are required to introduce with respect to their single constituents. A finite set of indicators and attributes was first determined by reviewing the literature. An analysis of data collected from domain experts revealed a relevance of performance variables at different levels of granularity. An empirical study was then carried out to assess which dimensions are the most relevant with respect to the mashup quality as perceived by users.


Web mashups Quality evaluation Model validation Relevance identification Empirical study 


  1. 1.
    Alonso-Ríos, D., Vázquez-García, A., Mosqueira-Rey, E., Moret-Bonillo, V.: Usability: a critical analysis and a taxonomy. Int. J. Hum. Comput. Interact. 26(1), 53–74 (2010)CrossRefGoogle Scholar
  2. 2.
    Bhattacherjee, A.: Understanding information systems continuance: an expectation-confirmation model. MIS Quart. 25(3), 351–370 (2001)CrossRefGoogle Scholar
  3. 3.
    Bovee, M., Srivastava, R., Mak, B.: A conceptual framework and belief-function approach to assessing overall information quality. Int. J. Intel. Syst. 18(1), 51–74 (2001)CrossRefzbMATHGoogle Scholar
  4. 4.
    Calero, C., Ruiz, J., Piattini, M.: Classifying Web metrics using the Web quality model. Online Inf. Rev. 29(3), 227–248 (2005)CrossRefGoogle Scholar
  5. 5.
    Cappiello, C., Daniel, F., Koschmider, A., Matera, M., Picozzi, M.: A quality model for mashups. In: Auer, S., Díaz, O., Papadopoulos, G.A. (eds.) ICWE 2011. LNCS, vol. 6757, pp. 137–151. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  6. 6.
    Cappiello, C., Daniel, F., Matera, M.: A quality model for mashup components. In: Gaedke, M., Grossniklaus, M., Díaz, O. (eds.) ICWE 2009. LNCS, vol. 5648, pp. 236–250. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  7. 7.
    Cappiello, C., Daniel, F., Matera, M., Pautasso, C.: Information quality in mashups. IEEE Internet Comput. 14(4), 14–22 (2010)CrossRefGoogle Scholar
  8. 8.
    Cappiello, C., Matera, M., Picozzi, M., Daniel, F., Fernandez, A.: Quality-aware mashup composition: issues, techniques and tools. In: Proceedings of the 8th International Conference on the Quality of Information and Communications Technology, pp. 10--19. IEEE, Lisbon (2012)Google Scholar
  9. 9.
    Cedillo, P., Fernandez, A., Insfran, E., Abrahão, S.: Quality of Web mashups: a systematic mapping study. In: Sheng, Q.Z., Kjeldskov, J. (eds.) ICWE Workshops 2013. LNCS, vol. 8295, pp. 66–78. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  10. 10.
    Cohen, J.: A power primer. Psychol. Bull. 112(1), 155–159 (1992)CrossRefGoogle Scholar
  11. 11.
    Daniel, F., Matera, M.: Mashups: Concepts, Models, and Architectures. Data-Centric Systems and Applications. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  12. 12.
    DeLone, W.H., McLean, E.R.: The DeLone and McLean model of information systems success: a ten-year update. J. Manag. Inf. Syst. 19(4), 9–30 (2003)Google Scholar
  13. 13.
    Fenton, N.E., Peeger, S.L.: Software Metrics: A Rigorous and Practical Approach. PWS Publishing, Boston (1997)Google Scholar
  14. 14.
    Insfran, E., Cedillo, P., Fernández, A., Abrahão, S., Matera, M.: Evaluating the usability of mashups applications. In: Proceedings of the 8th International Conference on the Quality of Information and Communications Technology, pp. 323–326. IEEE, Lisbon (2012)Google Scholar
  15. 15.
    ISO/IEC 25010: Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models (2011)Google Scholar
  16. 16.
    ISO/IEC 25012: Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) – Data quality model (2008)Google Scholar
  17. 17.
    Ivory, M.Y., Megraw, R.: Evolution of Web site design patterns. ACM Trans. Inf. Syst. 23, 463–497 (2005)CrossRefGoogle Scholar
  18. 18.
    Hassenzahl, M., Tractinsky, N.: User experience - a research agenda. Behav. Inf. Technol. 25(2), 91–97 (2006)CrossRefGoogle Scholar
  19. 19.
    Kan, S.H.: Metrics and Models in Software Quality Engineering. Addison-Wesley Longman Publishing Co., Boston (2002)zbMATHGoogle Scholar
  20. 20.
    Koschmider, A., Hoyer, V., Giessmann, A.: Quality metrics for mashups. In: Proceedings of the Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists, pp. 376–380. ACM, Bela-Bela (2010)Google Scholar
  21. 21.
    Law, E.L.-C., Van Schaik, P.: Modelling user experience – an agenda for research and practice. Interact. Comput. 22(5), 313–322 (2010)CrossRefGoogle Scholar
  22. 22.
    Lawshe, C.H.: A quantitative approach to content validity. Pers. Psychol. 28(4), 563–575 (1975)CrossRefGoogle Scholar
  23. 23.
    Lewis, J.R.: IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum. Comput. Inter. 7(1), 57–78 (1995)CrossRefGoogle Scholar
  24. 24.
    Liao, C., Palvia, P., Chen, J.-L.: Information technology adoption behavior life cycle: toward a Technology Continuance Theory (TCT). Int. J. Inf. Manage. 29(4), 309–320 (2009)CrossRefGoogle Scholar
  25. 25.
    Mavromoustakos, S., Andreou, A.S.: WAQE: a Web Application Quality Evaluation model. Int. J. Web Eng. Technol. 3, 96–120 (2007)CrossRefGoogle Scholar
  26. 26.
    Nielsen, J., Mack, R.L.: Usability Inspection Methods. Wiley, New York (1994)CrossRefGoogle Scholar
  27. 27.
    Olsina, L., Covella, G., Rossi, G.: Web quality. In: Mendes, E., Mosley, N. (eds.) Web Engineering: Theory and Practice of Metrics and Measurement for Web Development, pp. 109–142. Springer, Heidelberg (2005)Google Scholar
  28. 28.
    Olsina, L., Lew, P., Dieser, A., Rivera, B.: Updating quality models for evaluating new generation Web applications. J. Web Eng. 11(3), 209–246 (2012)Google Scholar
  29. 29.
    Olsina, L., Rossi, G.: Measuring Web application quality with WebQEM. IEEE Multimedia 9, 20–29 (2002)CrossRefGoogle Scholar
  30. 30.
    Orehovački, T.: Perceived quality of cloud based applications for collaborative writing. In: Pokorny, J., et al. (eds.) Information Systems Development – Business Systems and Services: Modeling and Development, pp. 575–586. Springer, Heidelberg (2011)Google Scholar
  31. 31.
    Orehovački, T.: Proposal for a set of quality attributes relevant for Web 2.0 application success. In: Proceedings of the 32nd International Conference on Information Technology Interfaces, pp. 319–326. IEEE Press, Cavtat (2010)Google Scholar
  32. 32.
    Orehovački, T., Babić, S.: Predicting students’ continuance intention related to the use of collaborative Web 2.0 applications. In: Proceedings of the 23rd International Conference on Information Systems Development, pp. 112–122. Faculty of Organization and Informatics, Varaždin (2014)Google Scholar
  33. 33.
    Orehovački, T., Babić, S., Jadrić, M.: Exploring the validity of an instrument to measure the perceived quality in use of Web 2.0 applications with educational potential. In: Zaphiris, P., Ioannou, A. (eds.) LCT 2014, Part I. LNCS, vol. 8523, pp. 192–203. Springer, Heidelberg (2014)Google Scholar
  34. 34.
    Orehovački, T., Bubaš, G., Kovačić, A.: Taxonomy of Web 2.0 applications with educational potential. In: Cheal, C., Coughlin, J., Moore, S. (eds.) Transformation in Teaching: Social Media Strategies in Higher Education, pp. 43–72. Informing Science Press, Santa Rosa (2012)Google Scholar
  35. 35.
    Orehovački, T., Granić, A., Kermek, D.: Evaluating the perceived and estimated quality in use of Web 2.0 applications. J. Syst. Softw. 86(12), 3039–3059 (2013)CrossRefGoogle Scholar
  36. 36.
    Orehovački, T., Granić, A., Kermek, D.: Exploring the quality in use of Web 2.0 applications: the case of mind mapping services. In: Harth, A., Koch, N. (eds.) ICWE 2011. LNCS, vol. 7059, pp. 266–277. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  37. 37.
    Orehovački, T., Granollers, T.: Subjective and objective assessment of mashup tools. In: Marcus, A. (ed.) DUXU 2014, Part I. LNCS, vol. 8517, pp. 340–351. Springer, Heidelberg (2014)Google Scholar
  38. 38.
    Pang, M., Suh, W., Hong, J., Kim, J., Lee, H.: A new Web site quality assessment model for the Web 2.0 Era. In: Murugesan, S. (ed.) Handbook of Research on Web 2.0, 3.0, and X.0: Technologies, Business, and Social Applications, pp. 387–410. IGI Global, Hershey (2010)Google Scholar
  39. 39.
    Rio, A., Brito e Abreu, F: Websites quality: Does it depend on the application domain? In: Proceedings of the 7th International Conference on the Quality of Information and Communications Technology, pp. 493–498. IEEE, Porto (2010)Google Scholar
  40. 40.
    Redman, T.C.: Data Quality for the Information Age. Artech House, Norwood (1996)Google Scholar
  41. 41.
    Sassano, R., Olsina, L., Mich, L.: Modeling content quality for the Web 2.0 and follow-on applications. In: Murugesan, S. (ed.) Handbook of Research on Web 2.0, 3.0, and X.0: Technologies, Business, and Social Applications, pp. 371–386. IGI Global, Hershey (2010)Google Scholar
  42. 42.
    Seffah, A., Donyaee, M., Kline, R.B., Padda, H.K.: Usability measurement and metrics: a consolidated model. Software Qual. J. 14(2), 159–178 (2006)CrossRefGoogle Scholar
  43. 43.
    Venkatesh, V., Bala, H.: Technology acceptance model 3 and a research agenda on interventions. Decis. Sci. 39(2), 273–315 (2008)CrossRefGoogle Scholar
  44. 44.
    Venkatesh, V., Thong, J.Y.L., Xu, X.: Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. MIS Q. 36(1), 157–178 (2012)Google Scholar
  45. 45.
    Webb, H.W., Webb, L.A.: SiteQual: an integrated measure of Web site quality. J. Enterp. Inf. Manage. 17(6), 430–440 (2004)CrossRefGoogle Scholar
  46. 46.
    World Wide Web Consortium: Web Content Accessibility Guidelines (WCAG) 2.0 (2008).

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Tihomir Orehovački
    • 1
    Email author
  • Cinzia Cappiello
    • 2
  • Maristella Matera
    • 2
  1. 1.Department of Information and Communication TechnologiesJuraj Dobrila University of PulaPulaCroatia
  2. 2.Dipartimento di Elettronica, Informazione e BioingegneriaPolitecnico di MilanoMilanItaly

Personalised recommendations