Skip to main content
SpringerLink
Log in

A fuzzy-based framework for evaluation of website design quality index

  • Published:
International Journal on Digital Libraries Aims and scope Submit manuscript

Abstract

An unrecognized significance of the web acts as a driving force for the massive and rapid growth of websites in each domain of social life. For making a successful website, it is necessary for developers to embrace appropriate web testing and evaluation methodology. Some valuable works in the past have striven to appraise the web applications quantitatively. Various parameters have been considered which are again sub-parameterized to measurable indicators. But their weighing criterion has not been appropriately taken into account according to the domain of the website. Also, the relative degrees of interactions among parameters have not been taken into consideration. The work presented in this paper aims at describing a framework, Quality Index Evaluation Method to gauge the design quality of a website in the form of index value. An automated tool has been designed and coded to measure the metrics quantitatively. A weighing technique based on Fuzzy-DEMATEL (Decision Making Trial and Evaluation Laboratory Method) has been applied on these metrics. Fuzzy trapezoidal numbers have been used for assessment of parameters and the final design quality index value. To verify the use of framework in different website domains, it has been exercised on eight academic (four institutional and four digital libraries), five informative and four commercial websites. The results have been validated through the most widely used method in literature, i.e., user judgment. Opinions of users for each website have been quantified and aggregated with fuzzy aggregation technique. Experimental results show that the proposed framework provides accurate and consistent results in very less time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Asano, Y., Koide, S., Iwayama, M., et al.: Constructing a site for publishing open data of the ministry of economy, trade, and industry. New Gener. Comput. 34(4), 341–366 (2016). https://doi.org/10.1007/s00354-016-0403-y

    Article  Google Scholar 

  2. Bai, B., Law, R., Wen, I.: The impact of website quality on customer satisfaction and purchase intentions: evidence from Chinese online visitors. Int. J. Hosp. Manag. 27(3), 391–402 (2008). https://doi.org/10.1016/j.ijhm.2007.10.008

    Article  Google Scholar 

  3. Calero, C., Ruiz, J., Piattini, M.: Classifying web metrics using the web quality model. Online Inf. Rev. 29(3), 227–248 (2005). https://doi.org/10.1108/14684520510607560

    Article  Google Scholar 

  4. Gregg, D.G., Walczak, S.: The relationship between website quality, trust and price premiums at online auctions. Electron. Commer. Res. 10(1), 1–25 (2010). https://doi.org/10.1007/s10660-010-9044-2

    Article  Google Scholar 

  5. Jones, C., Kim, S.: Influences of retail brand trust, off-line patronage, clothing involvement and website quality on online apparel shopping intention. Int. J. Consum. Stud. 34(6), 627–637 (2010). https://doi.org/10.1111/j.1470-6431.2010.00871.x

    Article  Google Scholar 

  6. Kaya, T., Kahraman, C.: A fuzzy approach to e-banking website quality assessment based on an integrated AHP-ELECTRE method. Technol. Econ. Dev. Econ. 17(2), 313–334 (2011)

    Article  Google Scholar 

  7. Aladwani, A.M., Palvia, P.C.: Developing and validating an instrument for measuring user-perceived web quality. Inf. Manag. 39(6), 467–476 (2002). https://doi.org/10.1016/S0378-7206(01)00113-6

    Article  Google Scholar 

  8. Cocquebert, E., Trentesaux, D., Tahon, C.: WISDOM: a website design method based on reusing design and software solutions. Inf. Softw. Technol. 52(12), 1272–1285 (2010). https://doi.org/10.1016/j.tourman.2010.02.008

    Article  Google Scholar 

  9. Éthier, J., Hadaya, P., Talbot, J., Cadieux, J.: Interface design and emotions experienced on B2C Web sites: empirical testing of a research model. Comput. Hum. Behav. 24(6), 2771–2791 (2008). https://doi.org/10.1016/j.chb.2008.04.004

    Article  Google Scholar 

  10. Huang, T.C.K., Huang, C.H.: An integrated decision model for evaluating educational web sites from the fuzzy subjective and objective perspectives. Comput. Educ. 55(2), 616–629 (2010). https://doi.org/10.1016/j.compedu.2010.02.022

    Article  MathSciNet  Google Scholar 

  11. Lee, S., Koubek, R.J.: The effects of usability and web design attributes on user preference for e-commerce web sites. Comput. Ind. 61(4), 329–341 (2010). https://doi.org/10.1016/j.compind.2009.12.004

    Article  Google Scholar 

  12. Şengel, E., Öncü, S.: Conducting preliminary steps to usability testing: investigating the website of Uludağ University. Proc. Soc. Behav. Sci. 2(2), 890 (2010). https://doi.org/10.1016/j.sbspro.2010.03.122

    Article  Google Scholar 

  13. Akbulut, Ö.E., Akbulut, K.: Web site designers’ opinions about the visual elements. Proc. Soc. Behav. Sci. 2(2), 1549–1553 (2010). https://doi.org/10.1016/j.sbspro.2010.03.233

    Article  Google Scholar 

  14. Moshagen, M., Thielsch, M.T.: Facets of visual aesthetics. Int. J. Hum Comput Stud. 68(10), 689–709 (2010). https://doi.org/10.1016/j.ijhcs.2010.05.006

    Article  Google Scholar 

  15. Zeng, L., Salvendy, G., Zhang, M.: Factor structure of web site creativity. Comput. Hum. Behav. 25(2), 568–577 (2009). https://doi.org/10.1016/j.chb.2008.12.023

    Article  Google Scholar 

  16. Caballero-Luque, A., Aragonés-Beltrán, P., García-Melón, M., Dema-Pérez, C.: Analysis of the alignment of company goals to web content using ANP. Int. J. Inform. Technol. Decis. Mak. 9(03), 419–436 (2010). https://doi.org/10.1142/S0219622010003889

    Article  MATH  Google Scholar 

  17. Cormany, D., Baloglu, S.: Medical travel facilitator websites: an exploratory study of web page contents and services offered to the prospective medical tourist. Tour. Manag. 32(4), 709–716 (2011). https://doi.org/10.1016/j.tourman.2010.02.008

    Article  Google Scholar 

  18. Xiaosong, L., Ye, L., Zizhou, F., Will, L.: A quantitative approach in heuristic evaluation of e-commerce websites. Int. J. Artif. Intell. Appl. (IJAIA) 9(1), (2018). https://arxiv.org/abs/1801.04829

  19. Gupta, S., Dubey, S.K.: Adaptability evaluation of E-commerce websites in Indian perspective. In: Singh, R., Choudhury, S., Gehlot, A. (eds.) Intelligent Communication, Control and Devices. Advances in Intelligent Systems and Computing, vol. 624, pp. 847–854. Springer, Singapore (2018). https://doi.org/10.1007/978-981-10-5903-2_88

    Chapter  Google Scholar 

  20. Cebi, S.: A quality evaluation model for the design quality of online shopping websites. Electron. Commer. Res. Appl. 12(2), 124–135 (2013). https://doi.org/10.1016/j.elerap.2012.12.001

    Article  Google Scholar 

  21. Mavromoustakos, S., Andreou, A.S.: WAQE: a web application quality evaluation model. Int. J. Web Eng. Technol. 3(1), 96–120 (2006). https://doi.org/10.1504/IJWET.2007.011529

    Article  Google Scholar 

  22. Mich, L., Franch, M., Gaio, L.: Evaluating and designing web site quality. IEEE Multimed. 10(1), 34–43 (2003). https://doi.org/10.1109/MMUL.2003.1167920

    Article  Google Scholar 

  23. Mich, L.: Evaluating website quality by addressing quality gaps: a modular process. In: (2014) IEEE International Conference on Software Science, Technology and Engineering (SWSTE), Jun 11, pp. 42–49, IEEE (2014). https://doi.org/10.1109/swste.2014.13

  24. Olsina, L., Rossi, G.: Measuring web application quality with WebQEM. IEEE Multimed. 9(4), 20–29 (2002). https://doi.org/10.1109/MMUL.2002.1041945

    Article  Google Scholar 

  25. Torrente, M.C.S., Prieto, A.B.M., GutiéRrez, D.A., De Sagastegui, M.E.A.: Sirius: a heuristic-based framework for measuring web usability adapted to the type of website. J. Syst. Softw. 86(3), 649–663 (2013). https://doi.org/10.1016/j.jss.2012.10.049

    Article  Google Scholar 

  26. Berners-Lee, T.: Web Index. http://theWebindex.org (2014). Accessed 11 July 2016

  27. Malhotra, R., Sharma, A.: A neuro-fuzzy classifier for website quality prediction. In: 2013 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 1274–1279 IEEE (2013). https://doi.org/10.1109/ICACCI.2013.6637361

  28. Alsmadi, I., Al-Taani, A.T., Zaid, N.A.: Web structural metrics evaluation. In: Developments in E-systems Engineering (DESE), pp. 225–230 IEEE (2010). https://doi.org/10.1109/dese.2010.43

  29. Banos, V., Manolopoulos, Y.: A quantitative approach to evaluate website archivability using the CLEAR + method. Int. J. Digit. Libr. 17(2), 119–141 (2016). https://doi.org/10.1007/s00799-015-0144-4

    Article  Google Scholar 

  30. Wu, H.H., Chen, H.K., Shieh, J.I.: Evaluating performance criteria of employment service outreach program personnel by DEMATEL method. Expert Syst. Appl. 37(7), 5219–5223 (2010). https://doi.org/10.1016/j.eswa.2009.12.068

    Article  Google Scholar 

  31. Wu, W.W., Lee, Y.T.: Developing global managers’ competencies using the fuzzy DEMATEL method. Expert Syst. Appl. 32(2), 499–507 (2007). https://doi.org/10.1016/j.eswa.2005.12.005

    Article  Google Scholar 

  32. Yang, J.L., Tzeng, G.H.: An integrated MCDM technique combined with DEMATEL for a novel cluster-weighted with ANP method. Expert Syst. Appl. 38(3), 1417–1424 (2011). https://doi.org/10.1016/j.eswa.2010.07.048

    Article  Google Scholar 

  33. Wu, H.H., Tsai, Y.N.: A DEMATEL method to evaluate the causal relations among the criteria in auto spare parts industry. Appl. Math. Comput. 218(5), 2334–2342 (2011). https://doi.org/10.1016/j.amc.2011.07.055

    Article  Google Scholar 

  34. Suo, W.L., Feng, B., Fan, Z.P.: Extension of the DEMATEL method in an uncertain linguistic environment. Soft. Comput. 16(3), 471–483 (2012). https://doi.org/10.1007/s00500-011-0751-y

    Article  Google Scholar 

  35. Chang, B., Chang, C.W., Wu, C.H.: Fuzzy DEMATEL method for developing supplier selection criteria. Expert Syst. Appl. 38(3), 1850–1858 (2011). https://doi.org/10.1016/j.eswa.2010.07.114

    Article  Google Scholar 

  36. DeMarco, T.: Management can Make Quality (IM) Possible. Cutter IT Summit, Boston (1999)

    Google Scholar 

  37. Lilburne, B., Prajwol, D., Khan, K.M., Khosrowpour, M.: Measuring quality metrics for web applications. In: Proceedings of the 15th Information Resources Management Association International Conference, Held in New Orleans, USA, 23–26 May (2004)

  38. ISO/IEC 9126:1991: Software product evaluation—quality characteristics and guidelines for their use. Geneva (1991)

  39. ISO 13407:1999: Human-centred design processes for interactive systems. Geneva (1999)

  40. ISO 9241-11:1998: Ergonomic requirements for office work with visual display terminals (VDTs)—Part 11: guidance on usability. Geneva (1998)

  41. ISO/IEC 25010:2011: Systems and software engineering—software product quality requirements and evaluation (SQuaRE)—software product quality and system quality in use models. Geneva (2011)

  42. ISO 9001:2015: Quality management system standard. Geneva (2015)

  43. Effendi, F.S., Alfina, I.: Quality evaluation of airline’s e-commerce website, a case study of AirAsia and Lion Air websites. In: 2014 International Conference on Advanced Computer Science and Information Systems (ICACSIS), pp. 125–128, IEEE (2014). https://doi.org/10.1109/ICACSIS.2014.7065833

  44. Joo, S., Lin, S., Lu, K.: A usability evaluation model for academic library websites: efficiency, effectiveness and learnability. J. Libr. Inform. Stud. 9(2), 11–26 (2011)

    Google Scholar 

  45. Santos, A.M.: Theoretical-Methodological proposal to evaluate the quality of educational websites to support education. In: Proceedings of the 3rd International Conference on Technological Ecosystems for Enhancing Multiculturality, pp. 397–401 ACM (2015). https://doi.org/10.1145/2808580.2808640

  46. Baloglu, S., Pekcan, Y.A.: The website design and internet site marketing practices of upscale and luxury hotels in Turkey. Tour. Manag. 27(1), 171–176 (2006). https://doi.org/10.1016/j.jbusres.2012.10.008

    Article  Google Scholar 

  47. Pranić, L., Garbin Praničević, D., Arnerić, J.: Hotel website performance: evidence from a transition country. Tour. Hosp. Manag. 20(1), 45–60 (2014)

    Google Scholar 

  48. Elkhani, N., Soltani, S., Bakri, A.: An effective model for evaluating website quality considering customer satisfaction and loyalty: evidence of airline websites. IJCSI Int. J. Comput. Sci. Issues 10(2), 109 (2013)

    Google Scholar 

  49. Law, R., Qi, S., Buhalis, D.: Progress in tourism management: a review of website evaluation in tourism research. Tour. Manag. 31(3), 297–313 (2010). https://doi.org/10.1016/j.tourman.2009.11.007

    Article  Google Scholar 

  50. Afonso, A.P., Lima, J.R., Cota, M.P.: A heuristic evaluation of usability of Web interfaces. In: 2012 7th Iberian Conference on Information Systems and Technologies (CISTI), 1–6 IEEE (2012)

  51. Schäfer, K., Kummer, T.F.: Determining the performance of website-based relationship marketing. Expert Syst. Appl. 40(18), 7571–7578 (2013). https://doi.org/10.1016/j.eswa.2013.07.051

    Article  Google Scholar 

  52. Kaur, A., Dani, D.: The web navigability structure of e-banking in India. Int. J. Inform. Technol. Comput. Sci. (IJITCS) 5(5), 29 (2013). https://doi.org/10.5815/ijitcs.2013.05.04

    Article  Google Scholar 

  53. Chinthakayala, K.C., Zhao, C., Kong, J., Zhang, K.: A comparative study of three social networking websites. World Wide Web 17(6), 1233–1259 (2014). https://doi.org/10.1007/s11280-013-0222-8

    Article  Google Scholar 

  54. Chu, S.C., Kim, Y.: Determinants of consumer engagement in electronic word-of-mouth (eWOM) in social networking sites. Int. J. Advert. 30(1), 47–75 (2011). https://doi.org/10.2501/IJA-30-1-047-075

    Article  Google Scholar 

  55. Kincl, T., Štrach, P.: Measuring website quality: asymmetric effect of user satisfaction. Behav. Inform. Technol. 31(7), 647–657 (2012). https://doi.org/10.1080/0144929X.2010.526150

    Article  Google Scholar 

  56. Suwawi, D.D.J., Darwiyanto, E., Rochmani, M.: Evaluation of academic website using ISO/IEC 9126. In: 2015 3rd International Conference on Information and Communication Technology (ICoICT), pp. 222–227 IEEE (2015). https://doi.org/10.1109/icoict.2015.7231426

  57. Lorca, P., de Andrés, J., Martínez, A.B.: Does web accessibility differ among banks? World Wide Web 19(3), 351–373 (2016). https://doi.org/10.1007/s11280-014-0314-0

    Article  Google Scholar 

  58. Xie, I., Joo, S., Matusiak, K.K.: Multifaceted evaluation criteria of digital libraries in academic settings: similarities and differences from different stakeholders. J. Acad. Libr. 44(6), 854–863 (2018). https://doi.org/10.1016/j.acalib.2018.09.002

    Article  Google Scholar 

  59. Asmaran, M.: Quantitative and qualitative evaluation of three search engines (Google, Yahoo, and Bing). Am. Sci. Res. J. Eng. Technol. Sci. (ASRJETS) 26(2), 97–106 (2016)

    Google Scholar 

  60. Ismail, A., Kuppusamy, K.S.: Accessibility of Indian universities’ homepages: an exploratory study. J. King Saud Univ. Comput. Inform. Sci. 30, 268–278 (2018). https://doi.org/10.1016/j.jksuci.2016.06.006

    Article  Google Scholar 

  61. Lin, C.L., Wu, W.W.: A fuzzy extension of the DEMATEL method for group decision making. Eur. J. Oper. Res. 156, 445–455 (2004)

    Article  Google Scholar 

  62. Lin, C.J., Wu, W.W.: A causal analytical method for group decision-making under fuzzy environment. Expert Syst. Appl. 34(1), 205–213 (2008). https://doi.org/10.1016/j.eswa.2006.08.012

    Article  MathSciNet  Google Scholar 

  63. Wei, G., Zhao, X., Wang, H.: An approach to multiple attribute group decision making with interval intuitionistic trapezoidal fuzzy information. Technol. Econ. Dev. Econ. 18(2), 317–330 (2012). https://doi.org/10.3846/20294913.2012.676995

    Article  Google Scholar 

  64. Zadeh, L.A.: The concept of a linguistic variable and its application to approximate reasoning. (Part II). Inform. Sci. 8, 301–357 (1975). https://doi.org/10.1016/0020-0255(75)90046-8

    Article  MATH  Google Scholar 

  65. Felix, A., Devadoss, A.V.: A fuzzy DEMATEL—trapezoidal structure for modeling cause and effect relationships of youth violence. Int. J. Data Min. Tech. Appl. 2, 292–303 (2013)

    Google Scholar 

  66. Pearson, J.M., Pearson, A.M.: An exploratory study into determining the relative importance of key criteria in web usability: a multi-criteria approach. J. Comput. Inform. Syst. 48(4), 115–127 (2008)

    Google Scholar 

  67. Rocha, A.: Framework for a global quality evaluation of a website. Online Inf. Rev. 36(3), 374–382 (2012). https://doi.org/10.1108/14684521211241404

    Article  Google Scholar 

  68. Tsabadze, T.: A method for aggregation of trapezoidal fuzzy estimates under group decision-making. Fuzzy Sets Syst. 266, 114–130 (2015). https://doi.org/10.1016/j.fss.2014.08.011

    Article  MathSciNet  MATH  Google Scholar 

  69. Ismailova, R., Inal, Y.: Accessibility evaluation of top university websites: a comparative study of Kyrgyzstan, Azerbaijan, Kazakhstan and Turkey. Univ. Access Inf. Soc. 17(2), 437–445 (2018). https://doi.org/10.1007/s10209-017-0541-0

    Article  Google Scholar 

  70. Chowdhury, G.: Sustainability of digital libraries: a conceptual model and a research framework. Int. J. Digit. Libr. 14(3–4), 181–195 (2014). https://doi.org/10.1007/s00799-014-0116-0

    Article  Google Scholar 

Download references

Acknowledgements

One of the authors gratefully acknowledges DST, New Delhi, for providing financial support to carry out this research work under PURSE.

Author information

Authors and Affiliations

  1. Department of Computer Engineering and Technology, Guru Nanak Dev University, Amritsar, Punjab, India

    Satinder Kaur

  2. IK Gujral Punjab Technical University, Kapurthala, Punjab, India

    Satinder Kaur

  3. Department of Computer Science and Engineering, Beant College of Engineering and Technology, Gurdaspur, Punjab, India

    Sunil Kumar Gupta

Authors
  1. Satinder Kaur
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sunil Kumar Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Satinder Kaur.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix: A1

See Tables 9, 10, 11, and 12.

Table 9 Aggregated direct relation matrices for sites of academic domain
Full size table
Table 10 Total relation matrices for sites of academic domain
Full size table
Table 11 Computed values for degrees of interaction weights for academic websites
Full size table
Table 12 Computation of total weight for each criterion and measures of criteria for four academic institutional sites
Full size table

Appendix: A2

2.1 User—questionnaire

User Profile

Name:

 

Age:

 

Gender:

 

Address for Correspondence:

 

E-mail ID:

 

Phone or Mobile No:

 

Organization in which he/she is working:

 

Designation:

 

Experience in using website:

 

2.2 Questionnaire for website users

This questionnaire has been designed for the evaluation of website in the domain of Academic, Commercial and Informative Sites. Each question has two parts; one part assesses the perceived quality of a specific feature by measuring the satisfaction level, whereas other part estimates the expected quality by appraising the level of importance. Please read the question properly and fill exactly the one circle of your answer with a black pen.

figure afigure a

See Table 13.

Table 13 Evaluation of perceived design quality index (DQIU) for four academic institutional sites
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kaur, S., Gupta, S.K. A fuzzy-based framework for evaluation of website design quality index. Int J Digit Libr 22, 15–47 (2021). https://doi.org/10.1007/s00799-020-00292-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00799-020-00292-6

Keywords

Search

Navigation