Soft Computing

, Volume 14, Issue 8, pp 887–897 | Cite as

A quality evaluation methodology for health-related websites based on a 2-tuple fuzzy linguistic approach

  • J. M. Moreno
  • J. M. Morales del Castillo
  • C. Porcel
  • E. Herrera-ViedmaEmail author


Nowadays, the patients and physicians use the health-related websites as an important information source and, therefore, it is critical the quality evaluation of health- related websites. The quality assessment of health-related websites becomes especially relevant because their use imply the existence of a wide range of threats which can affect people’s health. Additionally, website quality evaluation can also contribute to maximize the exploitation of invested resources by organizations in the development of user-perceived quality websites. But there is not yet a clear and unambiguous definition of the concept of website quality and the debate about quality evaluation on the Web remains open. In this paper, we present a qualitative and user-oriented methodology for assessing quality of health-related websites based on a 2-tuple fuzzy linguistic approach. To identify the quality criteria set, a qualitative research has been carried out using the focus groups technique. The measurement method generates linguistic quality assessments considering the visitors’ judgements with respect to those quality criteria. The combination of the linguistic judgements is implemented without a loss of information by applying a 2-tuple linguistic weighted average operator. This methodology means an improvement on quality evaluation of health websites through the commitment to put users first.


Quality evaluation Health websites Computing with words Linguistic modelling 



This work has been supported by the research project EMCA08-05 from “Programa EMCA”, Murcia, Spain. It has also been developed with the financing of FEDER funds in FUZZYLING project (TIN2007-61079), PETRI project (PET2007-0460), and project of Ministry of Public Works (90/07).


  1. Aladwani A, Palvia P (2002) Developing and validating an instrument for measuring user-perceived web quality. Inform Manag 39:467–476CrossRefGoogle Scholar
  2. Arfi B (2005) Fuzzy decision making in politics: A linguistic fuzzy-set approach (lfsa). Political Anal 13(1):23–56CrossRefMathSciNetGoogle Scholar
  3. Bedell S, Agrawal A, Petersen L (2004) A systematic critique of diabetes on the world wide web for patients and their physicians. Int J Med Inform 73:687–694CrossRefGoogle Scholar
  4. Berland G, Elliott M, Morales L, Algazy J (2001) Health information on the internet: Accessibility, quality, and readability in english and spanish. J Am Med Assoc 285(20):2612–2621CrossRefGoogle Scholar
  5. Bernhardt J, Felter E (2004) Online pediatric information seeking among mothers of young children: results from a qualitative study using focus groups. J Med Internet Res 6:e7CrossRefGoogle Scholar
  6. Breckons M, Jones R, Morris J, Richardson J (2008) What do evaluation instruments tell us about the quality of complementary medicine information on the internet? J Med Internet Res 10(1):e3. doi: 10.2196/jmir.961 CrossRefGoogle Scholar
  7. Childs S (2004) Developing health web site quality assessment guidelines for the voluntary sector: outcomes from the judge project. Health Inform Libr 21:14–26CrossRefGoogle Scholar
  8. Childs S (2005) Judging the quality of internet-based health information. Perform Meas Metr 6(2):80–96CrossRefGoogle Scholar
  9. Christensen H, Griffiths K (2000) Sites for depression on the web: a comparison of consumer, professional and commercial sites. Aust N Z J Public Health (24):396–400Google Scholar
  10. Coiera E (1998) Information epidemics, economics, and immunity on the internet. Br Med J 317:1469–1470Google Scholar
  11. Cooke A, Gray L (2002) Evaluating the quality of internet-based information about alternative therapies: development of the biome guidelines. J Public Health Med 24:261–267CrossRefGoogle Scholar
  12. Craigie M, Loader B, Burrows R, Muncer S (2002) Realibility of health information on the internet: an examination of expert ratings. J Med Internet Res 4(1):e2CrossRefGoogle Scholar
  13. Dhyani D, Keong Ng W, Bhowmick S (2002) A survey of web metrics. ACM Comput Surv 34(4):469–503CrossRefGoogle Scholar
  14. Diering C, Palmer M (2001) Professional information about urinary incontinence on the world wide web. J Wound Ostomy Cont Nurs 28:55–62Google Scholar
  15. Eysenbach G (2003) The impact of the internet on cancer outcomes. CA Cancer J Clin (53):356–371Google Scholar
  16. Eysenbach G, Ryoung E, Diepgen T (1999) Shopping around the internet today and tomorrow: towards the millennium of cybermedicine. Br Med J 319:1294Google Scholar
  17. Eysenbach G, Powell J, Kuss O, Sa E (2002) Empirical studies assessing the quality of health information for consumers on the world wide web. A systematic review. J Am Med Assoc 287(20):2691–2700CrossRefGoogle Scholar
  18. Fox S, Rainie L (2003) The online health care revolution: how the web helps Americans take care of themselves. Pew Internet and American Life Project, Washington DCGoogle Scholar
  19. Galimberti A, Jain S (2000) Gynaecology on the net: evaluation of the information on hysterectomy contained in health-related web sites. J Obstet Gynaecol 20:297–299CrossRefGoogle Scholar
  20. Griffiths K, Christensen H (2000) Quality of web based information on treatment of depression: cross sectional survey. Br Med J 321:1511–1515CrossRefGoogle Scholar
  21. Grigoroudis E, Litos C, Moustakis V, Politis Y, Tsironis L (2008) The assessment of user-perceived web quality: application of a satisfaction benchmarking approach. Eur J Oper Res187:1346–1357zbMATHCrossRefGoogle Scholar
  22. Herrera F, Herrera-Viedma E (1997) Aggregation operators for linguistic weighted information. IEEE Trans Syst Man Cybern Part A Syst Hum 27:646–656CrossRefGoogle Scholar
  23. Herrera F, Herrera-Viedma E (2000) Linguistic decision analysis: steps for solving decisions problems under linguistic information. Fuzzy Sets Syst 115:67–82zbMATHCrossRefMathSciNetGoogle Scholar
  24. Herrera F, Martínez L (2000) A 2-tuple fuzzy linguistic representation model for computing with words. IEEE Trans Fuzzy Syst 8(6):746–752CrossRefGoogle Scholar
  25. Herrera F, Herrera-Viedma E, Verdegay J (1996) Direct approach processes in group decision making using linguistic owa operators. Fuzzy Sets Syst 79:175–190zbMATHCrossRefMathSciNetGoogle Scholar
  26. Herrera-Viedma E (2001) An information retrieval system with ordinal linguistic weighted queries based on two weighting elements. Int J Uncertain Fuzziness Knowl Based Syst 9:77–88zbMATHCrossRefMathSciNetGoogle Scholar
  27. Herrera-Viedma E, Peis E (2003) Evaluating the informative quality of documents in sgml-format using fuzzy linguistic techniques based on computing with words. Inform Process Manag 39(2):195–213CrossRefGoogle Scholar
  28. Herrera-Viedma E, Pasi G, Lopez-Herrera A, Porcel C (2006) Evaluating the information quality of web sites: a methodology based on fuzzy computing with words. J Am Soc Inform Sci Technol 57(4):538–549CrossRefGoogle Scholar
  29. Huizingh E (2000) The content and design of web sites: an empirical study. Inform Manag 37(3):123–134CrossRefGoogle Scholar
  30. Impicciatore P, Pandolfini C, Casella N, Bonati M (1997) Reliability of health information for the public on the world wide web. Br Med J 314:1875–1879Google Scholar
  31. Ivory M (2003) Automated web site evaluation. Kluwer, DordrechtGoogle Scholar
  32. Katerattanakul P, Siau K (1999) Measuring information quality of web sites: development of an instrument. In: Proceedings of 20th international conference on information systems, pp 279–285Google Scholar
  33. Kim P, Eng T, Deering M, Maxfield A (1999) Published criteria for evaluating health-related web sites: a review. Br Med J 318:647–649Google Scholar
  34. Kirakowski J, Cierlik B (1998) Measuring the usability of web sites. In: Proceeding of human factors and ergonomics society 42nd annual meeting, Santa Monica, CAGoogle Scholar
  35. Kitzinger J (1994) The methodology of focus groups: the importance of interactions between research participants. Sociol Health Illn 16:103–121CrossRefGoogle Scholar
  36. Kitzinger J (1995) Introducing focus groups. Br Med J 311:299–302Google Scholar
  37. Korgaonkar P, Wolin L (1999) A multivariate analysis of web usage. J Advert Res 39:53–68Google Scholar
  38. Krueger R (1997) Developing questions for focus groups (focus group kit). Sage, Thousand OaksGoogle Scholar
  39. Li L, Irvin E, Guzman J, Bombardier C (2001) Surfing for back pain patients: the nature and quality of back pain information on the internet. Spine 26:545–557CrossRefGoogle Scholar
  40. Lissman T, Boehnlein JK (2001) A critical review of internet information about depression. Psychiatr Serv 52(8):1046–1050CrossRefGoogle Scholar
  41. Marshall L, Willians D (2006) Health information: does quality count for the consumer? how consumers evaluate the quality of health information materials across a variety of media. J Librariansh Inform Sci 38:141–156CrossRefGoogle Scholar
  42. Meric F, Bernstam E, Mirza N, Hunt K, Ames F, Ross M, Kuerer H, Pollock R, Musen M, Singletary S (2002) Breast cancer on the world wide web: cross sectional survey of quality of information and popularity of websites. Br Med J 324:577–581CrossRefGoogle Scholar
  43. Morgan D (1997) The focus groups guidebook (focus group kit). Sage, Thousand OaksGoogle Scholar
  44. Negash S, Ryan T, Igbaria M (2003) Quality and effectiveness in web-based customer support systems. Inform Manag 40:757–768CrossRefGoogle Scholar
  45. Olsina L, Rossi G (2002) Measuring web application quality with webqem. IEEE Multimed October–December:20–29Google Scholar
  46. Olsina L, Covella G, Rossi G (2006) Web engineering. In: Mendes E, Mosley N (eds) Web quality. Springer, BerlinGoogle Scholar
  47. Pandolfini C, Impicciatore P, Bonati M (2000) Parents on the web: risks for quality management of cough in children. Pediatrics 105:e1CrossRefGoogle Scholar
  48. Powell T, Jones D, Cutts D (1998) Web site engineering: beyond web page design. Prentice Hall, Englewood CliffsGoogle Scholar
  49. Provost M, Koompalum D, Dong D, Martin B (2006) The initial development of the webmedqual scale: domain assessment of the construct of quality of health web sites. Int J Med Inform 75:42–57CrossRefGoogle Scholar
  50. Quintana Y, Feightner J, Wathen C, Sangster L, Marshall J (2001) Preventive health information on the internet—qualitative study of consumers’ perspectives. Can Fam Phys 47:1759–1765Google Scholar
  51. Rieh S (2002) Judgment of information quality and cognitive authority in the web. J Am Soc Inform Sci Technol 53(2):145–161CrossRefGoogle Scholar
  52. Risk A, Dzenowagis J (2001) Review of internet health information quality initiatives. J Med Internet Res 3(4):e28CrossRefGoogle Scholar
  53. Sellito C, Burgess S (2005) Towards a weighted average framework for evaluating the quality of web-located health information. J Inform Sci 31(4):260–272CrossRefGoogle Scholar
  54. Sieving P (1999) Factors driving the increase in medical information on the web—one american perspective. J Med Internet Res 1(1):e3CrossRefGoogle Scholar
  55. Stavri P, Freeman D, Burroughs C (2003) Perception of quality and trustworthiness of internet resources by personal health information seekers. In: AMIA symposium proceedings, pp 629–633Google Scholar
  56. Stewart D, Shamdasani P (1990) Focus groups: theory and practice (applied social research methods). Sage, Newbury ParkGoogle Scholar
  57. Yang Z, Cai S, Zhou Z, Zhou N (2005) Development and validation of an instrument to measure user perceived service quality of information presenting web portals. Inform Manag 42:575–589CrossRefGoogle Scholar
  58. Zadeh L (1975) The concept of a linguistic variable and its applications to approximate reasoning. Inform Sci Part I, II, III 8, 8, 9:199–249, 301–357, 43–80Google Scholar
  59. Zhang P, von Dran G (2001) User expectations and rankings of quality factors in different website domains. Int J Electron Commer 6(2):9–33Google Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  • J. M. Moreno
    • 1
  • J. M. Morales del Castillo
    • 2
  • C. Porcel
    • 3
  • E. Herrera-Viedma
    • 2
    Email author
  1. 1.Department of Information and Communication EngineeringUniversity of MurciaMurciaSpain
  2. 2.Department of Computer Science and A.IUniversity of GranadaGranadaSpain
  3. 3.Department of Computer ScienceUniversity of JaénJaénSpain

Personalised recommendations