Medical & Biological Engineering & Computing

, Volume 55, Issue 4, pp 527–536 | Cite as

Evaluation of an automatic dry eye test using MCDM methods and rank correlation

  • Diego Peteiro-Barral
  • Beatriz Remeseiro
  • Rebeca Méndez
  • Manuel G. Penedo
Original Article


Dry eye is an increasingly common disease in modern society which affects a wide range of population and has a negative impact on their daily activities, such as working with computers or driving. It can be diagnosed through an automatic clinical test for tear film lipid layer classification based on color and texture analysis. Up to now, researchers have mainly focused on the improvement of the image analysis step. However, there is still large room for improvement on the machine learning side. This paper presents a methodology to optimize this problem by means of class binarization, feature selection, and classification. The methodology can be used as a baseline in other classification problems to provide several solutions and evaluate their performance using a set of representative metrics and decision-making methods. When several decision-making methods are used, they may offer disagreeing rankings that will be solved by conflict handling in which rankings are merged into a single one. The experimental results prove the effectiveness of the proposed methodology in this domain. Also, its general purpose allows to adapt it to other classification problems in different fields such as medicine and biology.


Dry eye syndrome Image analysis Pattern recognition Multiple criteria decision-making Rank correlation 



This research has been partially funded by the Secretaría de Estado de Investigación of the Spanish Government and FEDER funds of the European Union through the research projects TIN2012-37954 and PI14/02161; and by the Consellería de Industria of the Xunta de Galicia through the research projects GPC2013/065 and GRC2014/035. We would also like to thank the Optometry Service of the University of Santiago de Compostela (Spain) for providing us with the annotated dataset.


  1. 1.
    Wolff E (1954) Anatomy of the eye and orbit, 4th edn. H. K. Lewis and Co., LondonGoogle Scholar
  2. 2.
    Nichols KK, Nichols JJ, Mitchell GL (2004) The lack of association between signs and symptoms in patients with dry eye disease. Cornea 23(8):762–770CrossRefPubMedGoogle Scholar
  3. 3.
    Rolando M, Refojo MF, Kenyon KR (1983) Increased tear evaporation in eyes with keratoconjunctivitis sicca. Arch Ophthalmol 101:557–558CrossRefPubMedGoogle Scholar
  4. 4.
    Lemp MA (1998) Epidemiology and classification of dry eye. Adv Exp Med Biol 438:791–803CrossRefPubMedGoogle Scholar
  5. 5.
    Guillon J (1998) Non-invasive tearscope plus routine for contact lens fitting . Contact Lens Anterior Eye 21(1):S31–S40CrossRefPubMedGoogle Scholar
  6. 6.
    King-Smith P, Finkd B, Fogt N (1999) Three interferometric methods for measuring the thickness of layers of the tear film. Optom Vis Sci 76:19–32CrossRefPubMedGoogle Scholar
  7. 7.
    Goto E, Yagi Y, Kaido M, Matsumoto Y, Konomi K, Tsubota K (2003) Improved functional visual acuity after punctal occlusion in dry eye patients. Am J Ophthalmol 135(5):704–705CrossRefPubMedGoogle Scholar
  8. 8.
    Ramos L, Penas M, Remeseiro B, Mosquera A, Barreira N, Yebra-Pimentel E (2011) Texture and color analysis for the automatic classification of the eye lipid layer. In: LNCS: advances in computational interlligence (international work conference on artificial neural networks, IWANN’11), vol 6692. pp 66–73Google Scholar
  9. 9.
    Remeseiro B, Penas M, Barreira N, Mosquera A, Novo J, García-Resúa C (2013) Automatic classification of the interferential tear film lipid layer using colour texture analysis. Comput Methods Programs Biomed 111:93–103CrossRefPubMedGoogle Scholar
  10. 10.
    Remeseiro B, Penas M, Mosquera A, Novo J, Penedo MG, Yebra-Pimentel E (2012) Statistical comparison of classifiers applied to the interferential tear film lipid layer automatic classification. Comput Math Methods Med 1–10:2012Google Scholar
  11. 11.
    Remeseiro B, Boló-Canedo V, Peteiro-Barral D, Alonso-Betanzos A, Guijarro-Berdinas B, Mosquera A, Penedo MG, Sánchez-Marono N (2014) A methodology for improving tear film lipid layer classification. IEEE J Biomed Health Inform 18(4):1485–1493CrossRefPubMedGoogle Scholar
  12. 12.
    Rebeca Méndez, Remeseiro B, Peteiro-Barral D, Penedo MG (2014) Evaluation of class binarization and feature selection in tear film classification using topsis. CCIS: agents and artificial intelligence. Rev Sel Pap ICAART 2013 449:179–193Google Scholar
  13. 13.
    Rokach L (2010) Ensemble-based classifiers. Artif Intell Rev 33(1–2):1–39CrossRefGoogle Scholar
  14. 14.
    Wei G-W (2010) Extension of TOPSIS method for 2-tuple linguistic multiple attribute group decision making with incomplete weight information. Knowl Inf Syst 25(3):623–634CrossRefGoogle Scholar
  15. 15.
    Laplante A (2009) Using the analytical hierarchy process in selecting commercial real-time operating systems. Int J Inf Technol Decis Mak 8(01):151–168CrossRefGoogle Scholar
  16. 16.
    Peng Yi, Wang G, Wang H (2012) User preferences-based software defect detection algorithms selection using MCDM. Inf Sci 191:3–13CrossRefGoogle Scholar
  17. 17.
    Wu D, Boyer KL, Nichols JJ, King-Smith PE (2010) Texture-based prelens tear film segmentation in interferometry images. Mach Vis Appl 21(3):253–259CrossRefGoogle Scholar
  18. 18.
    Ramos L, Barreira N, Mosquera A, Penedo MG, Yebra-Pimentel E, García-Resúa C (2014) Analysis of parameters for the automatic computation of the tear film break-up time test based on CCLRU standards. Comput Methods Programs Biomed 113(3):715–724CrossRefPubMedGoogle Scholar
  19. 19.
    Carpente A, Ramos L, Barreira N, Penedo MG, Pena-Verdeal H and Giráldez MJ (2014) On the automation of the tear film non-invasive break-up test. In: 2nd International symposium on computer-based medical systems (CBMS), pp 185–188Google Scholar
  20. 20.
    Guillon J, Guillon M (1997) Tearscope Plus Clinical Handbook and Tearscope Plus Instructions Keeler Ltd, Windsor, Berkshire, Keeler Inc, Broomall, PAGoogle Scholar
  21. 21.
    Calvo D, Mosquera A, Penas M, García-Resúa C, Remeseiro B (2010) Color texture analysis for tear film classification: a preliminary study. LNCS Int Conf Image Anal Recognit (ICIAR) 6112:388–397CrossRefGoogle Scholar
  22. 22.
    McLaren K (1976) The development of the CIE 1976 (L*a*b) uniform colour-space and colour-difference formula. J Soc Dyers Colour 92(9):338–341CrossRefGoogle Scholar
  23. 23.
    Bradski G (2000) The OpenCV Library. Dr. Dobb’s J 25(11):120–126Google Scholar
  24. 24.
    Haralick Robert M, Shanmugam K, Its’Hak Dinstein (1973) Texture features for image classification. IEEE Trans Syst Man Cybernet Syst Man Cybernet 3:610–621CrossRefGoogle Scholar
  25. 25.
    VOPTICAL_I1, VARPA optical dataset acquired and annotated by optometrists from the Optometry Service of the University of Santiago de Compostela (Spain), 2012. Accessed Apr 2016
  26. 26.
    Furnkranz J (2003) Round robin ensembles. Intell Data Anal 7(5):385–403Google Scholar
  27. 27.
    Dietterich TG and Bakiri G (1995) Solving multiclass learning problems via error-correcting output codes. Arxiv preprint arXiv:9501101
  28. 28.
    Dietterich TG, Bakiri G (1995) Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 2:263–286Google Scholar
  29. 29.
    Allwein EL, Schapire RE, Singer Y (2001) Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1:113–141Google Scholar
  30. 30.
    Guyon I, Gunn S, Nikravesh M, Zadeh L (2006) Feature extraction: foundations and applications. Springer, New YorkCrossRefGoogle Scholar
  31. 31.
    Manning Christopher D, Prabhakar R, Hinrich S (2008) Introduction to information retrieval, vol 1. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  32. 32.
    Loughrey J, Cunningham P (2005) Overfitting in wrapper-based feature subset selection: the harder you try the worse it gets. Res Dev Intell Syst XXI:33–43CrossRefGoogle Scholar
  33. 33.
    Bolón-Canedo V, Sánchez-Maroño N, Alonso-Betanzos A (2011) On the behavior of feature selection methods dealing with noise and relevance over synthetic scenarios. In: The 2011 international joint conference on neural networks (IJCNN), pp 1530–1537Google Scholar
  34. 34.
    Hall MA (1999) Correlation-based feature selection for machine learning. PhD thesis, The University of WaikatoGoogle Scholar
  35. 35.
    Dash M, Liu H (2003) Consistency-based search in feature selection. Artif Intell 151(1–2):155–176CrossRefGoogle Scholar
  36. 36.
    Zhao Z, Liu H (2007) Searching for interacting features. In: Proceedings of the 20th international joint conference on Artifical intelligence. pp 1156–1161Google Scholar
  37. 37.
    Mitchell T (1997) Machine learning. McGraw-HillGoogle Scholar
  38. 38.
    Kotsiantis SB (2007) Supervised machine learning: a review of classification techniques. Informatica 31:249–268Google Scholar
  39. 39.
    Friedman JH (1989) Regularized discriminant analysis. J Am Stat Asso 84(405):165–175Google Scholar
  40. 40.
    Jensen F (1996) An introduction to bayesian networks. Springer, New YorkGoogle Scholar
  41. 41.
    Murthy SK (1998) Automatic construction of decision trees from data a multi-disciplinary survey. Data Min Knowl Discov 2:345–389CrossRefGoogle Scholar
  42. 42.
    Burges C (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2(2):1–47CrossRefGoogle Scholar
  43. 43.
    Rosenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 65:386–408CrossRefPubMedGoogle Scholar
  44. 44.
    Fernandez Caballero JC, Martínez FJ, Hervás C, Gutiérrez PA (2010) Sensitivity versus accuracy in multiclass problems using memetic pareto evolutionary neural networks. IEEE Trans Neural Netw 21(5):750–770CrossRefPubMedGoogle Scholar
  45. 45.
    Hwang CL, Yoon K (1981) Multiple attribute decision making: methods and applications, vol 13. Springer, New YorkCrossRefGoogle Scholar
  46. 46.
    Opricovic S, Tzeng GH (2004) Compromise solution by mcdm methods: a comparative analysis of vikor and topsis. Eur J Oper Res 156(2):445–455CrossRefGoogle Scholar
  47. 47.
    Kuo Y, Yang T, Huang GW (2008) The use of grey relational analysis in solving multiple attribute decision-making problems. Comput Ind Eng 55(1):80–93CrossRefGoogle Scholar
  48. 48.
    Opricovic S (1998) Multicriteria optimization of civil engineering systems. Fac Civil Eng Belgrade 2(1):5–21Google Scholar
  49. 49.
    Peng Yi, Kou Gang, Wang Guoxun, Shi Yong (2011) FAMCDM: a fusion approach of MCDM methods to rank multiclass classification algorithms. Omega 39(6):677–689CrossRefGoogle Scholar
  50. 50.
    Gautheir TD (2001) Detecting trends using spearman’s rank correlation coefficient. Environ Forensics 2(4):359–362CrossRefGoogle Scholar

Copyright information

© International Federation for Medical and Biological Engineering 2016

Authors and Affiliations

  • Diego Peteiro-Barral
    • 1
  • Beatriz Remeseiro
    • 1
  • Rebeca Méndez
    • 1
  • Manuel G. Penedo
    • 1
  1. 1.Departamento de ComputaciónUniversidade da CoruñaA CoruñaSpain

Personalised recommendations