Preliminary numerical investigations of conformal predictors based on fuzzy logic classifiers

  • A. Murari
  • J. Vega
  • D. Mazon
  • T. Courregelongue
  • JET-EFDA Contributors


A new family of techniques, called conformal predictors, have very recently been developed to hedge the estimates of machine learning methods, by providing two parameters, credibility and confidence, which can assess the level of trust that can be attributed to their outputs. In this paper, the main steps required to extend this approach to fuzzy logic classifiers are reported. The more delicate aspect is the definition of an appropriate nonconformity score, which has to be based on the fuzzy membership function to preserve the specificities of Fuzzy Logic. Various examples of increasing complexity are introduced, to describe the main properties of fuzzy logic based conformal predictors and to compare their performance with alternative approaches. The obtained results are quite promising, since conformal predictors based on fuzzy classifiers outperform solutions based on the nearest neighbour in terms of ambiguity, robustness and interpretability.


Conformal predictors Fuzzy logic Membership function Non conformity score 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Vapnik, V.N.: The Nature of Statistical Learning Theory, 2nd edn. Springer, New York (2000)CrossRefGoogle Scholar
  2. 2.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition. Elsevier, Academic Press, Canada (2009)Google Scholar
  3. 3.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer Science+Bussiness Media Singapore, Singapore (2006)zbMATHGoogle Scholar
  4. 4.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Identification, 2nd edn.Wiley-Interscience, New York (2000)Google Scholar
  5. 5.
    Vovk, V., Gammerman, A., Shafer, G.: Algorithmic Learning in a Random World. Springer, New York (2005)zbMATHGoogle Scholar
  6. 6.
    Vega, J., Murari, A., Pereira, A., Gonz´alez, S., Pastor, I.: Rev. Sci. Instrum. 81, 10E118 (2010)Google Scholar
  7. 7.
    Ross, T.J.: Fuzzy Logic with Engineering Applications, 2nd edn. Wiley, Singapore (2004)Google Scholar
  8. 8.
    Riid, A., Ruistern, E.: Transparent fuzzy systems in modelling and control. In: Casillas, J., Cordon, O., Herrera, F., Magdalena, L. (eds.): Interpretability Issues in Fuzzy Modeling, pp. 452–476. Springer (2003)Google Scholar
  9. 9.
    Nauck, D., Kruse, R.: How the learning of rule weights affects the interpretability of fuzzy systems. In: Proceedings IEEE International Conference on Fuzzy Systems 1998. pp. 1235–1240. Anchorage (1998)Google Scholar
  10. 10.
    Papadopoulos, H., Vovk, V., Gammerman, A.: Regression conformal prediction with nearest neighbours. J. Artif. Intell. Res. 40, 815–840 (2011). doi: 10.1613/jair.3198 zbMATHMathSciNetGoogle Scholar
  11. 11.
    Nouretdinov, I., Melluish, T., Vovk, V.: Ridge regression confidence machine. In: Proceedings of the 18th International Conference on Machine Learning (ICML’01), pp. 385–392. Morgan Kaufmann, San Francisco (2001)Google Scholar
  12. 12.
    Proedrou, K., Nouretdinov, I., Vovk, V., Gammerman, A.: Transductive confidence machines for pattern recognition. In: Proceedings of the 13th European Conference on Machine Learning (ECML’02), ser. LNCS, vol. 2430, pp. 381–390. Springer (2002)Google Scholar
  13. 13.
    Papadopoulos, H., Haralambous, H.: Reliable prediction intervals with regression neural networks. Neural Netw. (2011). doi: 10.1016/j.neunet.2011.05.008 Google Scholar
  14. 14.
    Bhattacharyya, S.: Confidence in predictions from random tree ensembles. In: Proceedings of the 11th IEEE International Conference on Data Mining (ICDM2011), pp. 71–80. Springer (2011)Google Scholar
  15. 15.
    Gammerman, A., Vovk, V., Burford, B., Nouretdinov, I., Luo, Z., Chervonenkis, A., Waterfield, M., Cramer, R., Tempst, P., Villanueva, J., Kabir, M., Camuzeaux, S., Timms, J., Menon, U., Jacobs, I.: Serum proteomic abnormality predating screen detection of ovarian cancer. Comput. J. 52(3), 326–333 (2009)CrossRefGoogle Scholar
  16. 16.
    Papadopoulos, H., Papatheocharous, E., Andreou, A.S.: Reliable confidence intervals for software effort estimation. In: Proceedings of the 2nd Workshop on Artificial Intelligence Techniques in Software Engineering (AISEW 2009), ser. CEUR Workshop Proceedings, vol. 475. CEUR-WS. org, [Online]. Available:
  17. 17.
    Bellotti, T., Luo, Z., Gammerman, A., Delft, F.W.V., Saha, V.: Qualified predictions for microarray and proteomics pattern diagnostics with confidence machines. Int. J. Neural Syst. 15(4), 247–258 (2005)CrossRefGoogle Scholar
  18. 18.
    Zhang, J., Li, G., Hu, M., Li, J., Luo, Z.: Recognition of hypoxia EEG with a preset confidence level based on EEG analysis. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN 2008), part of the IEEE World Congress on Computational Intelligence (WCCI 2008), pp.3005–3008. IEEE (2008)Google Scholar
  19. 19.
    Lambrou, A., Papadopoulos, H., Kyriacou, E., Pattichis, C.S., Pattichis, M.S., Gammerman, A., Nicolaides, A.: Evaluation of the risk of stroke with confidence predictions based on ultrasound carotid image analysis. Int. J. Artif. Intell. Tools 21(04), 1240016 (2012). [Online]. Available doi: 10.1142/S0218213012400167 CrossRefGoogle Scholar
  20. 20.
    Shahmuradov, A., Solovyev, V.V., Gammerman, A.J.: Plant promoter prediction with confidence estimation. Nucleic Acids Res. 33(3), 1069–1076 (2005)CrossRefGoogle Scholar
  21. 21.
    Papadopoulos, H., Vovk, V., Gammerman, A.: Conformal prediction with neural networks. In: Proceedings of the 19th IEEE International Conference on Tools with Artificial Intelligence (ICTAI’07), vol. 2, pp. 388–395. IEEE Computer Society (2007)Google Scholar
  22. 22.
    Papadopoulos, H., Gammerman, A., Vovk, V.: Reliable diagnosis of acute abdominal pain with conformal prediction. Eng. Intell. Syst. 17(2–3), 127–137. CRL Publishing (2009)Google Scholar
  23. 23.
    Lambrou, A., Papadopoulos, H., Gammerman, A.: Reliable confidence measures for medical diagnosis with evolutionary algorithms. IEEE Trans. Inf. Technol. Biomed. 15(1), 93–99 (2011). IEEEGoogle Scholar
  24. 24.
    Lambrou, A., Papadopoulos, H., Gammerman, A. (2009)Google Scholar
  25. 25.
    Abonyi, J., Feil, B.: Cluster Analysis for Data Mining and System Identification. Birkauser Verlag AG, Basel (2007)zbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2014

Authors and Affiliations

  • A. Murari
    • 1
  • J. Vega
    • 2
  • D. Mazon
    • 3
  • T. Courregelongue
    • 4
  • JET-EFDA Contributors
    • 5
  1. 1.Consorzio RFX-Associazione EURATOM ENEA per la FusionePadovaItaly
  2. 2.Asociación EURATOM-CIEMAT para Fusión, CIEMATMadridSpain
  3. 3.Association EURATOM-CEASaint-Paul-lez-DuranceFrance
  4. 4.Arts et Métiers ParisTech Engineering College (ENSAM)ParisFrance
  5. 5.JET-EFDA, Culham Science CentreAbingdonUK

Personalised recommendations