Practical Notes on Applying Generalised Stochastic Orderings to the Study of Performance of Classification Algorithms for Low Quality Data

  • Patryk Żywica
  • Katarzyna Basiukajc
  • Inés Couso
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 643)


This paper presents an approach to applying stochastic orderings to evaluate classification algorithms for low quality data. It discusses some known stochastic orderings along with practical notes about their application to classifier evaluation. Finally, a new approach based on fuzzy cost function is presented. The new method allows comparing any two classifiers, but does not require a precise definition of the cost function. All proposed methods were evaluated on real life medical data. The obtained results are very similar to those previously reported but comparatively much weaker assumptions about costs values are adopted.


Classification Loss function Stochastic ordering Low quality data Fuzzy random variable 


  1. 1.
    Żywica, P., Wójtowicz, A., et al.: Improving medical decisions under incomplete data using interval–valued fuzzy aggregation. In: Proceedings of 9th European Society for Fuzzy Logic and Technology (EUSFLAT), Gijón, Spain, pp. 577–584 (2015)Google Scholar
  2. 2.
    Wójtowicz, A., Żywica, P., et al.: Solving the problem of incomplete data in medical diagnosis via interval modeling. Appl. Soft Comput. 47, 424–437 (2016)CrossRefGoogle Scholar
  3. 3.
    Diering, M., Dyczkowski, K., Hamrol, A.: New method for assessment of raters agreement based on fuzzy similarity. In: Proceedings of the 10th International Conference on Soft Computing Models in Industrial and Environmental Applications, pp. 415–425. Springer (2015)Google Scholar
  4. 4.
    Stachowiak, A., Dyczkowski, K.: A similarity measure with uncertainty for incompletely known fuzzy sets. In: 2013 Joint IFSA World Congress and NAFIPS Annual Meeting (IFSA/NAFIPS), pp. 390–394. IEEE (2013)Google Scholar
  5. 5.
    Stukan, M., Dudziak, M., et al.: Usefulness of diagnostic indices comprising clinical, sonographic, and biomarker data for discriminating benign from malignant ovarian masses. J. Ultrasound Med. 34(2), 207–217 (2015)CrossRefGoogle Scholar
  6. 6.
    Moszyński, R., Żywica, P., et al.: Menopausal status strongly influences the utility of predictive models in differential diagnosis of ovarian tumors: an external validation of selected diagnostic tools. Ginekol. Pol. 85(12), 892–899 (2014)CrossRefGoogle Scholar
  7. 7.
    Japkowicz, N., Shah, M.: Evaluating Learning Algorithms: A Classification Perspective. Cambridge University Press, New York (2011)CrossRefMATHGoogle Scholar
  8. 8.
    Hatch, S.: Snowball in a Blizzard: A Physician’s Notes on Uncertainty in Medicine. Basic Books, New York (2016)Google Scholar
  9. 9.
    Couso, I., Sánchez, L.: Generalized stochastic orderings applied to the study of performance of machine learning algorithms for low quality data. In: Proceedings of 16th International Fuzzy Systems Association World Congress and 9th Conference of the European Society for Fuzzy Logic and Technology (IFSA-EUSFLAT 2015), pp. 1534–1541. Atlantis Press (2015)Google Scholar
  10. 10.
    Couso, I., Sánchez, L.: Machine learning models, epistemic set-valued data and generalized loss functions: an encompassing approach. Inf. Sci. 358, 129–150 (2016)CrossRefGoogle Scholar
  11. 11.
    Shaked, M., Shanthikumar, G.: Stochastic Orders. Springer Science & Business Media, New York (2007)CrossRefMATHGoogle Scholar
  12. 12.
    Savage, L.J.: The Foundations of Statistics. Courier Corporation, New York (1972)MATHGoogle Scholar
  13. 13.
    Hadar, J., Russell, W.R.: Rules for ordering uncertain prospects. Am. Econ. Rev. 59(1), 25–34 (1969)Google Scholar
  14. 14.
    David, H.A.: The Method of Paired Comparisons, vol. 12. Charles Griffin & D. Ltd., London (1963)Google Scholar
  15. 15.
    Page, L., Brin, S., et al.: The PageRank citation ranking: bringing order to the web. Technical report, Stanford InfoLab (1999)Google Scholar
  16. 16.
    Couso, I., Dubois, D.: A perspective on the extension of stochastic orderings to fuzzy random variables. In: Proceedings of 16th International Fuzzy Systems Association World Congress and 9th Conference of the European Society for Fuzzy Logic and Technology, IFSA-EUSFLAT 2015, pp. 1486–1492. Atlantis Press (2015)Google Scholar
  17. 17.
    Dubois, D., Prade, H.: Gradualness, uncertainty and bipolarity: making sense of fuzzy sets. Fuzzy Sets Syst. 192, 3–24 (2012)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Wright, S., Nocedal, J.: Numerical Optimization, vol. 35. Springer Science, New York (1999). pp. 67–68Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Patryk Żywica
    • 1
  • Katarzyna Basiukajc
    • 1
  • Inés Couso
    • 2
  1. 1.Faculty of Mathematics and Computer ScienceAdam Mickiewicz University in PoznańPoznańPoland
  2. 2.Department of Statistics and Operational ResearchUniversity of OviedoGijónSpain

Personalised recommendations