Advertisement

Learning Monotone Nonlinear Models Using the Choquet Integral

  • Ali Fallah Tehrani
  • Weiwei Cheng
  • Krzysztof Dembczy
  • Eyke Hüllermeier
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6913)

Abstract

The learning of predictive models that guarantee monotonicity in the input variables has received increasing attention in machine learning in recent years. While the incorporation of monotonicity constraints is rather simple for certain types of models, it may become a more intricate problem for others. By trend, the difficulty of ensuring monotonicity increases with the flexibility or, say, nonlinearity of a model. In this paper, we advocate the so-called Choquet integral as a tool for learning monotone nonlinear models. While being widely used as a flexible aggregation operator in different fields, such as multiple criteria decision making, the Choquet integral is much less known in machine learning so far. Apart from combining monotonicity and flexibility in a mathematically sound and elegant manner, the Choquet integral has additional features making it attractive from a machine learning point of view. Notably, it offers measures for quantifying the importance of individual predictor variables and the interaction between groups of variables. As a concrete application of the Choquet integral, we propose a generalization of logistic regression. The basic idea of our approach, referred to as choquistic regression, is to replace the linear function of predictor variables, which is commonly used in logistic regression to model the log odds of the positive class, by the Choquet integral.

Keywords

Logistic Regression Aggregation Operator Fuzzy Measure Positive Class Interaction Index 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Ben-David, A., Sterling, L., Pao, Y.-H.: Learning and classification of monotonic ordinal concepts. Computational Intelligence 5(1), 45–49 (1989)CrossRefGoogle Scholar
  2. 2.
    Duivesteijn, W., Feelders, A.: Nearest neighbour classification with monotonicity constraints. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part I. LNCS (LNAI), vol. 5211, pp. 301–316. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  3. 3.
    Sill, J.: Monotonic networks. In: Advances in Neural Information Processing Systems, pp. 661–667. The MIT Press, Denver (1998)Google Scholar
  4. 4.
    Ben-David, A.: Monotonicity maintenance in information-theoretic machine learning algorithms. Machine Learning 19, 29–43 (1995)Google Scholar
  5. 5.
    Potharst, R., Feelders, A.: Classification trees for problems with monotonicity constraints. ACM SIGKDD Explorations Newsletter 4(1), 1–10 (2002)CrossRefGoogle Scholar
  6. 6.
    Dembczyński, K., Kotlowski, W., Slowinski, R.: Learning rule ensembles for ordinal classification with monotonicity constraints. Fundamenta Informaticae 94(2), 163–178 (2009)zbMATHMathSciNetGoogle Scholar
  7. 7.
    Chandrasekaran, R., Ryu, Y., Jacob, V., Hong, S.: Isotonic separation. INFORMS Journal on Computing 17, 462–474 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Dembczyński, K., Kotłowski, W., Słowiński, R.: Additive preference model with piecewise linear components resulting from dominance-based rough set approximations. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Żurada, J.M. (eds.) ICAISC 2006. LNCS (LNAI), vol. 4029, pp. 499–508. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Feelders, A.: Monotone relabeling in ordinal classification. In: Proceedings of the 10th IEEE International Conference on Data Mining, pp. 803–808. IEEE Computer Society, Los Alamitos (2010)Google Scholar
  10. 10.
    Kotłowski, W., Dembczyński, K., Greco, S., Słowiński, R.: Stochastic dominance-based rough set model for ordinal classification. Information Sciences 178(21), 3989–4204 (2008)CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Grabisch, M., Murofushi, T., Sugeno, M. (eds.): Fuzzy Measures and Integrals: Theory and Applications. Physica, Heidelberg (2000)zbMATHGoogle Scholar
  12. 12.
    Grabisch, M.: Fuzzy integral in multicriteria decision making. Fuzzy Sets and Systems 69(3), 279–298 (1995)CrossRefzbMATHMathSciNetGoogle Scholar
  13. 13.
    Torra, V.: Learning aggregation operators for preference modeling. In: Preference Learning, pp. 317–333. Springer, Heidelberg (2011)Google Scholar
  14. 14.
    Torra, V., Narukawa, Y.: Modeling Decisions: Information Fusion and Aggregation Operators. Springer, Heidelberg (2007)zbMATHGoogle Scholar
  15. 15.
    Grabisch, M.: Modelling data by the Choquet integral. In: Information Fusion in Data Mining, pp. 135–148. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  16. 16.
    Mori, T., Murofushi, T.: An analysis of evaluation model using fuzzy measure and the Choquet integral. In: Proceedings of the 5th Fuzzy System Symposium, pp. 207–212. Japan Society for Fuzzy Sets and Systems (1989)Google Scholar
  17. 17.
    Grabisch, M.: A new algorithm for identifying fuzzy measures and its application to pattern recognition. In: Proceedings of IEEE International Conference on Fuzzy Systems, vol. 1, pp. 145–150. IEEE, Los Alamitos (1995)Google Scholar
  18. 18.
    Angilella, S., Greco, S., Matarazzo, B.: Non-additive robust ordinal regression with Choquet integral, bipolar and level dependent Choquet integrals. In: Proceedings of the Joint 2009 International Fuzzy Systems Association World Congress and 2009 European Society of Fuzzy Logic and Technology Conference, IFSA/EUSFLAT, pp. 1194–1199 (2009)Google Scholar
  19. 19.
    Beliakov, G., James, S.: Citation-based journal ranks: the use of fuzzy measures. Fuzzy Sets and Systems 167(1), 101–119 (2011)CrossRefzbMATHMathSciNetGoogle Scholar
  20. 20.
    Grabisch, M., Nicolas, J.-M.: Classification by fuzzy integral: performance and tests. Fuzzy Sets and Systems 65(2-3), 255–271 (1994)CrossRefMathSciNetGoogle Scholar
  21. 21.
    Sugeno, M.: Theory of Fuzzy Integrals and its Application. PhD thesis, Tokyo Institute of Technology (1974)Google Scholar
  22. 22.
    Murofushi, T., Soneda, S.: Techniques for reading fuzzy measures (III): interaction index. In: Proceedings of the 9th Fuzzy Systems Symposium, pp. 693–696 (1993)Google Scholar
  23. 23.
    Grabisch, M.: k-order additive discrete fuzzy measures and their representation. Fuzzy Sets and Systems 92(2), 167–189 (1997)CrossRefzbMATHMathSciNetGoogle Scholar
  24. 24.
    Vitali, G.: Sulla definizione di integrale delle funzioni di una variabile. Annali di Matematica Pura ed Applicata 2(1), 111–121 (1925)CrossRefzbMATHMathSciNetGoogle Scholar
  25. 25.
    Choquet, G.: Theory of capacities. Annales de l’institut Fourier 5, 131–295 (1954)CrossRefzbMATHMathSciNetGoogle Scholar
  26. 26.
    Hosmer, D., Lemeshow, S.: Applied Logistic Regression, 2nd edn. Wiley, Chichester (2000)CrossRefzbMATHGoogle Scholar
  27. 27.
    Jaccard, J.: Interaction Effects in Logistic Regression. Saga Publications, Thousand Oaks (2001)CrossRefzbMATHGoogle Scholar
  28. 28.
    Modave, F., Grabisch, M.: Preference representation by a Choquet integral: commensurability hypothesis. In: Proceedings of the 7th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, pp. 164–171. Editions EDK (1998)Google Scholar
  29. 29.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.: The WEKA data mining software: an update. ACM SIGKDD Explorations Newsletter 11(1), 10–18 (2009)CrossRefGoogle Scholar
  30. 30.
    Daniels, H., Kamp, B.: Applications of mlp networks to bond rating and house pricing. Neural Computation and Applications 8, 226–234 (1999)CrossRefGoogle Scholar
  31. 31.
    Ben-David, A., Sterling, L., Tran, T.: Adding monotonicity to learning algorithms impair their accuracy. Expert Systems with Applications 36(3), 6627–6634 (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Ali Fallah Tehrani
    • 1
  • Weiwei Cheng
    • 1
  • Krzysztof Dembczy
    • 1
    • 2
  • Eyke Hüllermeier
    • 1
  1. 1.Mathematics and Computer ScienceUniversity of MarburgGermany
  2. 2.Institute of Computing SciencePoznań University of TechnologyPoland

Personalised recommendations