Abstract
This paper presents a principled two-step method for example-dependent cost binary classification problems. The first step obtains a consistent estimate of the posterior probabilities by training a Multi-Layer Perceptron with a Bregman surrogate cost. The second step uses the provided estimates in a Bayesian decision rule. When working with imbalanced datasets, neutral re-balancing allows getting better estimates of the posterior probabilities. Experiments with real datasets show the good performance of the proposed method in comparison with other procedures.
This work has been partially supported by Research Grant MacroADOBE (TEC2015-67719-P, MINECO/FEDER, EU) and by Research Project 2-BARBAS (Fundación BBVA).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
\(\widetilde{Q}\) can be interpreted as \(\widetilde{Q}_P\).
- 2.
Dataset available at http://www.creditriskanalytics.net.
- 3.
- 4.
References
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley, New York (2001)
Van Trees, H.L.: Detection, Estimation, and Modulation Theory: Part I. Wiley, New York (1968)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)
Zhang, G.P.: Neural networks for classification: a survey. IEEE Trans. Syst. Man Cybern. 30(4), 451–462 (2000)
Widrow, B., Rumelhart, D.E., Lehr, M.A.: Neural networks: applications in industry, business and science. Commun. ACM 37(3), 93–105 (1994)
He, H., Garcia, E.A.: Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21(9), 1263–1284 (2009)
Panigrahi, S., Kundu, A., Surai, S., Majumdar, A.K.: Credit card fraud detection: a fusion approach using Dempster-Shafer theory and Bayesian learning. Inf. Fusion 10(4), 354–363 (2009)
Bhattacharyya, S., Jha, S., Tharakunnel, K., Westland, J.C.: Data mining for credit card fraud: a comparative study. Decis. Support Syst. 50(3), 602–613 (2011)
Verbraken, T., Bravo, C., Webber, R., Baesens, B.: Development and application of consumer credit scoring models using profit-based classification measures. Eur. J. Oper. Res. 238(2), 505–513 (2014)
Bahnsen, A.C., Aouada, D., Ottersten, B.: Example-dependent cost-sensitive logistic regression for credit scoring. In: Proceedings of 13th International Conference on Machine Learning and Applications, pp. 263–269. IEEE Computer Society (2014)
Ngai, E.W.T., Xiu, L., Chau, D.C.K.: Application of data mining techniques in customer relationship management: a literature review and classification. Expert Syst. Appl. 36(2), 2592–2602 (2009)
Moro, S., Laureano, R.M.S., Cortez, P.: Using data mining for bank direct marketing: an application of the CRISP-DM methodology. In: Proceedings of European Simulation and Modeling Conference, Guimaraes (Portugal), pp. 117–121 (2011)
Elkan, C.: The foundations of cost-sensitive learning. In: Proceedings of 17th International Joint Conference on Artificial Intelligence, vol. 2, pp. 973–978 (2001)
Zadrozny, B., Langford, J., Abe, N.: Cost-sensitive learning by cost-proportionate example weighting. In: Proceedings of Third International Conference on Data Mining, pp. 435–442 (2003)
Branco, P., Torgo, L., Ribeiro, R.P.: A survey of predictive modeling on imbalanced domains. ACM Comput. Surv. 49(2), 31:1–31:50 (2016)
Brefeld, U., Geibel, P., Wysotzki, F.: Support vector machines with example dependent costs. In: Lavrač, N., Gamberger, D., Blockeel, H., Todorovski, L. (eds.) ECML 2003. LNCS (LNAI), vol. 2837, pp. 23–34. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-39857-8_5
González, P., et al.: Multiclass support vector machines with example dependent costs applied to plankton biomass estimation. IEEE Trans. Neural Netw. Learn. Syst. 24(11), 1901–1905 (2013)
Bahnsen, A.C., Stojanovic, A., Aouada, D., Ottersten, B.: Cost sensitive credit card fraud detection using Bayes minimization risk. In: Proceedings of 12th International Conference on Machine Learning and Applications, pp. 333–338. IEEE Computer Society (2013)
Bahnsen, A.C., Stojanovic, A., Aouada, D., Ottersten, B.: Improving credit card fraud detection with calibrated probabilities. In: Proceedings of 14th International Conference on Data Mining, Philadelphia, USA, pp. 677–685. SIAM (2014)
Bahnsen, A.C., Aouada, D., Ottersten, B.: A novel cost-sensitive framework for customer churn predictive modeling. Decis. Anal. 2(5), 1–15 (2015)
Bahnsen, A.C., Aouada, D., Ottersten, B.: Example-dependent cost-sensitive decision trees. Expert Syst. Appl. 42(19), 6609–6619 (2015)
Bregman, L.M.: The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comput. Math. Math. Phys. 7, 200–217 (1967)
Cid-Sueiro, J., Arribas, J.I., Urbán-Muñoz, S., Figueiras-Vidal, A.R.: Cost functions to estimate a posteriori probabilities in multiclass problems. IEEE Trans. Neural Netw. 10(3), 645–656 (1999)
Cid-Sueiro, J., Figueiras-Vidal, A.R.: On the structure of strict sense Bayesian cost functions and its applications. IEEE Trans. Neural Netw. 12(3), 445–455 (2001)
Bahnsen, A.C.: Cost Sensitive Classification (COSTCLA) Python module for cost-sensitive machine learning (classification), Version 0.5 (1996). https://pypi.org/project/costcla/
Baesens, B., Roesch, D., Scheule, H.: Credit Risk Analytics: Measurement Techniques, Applications, and Examples in SAS. Wiley, New York (2016)
Pedregosa, F., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Lázaro, M., Hayes, M.H., Figueiras-Vidal, A.R.: Training neural network classifiers through Bayes risk minimization applying unidimensional Parzen windows. Pattern Recognit. 77, 204–215 (2018)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Mediavilla-Relaño, J., Gutiérrez-López, A., Lázaro, M., Figueiras-Vidal, A.R. (2019). A Principled Two-Step Method for Example-Dependent Cost Binary Classification. In: Ferrández Vicente, J., Álvarez-Sánchez, J., de la Paz López, F., Toledo Moreo, J., Adeli, H. (eds) From Bioinspired Systems and Biomedical Applications to Machine Learning. IWINAC 2019. Lecture Notes in Computer Science(), vol 11487. Springer, Cham. https://doi.org/10.1007/978-3-030-19651-6_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-19651-6_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-19650-9
Online ISBN: 978-3-030-19651-6
eBook Packages: Computer ScienceComputer Science (R0)