Skip to main content
SpringerLink
Log in

A deep learning model for behavioural credit scoring in banks

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The main aim of this paper is to help bank management in scoring credit card clients using machine learning by modelling and predicting the consumer behaviour concerning three aspects: the probability of single and consecutive missed payments for credit card customers, the purchasing behaviour of customers, and grouping customers based on a mathematical expectation of loss. Two models are developed: the first provides the probability of a missed payment during the next month for each customer, which is described as Missed payment prediction Long Short Term Memory model (MP-LSTM), whilst the second estimates the total monthly amount of purchases, which is defined as Purchase Estimation Prediction Long Short Term Memory model (PE-LSTM). Based on both models, a customer behavioural grouping is provided, which can be helpful for the bank’s decision-making. Both models are trained on real credit card transactional datasets. Customer behavioural scores are analysed using classical performance evaluation measures. Calibration analysis of MP-LSTM scores showed that they could be considered as probabilities of missed payments. Obtained purchase estimations were analysed using mean square error and absolute error. The MP-LSTM model was compared to four traditional well-known machine learning algorithms. Experimental results show that, compared with conventional methods based on feature extraction, the consumer credit scoring method based on the MP-LSTM neural network has significantly improved consumer credit scoring.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Notes

  1. The dataset is available at https://archive.ics.uci.edu/ml/datasets/default+of+credit+card+clients.

References

  1. Acuna E, Rodriguez C (2004) The treatment of missing values and its effect on classifier accuracy. In Journal of Classification, pp 639–647. https://doi.org/10.1007/978-3-642-17103-1_60

  2. Addo P, Guegan D, Hassani B (2018) Credit risk analysis using machine and deep learning models. Risks 6(2):38. https://doi.org/10.3390/risks6020038

    Article  Google Scholar 

  3. Adeodato P, Melo S (2016) On the equivalence between Kolmogorov-Smirnov and ROC curve metrics for binary classification

  4. Agarwal S, Chomsisengphet S, Liu C, Song C, Souleles NS (2018) Benefits of relationship banking: Evidence from consumer credit markets. J Monetary Econ 96:16–32

    Article  Google Scholar 

  5. Akkoç S (2012) An empirical comparison of conventional techniques, neural networks and the three-stage hybrid Adaptive Neuro Fuzzy Inference System (ANFIS) model for credit scoring analysis: the case of Turkish credit card data. Eur J Oper Res 222(1):168–178. https://doi.org/10.1016/j.ejor.2012.04.009

    Article  Google Scholar 

  6. Ala’raj, M., & Abbod, M. F. (2016). Classifiers consensus system approach for credit scoring. Knowl Based Syst 104: 89–105. 10.1016/j.knosys.2016.04.013

  7. Ala’raj M, Majdalawieh M, Abbod MF (2020) Improving binary classification using filtering based on k-NN proximity graphs. J Big Data 7(1): 1–18. https://doi.org/10.1186/s40537-020-00297-7

  8. Alborzi M, Khanbabaei M (2016) Using data mining and neural networks techniques to propose a new hybrid customer behaviour analysis and credit scoring model in banking services based on a developed RFM analysis method. Int J Bus Inf Syst 23(1):1–22. https://doi.org/10.1504/ijbis.2016.078020

    Article  Google Scholar 

  9. Anderson R (2007) The credit scoring toolkit: theory and practice for retail credit risk management and decision automation. OUP Oxford, p 731

  10. Angelini E, Tollo G, Roli A (2008) A neural network approach for credit risk evaluation. Q Rev Econ Finance 48:733–755. https://doi.org/10.1016/j.qref.2007.04.001

    Article  Google Scholar 

  11. Atiya AF, Parlos AG (2000) New results on recurrent network training: unifying the algorithms and accelerating convergence. IEEE Trans Neural Netw 11(3):697–709. https://doi.org/10.1109/72.846741

    Article  Google Scholar 

  12. Baesens B, Gestel T, Van Stepanova M, Van Den Poel D, Vanthienen J (2005) Neural network survival analysis for personal loan data. J Oper Res Soc 56(9): 1089–1098. https://doi.org/10.1057/palgrave.jors.2601990

  13. Baesens B, Gestel TV, Viaene S, Stepanova M, Suykens J, Vanthienen J (2003) Benchmarking state-of-the-art classification algorithms for credit scoring. J Oper Res Soc 54(6):627–635. https://doi.org/10.1057/palgrave.jors.2601545

    Article  MATH  Google Scholar 

  14. Bastani K, Asgari E, Namavari H (2019) Wide and deep learning for peer-to-peer lending. Expert Syst Appl 134:209–224. https://doi.org/10.1016/j.eswa.2019.05.042

    Article  Google Scholar 

  15. Bauer A, Züfle M, Herbst N, Kounev S (2019) Best practices for time series forecasting (Tutorial Paper). https://doi.org/10.1109/FAS-W.2019.00069

  16. Bellotti T, Crook J (2007) Modelling and predicting loss given default for credit cards. In: Credit scoring and credit control XI conference

  17. Bellotti T, Crook J (2009) Credit scoring with macroeconomic variables using survival analysis. J Oper Res Soc 60(12):1699–1707. https://doi.org/10.1057/jors.2008.130

    Article  MATH  Google Scholar 

  18. Bellotti T, Crook J (2013) Forecasting and stress testing credit card default using dynamic models. Int J Forecast 29(4):563–574. https://doi.org/10.1016/j.ijforecast.2013.04.003

    Article  Google Scholar 

  19. Bensic M, Sarlija N, Zekic-Susac M (2005) Modelling small-business credit scoring by using logistic regression, neural networks and decision trees. Intell Syst Account Finance Manag 13(3):133–150. https://doi.org/10.1002/isaf.261

    Article  Google Scholar 

  20. Bertola G, Disney R, Grant C (2006) The economics of consumer credit

  21. Bhatia S, Sharma P, Burman R, Hazari S, Hande R (2017) Credit scoring using machine learning techniques. Int J Comput Appl 161:1–4

    Google Scholar 

  22. Bhattacharyya S, Maulik U (2013) Soft computing for image and multimedia data processing. https://doi.org/10.1007/978-3-642-40255-5

  23. Breiman L (2001) Random forests. Mach Learn 45(1):5–32. https://doi.org/10.1023/A:1010933404324

    Article  MATH  Google Scholar 

  24. Brier GW (1950) Verification of forecasts expressed in terms of probability. Mon Weather Rev 78:1–3

    Article  Google Scholar 

  25. Bröcker J, Smith L (2007) Increasing the reliability of reliability diagrams. Weather Forecast 22(3):651–661. https://doi.org/10.1175/WAF993.1

    Article  Google Scholar 

  26. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297. https://doi.org/10.1007/BF00994018

    Article  MATH  Google Scholar 

  27. Cui Z, Ke R, Wang Y (2018) Deep bidirectional and unidirectional LSTM recurrent neural network for network-wide traffic speed prediction. arXiv preprint arXiv:1801.02143

  28. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  29. Dietterich TG (1998) Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput 10(7):1895–1923. https://doi.org/10.1162/089976698300017197

    Article  Google Scholar 

  30. Dyché J (2001) The CRM handbook: a business guide to customer relationship management. Addison-Wesley Longman Publishing Co., Inc. arXiv preprint arXiv:1801.02143

  31. Feng X, Xiao Z, Zhong B, Qiu J, Dong Y (2018) Dynamic ensemble classification for credit scoring using soft probability. Appl Soft Comput 65:139–151

    Article  Google Scholar 

  32. Glennon D, Kiefer N, Larson C, Choi H (2008) Development and validation of credit scoring models. J Credit Risk. https://doi.org/10.21314/JCR.2008.075

    Article  Google Scholar 

  33. Graves A, Mohamed A, Hinton G (2013) Speech recognition with deep recurrent neural networks. In: ICASSP, IEEE international conference on acoustics, speech and signal processing—Proceedings, vol 38, pp 6645–6649. https://doi.org/10.1109/ICASSP.2013.6638947

  34. Gui L (2019) Application of machine learning algorithms in predicting credit card default payment. University of California

  35. Haltuf M (2014) Support vector machines for credit scoring. University of Economics in Prague Faculty of Finance, Departe-ment of Banking and Inscurance

    Google Scholar 

  36. Hancock JT, Khoshgoftaar TM (2020) Survey on categorical data for neural networks. J Big Data 7(1):28. https://doi.org/10.1186/s40537-020-00305-w

    Article  Google Scholar 

  37. Hand DJ (2009) Measuring classifier performance: a coherent alternative to the area under the ROC curve. Mach Learn 77(1):103–123. https://doi.org/10.1007/s10994-009-5119-5

    Article  MATH  Google Scholar 

  38. Hand DJ, Henley WE (1997) Statistical classification methods in consumer credit scoring: a review. J R Stat Soc A Stat Soc 160(3):523–541. https://doi.org/10.1111/j.1467-985x.1997.00078.x

    Article  Google Scholar 

  39. Haykin SS (2009) Neural networks and learning machines (Third). Pearson Education.

  40. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9:1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  41. Hsieh H-I, Lee T-P, Lee T-S (2010) Data mining in building behavioral scoring models. In: 2010 international conference on computational intelligence and software engineering, pp 1–4. https://doi.org/10.1109/cise.2010.5677005

  42. Hsieh N-C (2004) An integrated data mining and behavioral scoring model for analyzing bank customers. Expert Syst Appl 27(4):623–633. https://doi.org/10.1016/j.eswa.2004.06.007

    Article  Google Scholar 

  43. Huang C-L, Chen M-C, Wang C-J (2007) Credit scoring with a data mining approach based on support vector machines. Expert Syst Appl 33(4):847–856. https://doi.org/10.1016/j.eswa.2006.07.007

    Article  Google Scholar 

  44. Jadhav S, He H, Jenkins K (2018) Information gain directed genetic algorithm wrapper feature selection for credit rating. Appl Soft Comput 69:541–553

    Article  Google Scholar 

  45. Kavzoglu T (2017) Chapter 33—Object-oriented random forest for high resolution land cover mapping using quickbird-2 imagery (Samui P, Sekhar S, T-H VEB. of Balas NC (eds)). Academic Press, pp 607–619. https://doi.org/10.1016/B978-0-12-811318-9.00033-8

  46. Kennedy K, Namee BM, Delany SJ, O’Sullivan M, Watson N (2013) A window of opportunity: assessing behavioural scoring. Expert Syst Appl 40(4):1372–1380. https://doi.org/10.1016/j.eswa.2012.08.052

    Article  Google Scholar 

  47. Kim H, Cho H, Ryu D (2018) An empirical study on credit card loan delinquency. Econ Syst 42:437–449. https://doi.org/10.1016/j.ecosys.2017.11.003

    Article  Google Scholar 

  48. Kumar PR, Ravi V (2007) Bankruptcy prediction in banks and firms via statistical and intelligent techniques: a review. Eur J Oper Res 180(1):1–28. https://doi.org/10.1016/j.ejor.2006.08.043

    Article  MATH  Google Scholar 

  49. Lahsasna A, Ainon R, Wah T (2010) Credit scoring models using soft computing methods: a survey. Int Arab J Inf Technol 7:115–123

    Google Scholar 

  50. Lessmann S, Baesens B, Seow H-V, Thomas L (2015) Benchmarking state-of-the-art classification algorithms for credit scoring: an update of research. pp.124–136. Eur J Oper Res. https://doi.org/10.1016/j.ejor.2015.05.030

  51. Li Y, Li Y, Li Y (2019) What factors are influencing credit card customer’s default behavior in China? A study based on survival analysis. Physica A 526:120861. https://doi.org/10.1016/j.physa.2019.04.097

    Article  Google Scholar 

  52. Lim MK, Sohn SY (2007) Cluster-based dynamic scoring model. Expert Syst Appl 32(2):427–431. https://doi.org/10.1016/j.eswa.2005.12.006

    Article  Google Scholar 

  53. Liu Y, Schumann M (2005) Data mining feature selection for credit scoring models. J Oper Res Soc 56(9):1099–1108. https://doi.org/10.1057/palgrave.jors.2601976

    Article  MATH  Google Scholar 

  54. Liu Y (2001) New issues in credit scoring application. Abteilung Wirtschaftsinformatik II, Georg-August-Universität, Institut für Wirtschaftsinformatik

    Google Scholar 

  55. Louzada F, Ara A, Fernandes GB (2016) Classification methods applied to credit scoring: systematic review and overall comparison. Surv Oper Res Manag Sci 21(2):117–134. https://doi.org/10.1016/j.sorms.2016.10.001

    Article  MathSciNet  Google Scholar 

  56. Malhotra P, Vig L, Shroff G, Agarwal P (2015) Long short term memory networks for anomaly detection in time series 89–94

  57. Malhotra R, Malhotra DK (2003) Evaluating consumer loans using neural networks. Omega 31:83–96. https://doi.org/10.2139/ssrn.314396

    Article  Google Scholar 

  58. Malik M, Thomas LC (2010) Modelling credit risk of portfolio of consumer loans. J Oper Res Soc 61(3):411–420. https://doi.org/10.1057/jors.2009.123

    Article  Google Scholar 

  59. McNab H, Wynn A (2000) Principles and practice of consumer credit risk management. CIB Publishing

  60. Mylonakis J, Diacogiannis G (2010) Evaluating the likelihood of using linear discriminant analysis as a commercial bank card owners credit scoring model. Int Bus Res 3(2):9. https://doi.org/10.5539/ibr.v3n2p9

    Article  Google Scholar 

  61. Neto R, Adeodato P, Salgado AC (2016) A framework for data transformation in credit behavioral scoring applications based on model driven development. Expert Syst Appl 72:293–305. https://doi.org/10.1016/j.eswa.2016.10.059

    Article  Google Scholar 

  62. Pereira S (2019) Modelling credit card customer behaviour. Work Project presented as a partial requirement for Degree of Master of Statistics and Information Management, with a specialization in Information Analysis and Management

  63. Sarlija N, Bensic M, Zekic-Susac M (2009) Comparison procedure of predicting the time to default in behavioural scoring. Expert Syst Appl 36(5):8778–8788. https://doi.org/10.1016/j.eswa.2008.11.042

    Article  Google Scholar 

  64. Setiono R, Thong JYL, Yap C-S (1998) Symbolic rule extraction from neural networks. Inf Manag 34(2):91–101. https://doi.org/10.1016/s0378-7206(98)00048-2

    Article  Google Scholar 

  65. Sharda R, Wilson RL (1996) Neural network experiments in business failures predication: a review of predictive performance issues. Int J Comput Intell Organ. https://doi.org/10.1109/hicss.1993.284245

    Article  Google Scholar 

  66. Siami Namini S, Siami Namin A (2018) Forecasting economics and financial time series: ARIMA vs. LSTM

  67. So MMC, Thomas LC (2011) Modelling the profitability of credit cards by Markov decision processes. Eur J Oper Res 212(1):123–130. https://doi.org/10.1016/j.ejor.2011.01.023

    Article  Google Scholar 

  68. Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, pp 3104–3112

  69. Thomas LC, Ho J, Scherer W (2001) Time will tell: behavioural scoring and the dynamics of consumer credit assessment. IMA J Manag Math 12(1):89–103. https://doi.org/10.1093/imaman/12.1.89

    Article  MATH  Google Scholar 

  70. Thomas LC (2000) A survey of credit and behavioural scoring: forecasting financial risk of lending to consumers. Int J Forecast 16(2):149–172. https://doi.org/10.1016/s0169-2070(00)00034-0

    Article  Google Scholar 

  71. Wang C, Han D, Liu Q, Luo S (2018) A Deep learning approach for credit scoring of peer-to-peer lending using attention mechanism LSTM. IEEE Access 7:2161–2168. https://doi.org/10.1109/access.2018.2887138

    Article  Google Scholar 

  72. Wang L, Lu W, Malhotra NK (2011) Demographics, attitude, personality and credit card features correlate with credit card debt: a view from China. J Econ Psychol 32(1):179–193. https://doi.org/10.1016/j.joep.2010.11.006

    Article  Google Scholar 

  73. Witten IH, Frank EF, Hall MA (2011) Credibility: evaluating what’s been learned. In Witten IH, Frank E, Hall MA (eds) Data mining: practical Machine learning tools and techniques, 3rd edn. Morgan Kaufmann, pp 147–187 https://doi.org/10.1016/B978-0-12-374856-0.00005-5

  74. Xie Y, Liu G, Cao R, Li Z, Yan C, Jiang C (2019) A feature extraction method for credit card fraud detection. In: 2019 2nd international conference on intelligent autonomous systems ({ICoIAS}). https://doi.org/10.1109/icoias.2019.00019

  75. Yeh IC, Lien CH (2009) The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients. Expert Syst Appl 36(2):2473–2480

    Article  Google Scholar 

  76. Yu J, Yao J, Chen J (2019) Credit scoring with AHP and fuzzy comprehensive evaluation based on behavioural data from weibo platform. Tehn Vjes Tech Gaz 26(2):462–470. https://doi.org/10.17559/tv-20181217180231

    Article  Google Scholar 

  77. Yu L, Liu H (2003) Feature selection for high-dimensional data: a fast correlation-based filter solution. Proc Twentieth Int Conf Mach Learn 2:856–863

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Office of Research, Zayed University [grant number R20053].

Author information

Authors and Affiliations

  1. Department of Information Systems, College of Technological Innovation, Zayed University, Dubai, 19282, UAE

    Maher Ala’raj & Munir Majdalawieh

  2. Department of Electronic and Computer Engineering, College of Engineering, Design and Physical Sciences, Brunel University London, Kingston Lane, Uxbridge, UB8 3PH, UK

    Maysam F. Abbod

  3. Logistic Sciences Department, School of Management and Logistic Science, German Jordanian University, Madaba Street, 35247, Amman, 11180, Jordan

    Luay Jum’a

Authors
  1. Maher Ala’raj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maysam F. Abbod
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Munir Majdalawieh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Luay Jum’a
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maher Ala’raj.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ala’raj, M., Abbod, M.F., Majdalawieh, M. et al. A deep learning model for behavioural credit scoring in banks. Neural Comput & Applic 34, 5839–5866 (2022). https://doi.org/10.1007/s00521-021-06695-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06695-z

Keywords

Search

Navigation