Skip to main content
SpringerLink
Log in

Extreme learning machines’ ensemble selection with GRASP

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Credit scoring, which is also called credit risk assessment has attracted the attention of many financial institutions and much research has been carried out. In this work, a new Extreme Learning Machines’ (ELMs) Ensemble Selection algorithm based on the Greedy Randomized Adaptive Search Procedure (GRASP), referred to as ELMsGraspEnS, is proposed for credit risk assessment of enterprises. On the one hand, the ELM is used as the base learner for ELMsGraspEnS owing to its significant advantages including an extremely fast learning speed, good generalization performance, and avoidance of issues like local minima and overfitting. On the other hand, to ameliorate the local optima problem faced by classical greedy ensemble selection methods, we incorporated GRASP, a meta-heuristic multi-start algorithm for combinatorial optimization problems, into the solution of ensemble selection, and proposed an ensemble selection algorithm based on GRASP (GraspEnS) in our previous work. The GraspEnS algorithm has the following three advantages. (1) By incorporating a random factor, a solution is often able to escape local optima. (2) GraspEnS realizes a multi-start search to some degree. (3) A better performing subensemble can usually be found with GraspEnS. Moreover, not much research on applying ensemble selection approaches to credit scoring has been reported in the literature. In this paper, we integrate the ELM with GraspEnS, and propose a novel ensemble selection algorithm based on GRASP (ELMsGraspEnS). ELMsGraspEnS naturally inherits the inherent advantages of both the ELM and GraspEnS, effectively combining their advantages. The experimental results of applying ELMsGraspEnS to three benchmark real world credit datasets show that in most cases ELMsGraspEnS significantly improves the performance of credit risk assessment compared with several state-of-the-art algorithms. Thus, it can be concluded that ELMsGraspEnS simultaneously exhibits relatively high efficiency and effectiveness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Laha A (2007) Building contextual classifiers by integrating fuzzy rule based classification technique and k-nn method for credit scoring. Adv Eng Inform 21:281–291

    Article  MATH  Google Scholar 

  2. Li ST, Shiue W, Huang MH (2006) The evaluation of consumer loans using support vector machines. Expert Syst Appl 30(4):772–782

    Article  Google Scholar 

  3. Vellido A, Lisboa PJG, Vaughan J (1999) Neural networks in business: A survey of applications (1992–1998). Expert Syst Appl 17(1):51–70

    Article  Google Scholar 

  4. Karels G, Prakash A (1987) Multivariate normality and forecasting of business bankruptcy. J Business Finance Acc 14(4):573–593

    Article  Google Scholar 

  5. Reichert AK, Cho CC, Wagner GM (1983) An examination of the conceptual issues involved in developing credit-scoring models. J Bus Econ Stat 1(2):101–114

    Google Scholar 

  6. West D (2000) Neural network credit scoring models. Comput Oper Res 27(11–12):1131–1152

    Article  MATH  Google Scholar 

  7. Thomas LC (2000) A survey of credit and behavioral scoring: Forecasting financial risks of lending to customers. Int J Forecast 16(2):149–172

    Article  MATH  Google Scholar 

  8. Friedman JH (1991) Multivariate adaptive regression splines. Ann Stat pp. 1–141

  9. Huang Z, Chen H, Hsu CJ, Chen WH, Wu SS (2004) Credit rating analysis with support vector machines and neural network: A market comparative study. Decis Support Syst pp. 543–558

  10. Desai V, Crook J, Overstreet G (1996) A comparison of neural networks and linear scoring models in the credit union environment. EJOR 95(1):24–37

    Article  MATH  Google Scholar 

  11. Hung C, Chen JH (2009) A selective ensemble based on expected probabilities for bankruptcy prediction. Expert Syst Appl 36(3):5297–5303

    Article  MathSciNet  Google Scholar 

  12. Polikar R (2006) Ensemble based systems in decision making. IEEE Circ Syst Mag 6(3):21–45

    Article  Google Scholar 

  13. Buta P (1994) Mining for financial knowledge with CBR. AI Expert 9(2):34–41

    Google Scholar 

  14. Shin KS, Han I (2001) A case-based approach using inductive indexing for corporate bond rating. Decis Support Syst 32(1):41–52

    Article  Google Scholar 

  15. Baesens B, van Gestel T, Viaene S, Stepanova M, Suykens J, Vanthienen J (2003) Benchmarking state-of-the-art classification algorithms for credit scoring. J Oper Res Soc 54(6):1082–1088

    Article  Google Scholar 

  16. Huang CL, Chen MC, Wang CJ (2007) Credit scoring with a data mining approach based on support vector machines. Expert Syst Appl 33(4):847–856

    Article  MathSciNet  Google Scholar 

  17. Schebesch KB, Stecking R (2005) Support vector machines for classifying and describing credit applicants: Detecting typical and critical regions. J Oper Res Soc 56(9):1082–1088

    Article  MATH  Google Scholar 

  18. Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140

    MathSciNet  MATH  Google Scholar 

  19. Dietterich TG (2000) An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Mach Learn 40(2):139–157

    Article  Google Scholar 

  20. Opitz D, Maclin R (1999) Popular ensemble methods: An empirical study. J Artif Intell Res 11:169–198

    MATH  Google Scholar 

  21. Schapire RE (1990) The strength of weak learnability. Mach Learn 5(2):197–227

    Google Scholar 

  22. Wolpert DH (1992) Stacked generalization. Neural Netw 5(2):241–259

    Article  MathSciNet  Google Scholar 

  23. Tsai CF, Wu JW (2008) Using neural network ensembles for bankruptcy prediction and credit scoring. Expert Syst Appl 34(4):2639–2649

    Article  Google Scholar 

  24. Yu LA, Wang SY, Lai KK (2008) Credit risk assessment with a multistage neural network ensemble learning approach. Expert Syst Appl 34(2):1434–1444

    Article  Google Scholar 

  25. Nanni L, Lumini A (2009) An experimental comparison of ensemble of classifiers for bankruptcy prediction and credit scoring. Expert Syst Appl 36(2):3028–3033

    Article  Google Scholar 

  26. Hung C, Chen JH (2009) A selective ensemble based on expected probabilities for bankruptcy prediction. Expert Syst Appl 36(3):5297–5303

    Article  MathSciNet  Google Scholar 

  27. Partalas I, Tsoumakas G, Vlahavas I (2010) An ensemble uncertainty aware measure for directed hill climbing ensemble pruning. Mach Learn 81(2):257–282

    Article  MathSciNet  Google Scholar 

  28. Dai Q (2013) A competitive ensemble pruning approach based on cross-validation technique. Knowl-Based Syst 37:394–414

    Article  Google Scholar 

  29. Dai Q, Liu Z (2013) ModEnPBT: A modified backtracking ensemble pruning algorithm. Appl Soft Comput 13:4292–4302

    Article  Google Scholar 

  30. Dai Q (2013) A novel ensemble pruning algorithm based on randomized greedy selective strategy and ballot. Neurocomputing 122:258–265

    Article  Google Scholar 

  31. Liu Z, Dai Q, Liu NZ (2013) Ensemble selection by GRASP. Appl Intell

  32. Caruana R, Niculescu-Mizil A, Crew G, Ksikes A (2004) Ensemble selection from libraries of models, presented at Proceedings of the twenty-first international conference on machine learning

  33. Caruana R, Munson A, Niculescu-Mizil A (2006) Getting the most out of ensemble selection. In: Proceedings of the international conference on data mining (ICDM), pp. 828–833

  34. Partalas I, Tsoumakas G, Vlahavas I (2012) A study on greedy algorithm for ensemble pruning. In: Technical report TR-LPIS-360-1, Department of Informatics. Aristotle University of Thessaloniki, Greece

  35. Zhou Z-H, Wu J, Tang W (2002) Ensembling neural networks: many could be better than all. Artif Intell 137:239–263

    Article  MathSciNet  MATH  Google Scholar 

  36. Martinez-Munoz G, Suarz A (2004) Aggregation ordering in bagging, Presented at international conference on artificial intelligence and applications

  37. Banfield RE, Hall LO, Bowyer KW, Kegelmeyer WP (2005) Ensemble diversity measures and their application to thinning. Information Fusion 6:49–62

    Article  Google Scholar 

  38. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(2):489–59

    Article  Google Scholar 

  39. Rong HJ, Huang GB, Ong YS (2008) Extreme learning machine for multi-categories classification applications. Int JT Conf Neural Netw:1709–1713

  40. Duan GL, Huang ZW, Wang JR (2009) Extreme learning machine for bank clients classification. Int Conf Inf Manag, Innov Manag Ind Eng 2:496–499

    Google Scholar 

  41. Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multi-class classification. IEEE Trans, Systems, Man, Cyber vol. 42

  42. Subbulakshmi CV, Deepa SN, Malathi N (2012) Extreme learning machine for two category data classification. IEEE Int Conf Adv Commun Control Comput Technol pp. 458–461

  43. Landa-Torres I, Ortiz-García EG, Salcedo-Sanz SM, Segovia-Vargas J, Gil-López S, Miranda M, Leiva-Murillo JM, Del Ser J (2012) Evaluating the internationalization success of companies through a hybrid grouping Harmony Search - Extreme Learning Machine approach. IEEE J Sel Top Sig Process 6(4):1– 11

    Article  Google Scholar 

  44. Feo TA, Resende MG (1995) Greedy randomized adaptive search procedures. J Glob Optim 6:109–133

    Article  MathSciNet  MATH  Google Scholar 

  45. Gevezes T, Pitsoulis L (2013) A greedy randomized adaptive search procedure with path relinking for the shortest superstring problem. J Glob Optim pp. 1–25

  46. Resende MGC, Ribeiro C (2003) Greedy randomized adaptive search procedures. In: Handbook of Metaheuristics. Springer, pp 219–249

  47. Huang GB (2003) Learning capability and storage capacity of two-hidden-layer feedforward networks. IEEE Trans Neural Netw 14:274–281

    Article  Google Scholar 

  48. Huang GB, Babri HA (1998) Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions. IEEE Trans Neural Netw 9(1):224–229

    Article  Google Scholar 

  49. Dietterich TG (2000) Ensemble methods in machine learning. In: Proceedings of the 1 st international workshop in multiple classifier systems, pp. 1–15

  50. Breiman L (1996) Bagging predictors. Mach Learn 24:123–140

    MathSciNet  MATH  Google Scholar 

  51. Schapire RE (1990) The strength of weak learnability. Mach Learn 5(2):197–227

    Google Scholar 

  52. Layeb A, Selmane M, Bencheikh Elhoucine M (2013) A new greedy randomized adaptive search procedure for multiple sequence alignment. Int J Bioinforma Res Appl 9:1–14

    Google Scholar 

  53. UCI repository of machine learning databases (2007). http://www.ics.uci.edu/~mlearn/MLRepository.html or ftp.ics.uci.edu:pub/machine-learning-databases

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China under Grants no. 61473150, 61100108, and 61375021. It is also supported by the Natural Science Foundation of Jiangsu Province of China under Grant no. BK20131365, and the Qing Lan Project, no. YPB13001. We would like to express our appreciation for the valuable comments from reviewers and editors.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, China

    Ting Zhang, Qun Dai & Zhongchen Ma

Authors
  1. Ting Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qun Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhongchen Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qun Dai.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, T., Dai, Q. & Ma, Z. Extreme learning machines’ ensemble selection with GRASP. Appl Intell 43, 439–459 (2015). https://doi.org/10.1007/s10489-015-0653-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-015-0653-2

Keywords

Search

Navigation