Soft Computing

, Volume 23, Issue 24, pp 13139–13159 | Cite as

An evolutionary deep belief network extreme learning-based for breast cancer diagnosis

  • Somayeh Ronoud
  • Shahrokh AsadiEmail author
Methodologies and Application


Cancer is one of the leading causes of morbidity and mortality worldwide with increasing prevalence. Breast cancer is the most common type among women, and its early diagnosis is crucially important. Cancer diagnosis is a classification problem, where its nature requires very high classification accuracy. As artificial neural networks (ANNs) have a high capability in modeling nonlinear relationships in data, they are frequently used as good global approximators in prediction and classification problems. However, in complex problems such as diagnosing breast cancer, shallow ANNs may cause certain problems due to their limited capacity of modeling and representation. Therefore, deep architectures are essential for extracting the complicated structure of cancer data. Under such circumstances, deep belief networks (DBNs) are appropriate choice whose application involves two major challenges: (1) the method of fine-tuning the network weights and biases and (2) the number of hidden layers and neurons. The present study suggests two novel evolutionary methods, namely E(T)-DBN-BP-ELM and E(T)-DBN-ELM-BP, that address the first challenge via combining DBN with extreme learning machine (ELM) classifier. In the proposed methods, because of the very large solution space of DBN topologies, the genetic algorithm (GA), which is able to search globally in the solutions space wondrously, has been applied for architecture optimization to tackle the second challenge. The third proposed method in this paper, E(TW)-DBN, uses GA to solve both challenges, in which DBN topology and weights evolve simultaneously. The proposed models are tested using two breast cancer datasets and compared with the state-of-the-art methods in the literature in terms of classification performance metrics and area under ROC (AUC) curves. According to the results, the proposed methods exhibit very high diagnostic performance in classification of breast cancer.


Medical decision support system Deep belief network Extreme learning machine Breast cancer diagnosis 


Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest to this work and this study was not funded by any grant.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.


  1. Abdel-Zaher AM, Eldeib AM (2016) Breast cancer classification using deep belief networks. Expert Syst Appl 46:139–144CrossRefGoogle Scholar
  2. Abonyi J, Szeifert F (2003) Supervised fuzzy clustering for the identification of fuzzy classifiers. Pattern Recogn Lett 24:2195–2207zbMATHCrossRefGoogle Scholar
  3. Ahmadizar F, Soltanian K, AkhlaghianTab F, Tsoulos I (2015) Artificial neural network development by means of a novel combination of grammatical evolution and genetic algorithm. Eng Appl Artif Intell 39:1–13CrossRefGoogle Scholar
  4. Albrecht AA, Lappas G, Vinterbo SA, Wong C, Ohno-Machado L (2002) Two applications of the LSA machine, neural information processing, 2002. In: Proceedings of the 9th international conference on ICONIP’02. Publishing, pp 184–189Google Scholar
  5. Asadi S (2019) Evolutionary fuzzification of RIPPER for regression: case study of stock prediction. Neurocomputing 331:121–137CrossRefGoogle Scholar
  6. Asadi S, Shahrabi J (2016) ACORI: a novel ACO algorithm for Rule Induction. Knowl Based Syst 97:175–187CrossRefGoogle Scholar
  7. Asadi S, Shahrabi J (2017) Complexity-based parallel rule induction for multiclass classification. Inf Sci 380:53–73CrossRefGoogle Scholar
  8. Asadi S, Hadavandi E, Mehmanpazir F, Nakhostin MM (2012) Hybridization of evolutionary Levenberg–Marquardt neural networks and data pre-processing for stock market prediction. Knowl Based Syst 35:245–258CrossRefGoogle Scholar
  9. Asadi S, Shahrabi J, Abbaszadeh P, Tabanmehr S (2013) A new hybrid artificial neural networks for rainfall–runoff process modeling. Neurocomputing 121:470–480CrossRefGoogle Scholar
  10. Bengio Y (2009) Learning deep architectures for AI. Foundations and trends® Mach Learn 2:1–127Google Scholar
  11. Bhardwaj A, Tiwari A (2015) Breast cancer diagnosis using genetically optimized neural network model. Expert Syst Appl 42:4611–4620CrossRefGoogle Scholar
  12. Bradley AP (1997) The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recogn 30:1145–1159CrossRefGoogle Scholar
  13. Cao L-L, Huang W-B, Sun F-C (2016) Building feature space of extreme learning machine with sparse denoising stacked-autoencoder. Neurocomputing 174:60–71CrossRefGoogle Scholar
  14. Chen H-L, Yang B, Liu J, Liu D-Y (2011) A support vector machine classifier with rough set-based feature selection for breast cancer diagnosis. Expert Syst Appl 38:9014–9022CrossRefGoogle Scholar
  15. Çınar M, Engin M, Engin EZ, Ateşçi YZ (2009) Early prostate cancer diagnosis by using artificial neural networks and support vector machines. Expert Syst Appl 36:6357–6361CrossRefGoogle Scholar
  16. Ciompi F, de Hoop B, van Riel SJ, Chung K, Scholten ET, Oudkerk M, de Jong PA, Prokop M, van Ginneken B (2015) Automatic classification of pulmonary peri-fissural nodules in computed tomography using an ensemble of 2D views and a convolutional neural network out-of-the-box. Med Image Anal 26:195–202CrossRefGoogle Scholar
  17. Deng L, Yu D (2014) Deep learning: methods and applications. Found Trends Signal Process 7:197–387MathSciNetzbMATHCrossRefGoogle Scholar
  18. Flores-Fernández JM, Herrera-López EJ, Sánchez-Llamas F, Rojas-Calvillo A, Cabrera-Galeana PA, Leal-Pacheco G, González-Palomar MG, Femat R, Martínez-Velázquez M (2012) Development of an optimized multi-biomarker panel for the detection of lung cancer based on principal component analysis and artificial neural network modeling. Expert Syst Appl 39:10851–10856CrossRefGoogle Scholar
  19. Fotouhi S, Asadi S, Kattan MW (2019) A comprehensive data level analysis for cancer diagnosis on imbalanced data. J Biomed Informs 90:1–30Google Scholar
  20. Frénay B, Verleysen M (2011) Parameter-insensitive kernel in extreme learning for non-linear support vector regression. Neurocomputing 74:2526–2531CrossRefGoogle Scholar
  21. Garro BA, Rodríguez K, Vázquez RA (2016) Classification of DNA microarrays using artificial neural networks and ABC algorithm. Appl Soft Comput 38:548–560CrossRefGoogle Scholar
  22. Guo Y, Liu Y, Oerlemans A, Lao S, Wu S, Lew MS (2015) Deep learning for visual understanding: A review. Neurocomputing 187:27–48CrossRefGoogle Scholar
  23. Hinton G (2010) A practical guide to training restricted Boltzmann machines. Momentum 9:926Google Scholar
  24. Hinton GE, Osindero S, Teh Y-W (2006) A fast learning algorithm for deep belief nets. Neural Comput 18:1527–1554MathSciNetzbMATHCrossRefGoogle Scholar
  25. Hrasko R, Pacheco AG, Krohling RA (2015) Time series prediction using restricted Boltzmann machines and backpropagation. Proc Comput Sci 55:990–999CrossRefGoogle Scholar
  26. Huang J, Ling CX (2005) Using AUC and accuracy in evaluating learning algorithms. IEEE Trans Knowl Data Eng 17:299–310CrossRefGoogle Scholar
  27. Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501CrossRefGoogle Scholar
  28. Huang G-B, Ding X, Zhou H (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74:155–163CrossRefGoogle Scholar
  29. Huang G-B, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cyber Part B Cybern 42:513–529CrossRefGoogle Scholar
  30. Karabatak M (2015) A new classifier for breast cancer detection based on Naïve Bayesian. Measurement 72:32–36CrossRefGoogle Scholar
  31. Kazemi S, Hadavandi E, Shamshirband S, Asadi S (2016) A novel evolutionary-negative correlated mixture of experts model in tourism demand estimation. Comput Hum Behav 64:641–655CrossRefGoogle Scholar
  32. Keyvanrad MA, Homayounpour MM (2015) Deep belief network training improvement using elite samples minimizing free energy. Int J Pattern Recognit Artif Intell 29:1551006CrossRefGoogle Scholar
  33. Kong H, Lai Z, Wang X, Liu F (2015) Breast cancer discriminant feature analysis for diagnosis via jointly sparse learning. NeurocomputingGoogle Scholar
  34. Koyuncu H, Ceylan R (2013) Artificial neural network based on rotation forest for biomedical pattern classification. In: 2013 36th international conference on telecommunications and signal processing (TSP). Publishing, pp 581–585Google Scholar
  35. Längkvist M, Karlsson L, Loutfi A (2014) A review of unsupervised feature learning and deep learning for time-series modeling. Pattern Recogn Lett 42:11–24CrossRefGoogle Scholar
  36. Larochelle H, Erhan D, Courville A, Bergstra J, Bengio Y (2007) An empirical evaluation of deep architectures on problems with many factors of variation. In: Proceedings of the 24th international conference on machine learning. Publishing, pp 473–480Google Scholar
  37. Lavanya D, Rani DKU (2011) Analysis of feature selection with classification: breast cancer datasets. IJCSE 2:756–763Google Scholar
  38. Le Roux N, Bengio Y (2008) Representational power of restricted Boltzmann machines and deep belief networks. Neural Comput 20:1631–1649MathSciNetzbMATHCrossRefGoogle Scholar
  39. Liu Q, He Q, Shi Z (2008) Extreme support vector machine classifier, Pacific-Asia conference on knowledge discovery and data mining. Publishing, pp 222–233Google Scholar
  40. Malmir H, Farokhi F, Sabbaghi-Nadooshan R (2013) Optimization of data mining with evolutionary algorithms for cloud computing application. In: 2013 3rd international econference on computer and knowledge engineering (ICCKE). Publishing, pp 343–347Google Scholar
  41. Mansourypoor F, Asadi S (2017) Development of a reinforcement learning-based evolutionary fuzzy rule-based system for diabetes diagnosis. Comput Biol Med 91:337–352CrossRefGoogle Scholar
  42. Marcano-Cedeño A, Quintanilla-Domínguez J, Andina D (2011) WBCD breast cancer database classification applying artificial metaplasticity neural network. Expert Syst Appl 38:9573–9579CrossRefGoogle Scholar
  43. Mehmanpazir F, Asadi S (2017) Development of an evolutionary fuzzy expert system for estimating future behavior of stock price. J Ind Eng Int 13:29–46CrossRefGoogle Scholar
  44. Milovic B (2012) Prediction and decision making in health care using data mining. IJPHS 1:69–78CrossRefGoogle Scholar
  45. Nauck D, Kruse R (1999) Obtaining interpretable fuzzy classification rules from medical data. Artif Intell Med 16:149–169CrossRefGoogle Scholar
  46. Örkcü HH, Bal H (2011) Comparing performances of backpropagation and genetic algorithms in the data classification. Expert Syst Appl 38:3703–3709CrossRefGoogle Scholar
  47. Palm RB (2012) Prediction as a candidate for learning deep hierarchical models of data. Technical University of Denmark, Palm, p 25Google Scholar
  48. Park K, Ali A, Kim D, An Y, Kim M, Shin H (2013) Robust predictive model for evaluating breast cancer survivability. Eng Appl Artif Intell 26:2194–2205CrossRefGoogle Scholar
  49. Pena-Reyes CA, Sipper M (1999) A fuzzy-genetic approach to breast cancer diagnosis. Artif Intell Med 17:131–155CrossRefGoogle Scholar
  50. Pham D, Sagiroglu S (2000) Neural network classification of defects in veneer boards. Proc Inst Mech Eng Part B J Eng Manuf 214:255–258CrossRefGoogle Scholar
  51. Polat K, Güneş S (2007) Breast cancer diagnosis using least square support vector machine. Digit Signal Proc 17:694–701CrossRefGoogle Scholar
  52. Qu B, Lang B, Liang J, Qin A, Crisalle O (2016) Two-hidden-layer extreme learning machine for regression and classification. Neurocomputing 175:826–834CrossRefGoogle Scholar
  53. Quinlan JR (1996) Improved use of continuous attributes in C4.5. J Artif Intell Res 4:77–90zbMATHCrossRefGoogle Scholar
  54. Razavi SH, Ebadati EOM, Asadi S, Kaur H (2015) An efficient grouping genetic algorithm for data clustering and big data analysis. Computational intelligence for big data analysis. Publishing, pp 119–142Google Scholar
  55. Saritas I, Ozkan IA, Sert IU (2010) Prognosis of prostate cancer by artificial neural networks. Expert Syst Appl 37:6646–6650CrossRefGoogle Scholar
  56. Shahrabi J, Hadavandi E, Asadi S (2013) Developing a hybrid intelligent model for forecasting problems: case study of tourism demand time series. Knowl Based Syst 43:112–122CrossRefGoogle Scholar
  57. Shen F, Chao J, Zhao J (2015) Forecasting exchange rate using deep belief networks and conjugate gradient method. Neurocomputing 167:243–253CrossRefGoogle Scholar
  58. Smolensky P (1986) Information processing in dynamical systems: Foundations of harmony theory. Parallel Distributed Processing: Volume 1: Foundations. MIT Press, Cambridge 1987:194–281Google Scholar
  59. Sumbaly R, Vishnusri N, Jeyalatha S (2014) Diagnosis of breast cancer using decision tree data mining technique. Int J Comput Appl 98:16–24Google Scholar
  60. Tahan MH, Asadi S (2018a) EMDID: evolutionary multi-objective discretization for imbalanced datasets. Inf Sci 432:442–461MathSciNetCrossRefGoogle Scholar
  61. Tahan MH, Asadi S (2018b) MEMOD: a novel multivariate evolutionary multi-objective discretization. Soft Comput 22:301–323CrossRefGoogle Scholar
  62. Tieleman T (2008) Training restricted Boltzmann machines using approximations to the likelihood gradient. In: Proceedings of the 25th international conference on machine learning. Publishing, pp 1064–1071Google Scholar
  63. Tieleman T, Hinton G (2009) Using fast weights to improve persistent contrastive divergence. In: Proceedings of the 26th annual international conference on machine learning. Publishing, pp 1033–1040Google Scholar
  64. Übeyli ED (2007) Implementing automated diagnostic systems for breast cancer detection. Expert Syst Appl 33:1054–1062CrossRefGoogle Scholar
  65. Wang Y, Xie Z, Xu K, Dou Y, Lei Y (2016) An efficient and effective convolutional auto-encoder extreme learning machine network for 3d feature learning. Neurocomputing 174:988–998CrossRefGoogle Scholar
  66. Wu Y, Wu Y, Wang J, Yan Z, Qu L, Xiang B, Zhang Y (2011) An optimal tumor marker group-coupled artificial neural network for diagnosis of lung cancer. Expert Syst Appl 38:11329–11334CrossRefGoogle Scholar
  67. Xue B, Zhang M, Browne WN (2014) Particle swarm optimisation for feature selection in classification: novel initialisation and updating mechanisms. Appl Soft Comput 18:261–276CrossRefGoogle Scholar
  68. Yu W, Zhuang F, He Q, Shi Z (2015) Learning deep representations via extreme learning machines. Neurocomputing 149:308–315CrossRefGoogle Scholar
  69. Zheng B, Yoon SW, Lam SS (2014) Breast cancer diagnosis based on feature extraction using a hybrid of K-means and support vector machine algorithms. Expert Syst Appl 41:1476–1482CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Data Mining Laboratory, Department of Engineering, College of FarabiUniversity of TehranTehranIran

Personalised recommendations