Skip to main content
SpringerLink
Log in

Advances in Machine Learning Modeling Reviewing Hybrid and Ensemble Methods

  • Conference paper
  • First Online:
Engineering for Sustainable Future (INTER-ACADEMIA 2019)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 101))

Included in the following conference series:

Abstract

The conventional machine learning (ML) algorithms are continuously advancing and evolving at a fast-paced by introducing the novel learning algorithms. ML models are continually improving using hybridization and ensemble techniques to empower computation, functionality, robustness, and accuracy aspects of modeling. Currently, numerous hybrid and ensemble ML models have been introduced. However, they have not been surveyed in a comprehensive manner. This paper presents the state of the art of novel ML models and their performance and application domains through a novel taxonomy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Abbreviations

ANN:

Artificial neural network

ELM:

Extreme learning machine

ML:

Machine learning

SVM:

Support vector machine

WNN:

Wavelet neural networks

DL:

Deep learning

ARIMA:

Autoregressive integrated moving average

EE-ANT:

Ensemble empirical with adaptive noise technology

DA-KF:

Data assimilation Kalman filter-based

OSELM:

Online sequential extreme learning machine

BAGNBT:

Bagging-based naïve bayes trees

EEMD:

Ensemble empirical mode decomposition

GOA:

Grasshopper optimization algorithm

HybPAS:

Hybrid of linear regression-deep neural network

TSM:

Trauma Severity model

GBDT:

Gradient boosting decision tree

EBFTM:

Evidential belief function and tree-based models

DTFNN:

Decision tree overfitting and neural network

ICEEMDMAN:

Improved complete ensemble empirical mode decomposition method with adaptive noise

RF:

Random forest

References

  1. Choubin, B., et al.: Snow avalanche hazard prediction using machine learning methods. J. Hydrol. 577 (2019)

    Article  Google Scholar 

  2. Dehghani, M., et al.: Prediction of hydropower generation using Grey wolf optimization adaptive neuro-fuzzy inference system. Energies 12(2) (2019)

    Article  Google Scholar 

  3. Mosavi, A., Bathla, Y., Varkonyi-Koczy, A.: Predicting the future using web knowledge: state of the art survey. In: Luca, D., Sirghi, L., Costin, C. (eds.), pp. 341–349. Springer (2018)

    Google Scholar 

  4. Mosavi, A., Rabczuk, T.: Learning and intelligent optimization for material design innovation. In: Kvasov, D.E., et al. (eds.), pp. 358–363. Springer (2017)

    Google Scholar 

  5. Mosavi, A., Rabczuk, T., Várkonyi-Kóczy, A.R.: Reviewing the novel machine learning tools for materials design. In: Luca, D., Sirghi, L., Costin, C. (eds.), pp. 50–58. Springer (2018)

    Google Scholar 

  6. Qasem, S.N., et al.: Estimating daily dew point temperature using machine learning algorithms. Water (Switzerland) 11(3) (2019)

    Article  Google Scholar 

  7. Taherei Ghazvinei, P., et al.: Sugarcane growth prediction based on meteorological parameters using extreme learning machine and artificial neural network. Eng. Appl. Comput. Fluid Mech. 12(1), 738–749 (2018)

    Google Scholar 

  8. Torabi, M., et al.: A Hybrid clustering and classification technique for forecasting short-term energy consumption. Environ. Prog. Sustain. Energy 38(1), 66–76 (2019)

    Article  Google Scholar 

  9. Torabi, M., et al.: A hybrid machine learning approach for daily prediction of solar radiation. In: Lecture Notes in Networks and Systems, pp. 266–274. Springer (2019)

    Google Scholar 

  10. Dineva, A., et al.: Review of soft computing models in design and control of rotating electrical machines. Energies 12(6) (2019)

    Article  Google Scholar 

  11. Mosavi, A., Ozturk, P., Chau, K.W.: Flood prediction using machine learning models: literature review. Water (Switzerland) 10(11) (2018)

    Article  Google Scholar 

  12. Mosavi, A., et al.: State of the art of machine learning models in energy systems, a systematic review. Energies 12(7) (2019)

    Article  Google Scholar 

  13. Mosavi, A., Várkonyi-Kóczy, A.R.: Integration of machine learning and optimization for robot learning. In: Jablonski, R., Szewczyk, R. (eds.), pp. 349–355. Springer (2017)

    Google Scholar 

  14. Cheng, L., Yu, T.: A new generation of AI: A review and perspective on machine learning technologies applied to smart energy and electric power systems. Int. J. Energy Res. 43(6), 1928–1973 (2019)

    Article  Google Scholar 

  15. Cheng, L., et al.: Machine Learning for Energy and Electric Power Systems: State of the Art and Prospects. Dianli Xitong Zidonghua/Autom. Electric Power Syst. 43(1), 15–31 (2019)

    Google Scholar 

  16. Chou, J.S., Tran, D.S.: Forecasting energy consumption time series using machine learning techniques based on usage patterns of residential householders. Energy 709–726 (2018)

    Article  Google Scholar 

  17. Chatterjee, B., et al.: RF-PUF: enhancing IoT security through authentication of wireless nodes using in-situ machine learning. IEEE Internet of Things J. 6(1), 388–398 (2019)

    Article  Google Scholar 

  18. Panesar, S.S., et al.: Machine learning versus logistic regression methods for 2-year mortality prognostication in a small, heterogeneous glioma database. World Neurosurg X 2 (2019)

    Article  Google Scholar 

  19. Thomas, P.B.M., et al.: Feasibility of simple machine learning approaches to support detection of non-glaucomatous visual fields in future automated glaucoma clinics. Eye (Basingstoke) 33(7), 1133–1139 (2019)

    Google Scholar 

  20. Alhajri, M.I., Ali, N.T., Shubair, R.M.: Classification of indoor environments for IoT applications: a machine learning approach. IEEE Antennas Wirel. Propag. Lett. 17(12), 2164–2168 (2018)

    Article  Google Scholar 

  21. Jamil, A., Bayram, B.: The delineation of tea gardens from high resolution digital orthoimages using mean-shift and supervised machine learning methods. Geocarto Int. (2019)

    Google Scholar 

  22. Maxwell, A.E., Warner, T.A., Fang, F.: Implementation of machine-learning classification in remote sensing: an applied review. Int. J. Remote Sens. 39(9), 2784–2817 (2018)

    Article  Google Scholar 

  23. Sehgal, V., et al.: Machine learning creates a simple endoscopic classification system that improves dysplasia detection in barrett’s oesophagus amongst non-expert endoscopists. Gastroenterol. Res. Pract. 2018 (2018)

    Google Scholar 

  24. Azeem, M.I., et al.: Machine learning techniques for code smell detection: a systematic literature review and meta-analysis. Inf. Softw. Technol. 108, 115–138 (2019)

    Article  Google Scholar 

  25. Jabeen, A., Ranganathan, S.: Applications of machine learning in GPCR bioactive ligand discovery. Curr. Opin. Struct. Biol. 55, 66–76 (2019)

    Article  Google Scholar 

  26. Xu, C., Jackson, S.A.: Machine learning and complex biological data. Genome Biol. 20(1) (2019)

    Google Scholar 

  27. Zhang, Z., Sejdić, E.: Radiological images and machine learning: trends, perspectives, and prospects. Comput. Biol. Med. 108, 354–370 (2019)

    Article  Google Scholar 

  28. Bock, F.E., et al., A review of the application of machine learning and data mining approaches in continuum materials mechanics. Front. Mater. 6 (2019)

    Google Scholar 

  29. Ekins, S., et al.: Exploiting machine learning for end-to-end drug discovery and development. Nat. Mater. 18(5), 435–441 (2019)

    Article  Google Scholar 

  30. Woldaregay, A.Z., et al.: Data-driven blood glucose pattern classification and anomalies detection: Machine-learning applications in type 1 diabetes. J. Med. Internet Res. 21(5) (2019)

    Article  Google Scholar 

  31. Najafzadeh, M., Ghaemi, A.: Prediction of the five-day biochemical oxygen demand and chemical oxygen demand in natural streams using machine learning methods. Environ. Monit. Assess. 191(6) (2019)

    Google Scholar 

  32. Singh, H., Rana, P.S., Singh, U.: Prediction of drug synergy in cancer using ensemble-based machine learning techniques. Mod. Phys. Lett. B 32(11) (2018)

    Article  Google Scholar 

  33. Choubin, B., et al.: River suspended sediment modelling using the CART model: A comparative study of machine learning techniques. Sci. Total Environ. 615, 272–281 (2018)

    Article  Google Scholar 

  34. Zarkogianni, K., Athanasiou, M., Thanopoulou, A.C.: Comparison of machine learning approaches toward assessing the risk of developing cardiovascular disease as a long-term diabetes complication. IEEE J. Biomed. Health Inf. 22(5), 1637–1647 (2018)

    Article  Google Scholar 

  35. Aram, F., et al.: Design and validation of a computational program for analysing mental maps: aram mental map analyzer. Sustainability (Switzerland) 11(14) (2019)

    Article  Google Scholar 

  36. Asadi, E., et al.: Groundwater quality assessment for drinking and agricultural purposes in Tabriz Aquifer, Iran (2019)

    Google Scholar 

  37. Asghar, M.Z., Subhan, F., Imran, M., Kundi, F.M., Shamshirband, S., Mosavi, A., Csiba, P., Várkonyi-Kóczy, A.R.: Performance evaluation of supervised machine learning techniques for efficient detection of emotions from online content (2019), 2019080019. https://doi.org/10.20944/preprints201908.0019.v1

  38. Bemani, A., Baghban, A., Shamshirband, S., Mosavi, A., Csiba, P., Várkonyi-Kóczy, A.R.: Applying ANN, ANFIS, and LSSVM models for estimation of acid solvent solubility in supercritical CO2 (2019), 2019060055. https://doi.org/10.20944/preprints201906.0055.v2

  39. Mosavi, A., et al.: Prediction of multi-inputs bubble column reactor using a novel hybrid model of computational fluid dynamics and machine learning. Eng. Appl. Comput. Fluid Mech. 13(1), 482–492 (2019)

    Google Scholar 

  40. Nosratabadi, S., et al.: Sustainable business models: a review. Sustainability (Switzerland) 11(6) (2019)

    Article  Google Scholar 

  41. Rezakazemi, M., Mosavi, A., Shirazian, S.: ANFIS pattern for molecular membranes separation optimization. J. Mol. Liq. 274, 470–476 (2019)

    Article  Google Scholar 

  42. Riahi-Madvar, H., et al.: Comparative analysis of soft computing techniques RBF, MLP, and ANFIS with MLR and MNLR for predicting grade-control scour hole geometry. Eng. Appl. Comput. Fluid Mech. 13(1), 529–550 (2019)

    Google Scholar 

  43. Shabani, S., Samadianfard, S., Taghi Sattari, M., Shamshirband, S., Mosavi, A., Kmet, T., Várkonyi-Kóczy, A.R.: Modeling daily pan evaporation in humid climates using gaussian process regression (2019), 2019070351. (https://doi.org/10.20944/preprints201907.0351.v1)

  44. Shamshirband, S., Hadipoor, M., Baghban, A., Mosavi, A., Bukor J., Várkonyi-Kóczy, A.R.: Developing an ANFIS-PSO model to predict mercury emissions in combustion flue gases (2019), 2019070165. https://doi.org/10.20944/preprints201907.0165.v1

  45. Shamshirband, S., et al.: Ensemble models with uncertainty analysis for multi-day ahead forecasting of chlorophyll a concentration in coastal waters. Eng. Appl. Comput. Fluid Mech. 13(1), 91–101 (2019)

    Google Scholar 

  46. Shamshirband, S., Mosavi, A., Rabczuk, T.: Particle swarm optimization model to predict scour depth around bridge pier (2019). arXiv:1906.08863

  47. Choubin, B., et al.: An ensemble prediction of flood susceptibility using multivariate discriminant analysis, classification and regression trees, and support vector machines. Sci. Total Environ. 651, 2087–2096 (2019)

    Article  Google Scholar 

  48. Dineva, A., et al.: Multi-label classification for fault diagnosis of rotating electrical machines (2019)

    Google Scholar 

  49. Farzaneh-Gord, M., et al.: Numerical simulation of pressure pulsation effects of a snubber in a CNG station for increasing measurement accuracy. Eng. Appl. Comput. Fluid Mech. 13(1), 642–663 (2019)

    Google Scholar 

  50. Ghalandari, M., et al.: Investigation of submerged structures’ flexibility on sloshing frequency using a boundary element method and finite element analysis. Eng. Appl. Comput. Fluid Mech. 13(1), 519–528 (2019)

    Google Scholar 

  51. Ghalandari, M., et al.: Flutter speed estimation using presented differential quadrature method formulation. Eng. Appl. Comput. Fluid Mech. 13(1), 804–810 (2019)

    Google Scholar 

  52. Karballaeezadeh, N., et al.: Prediction of remaining service life of pavement using an optimized support vector machine (case study of Semnan-Firuzkuh road). Eng. Appl. Comput. Fluid Mech. 13(1), 188–198 (2019)

    Google Scholar 

  53. Menad, N.A., et al.: Modeling temperature dependency of oil-water relative permeability in thermal enhanced oil recovery processes using group method of data handling and gene expression programming. Eng. Appl. Comput. Fluid Mech. 13(1), 724–743 (2019)

    MathSciNet  Google Scholar 

  54. Mohammadzadeh, S., et al.: Prediction of compression index of fine-grained soils using a gene expression programming model. Infrastructures 4(2), 26 (2019)

    Article  Google Scholar 

  55. Mosavi, A., Edalatifar, M.: A hybrid neuro-fuzzy algorithm for prediction of reference evapotranspiration. In: Lecture Notes in Networks and Systems, pp. 235–243. Springer (2019)

    Google Scholar 

  56. Mosavi, A., Lopez, A., Várkonyi-Kóczy, A.R.: Industrial applications of big data: state of the art survey. Luca, D., Sirghi, L., Costin, C. (eds.), pp. 225–232. Springer (2018)

    Google Scholar 

  57. Bui, D.T., et al.: Shallow landslide prediction using a novel hybrid functional machine learning algorithm. Remote Sens. 11(8) (2019)

    Google Scholar 

  58. Pham, B.T., et al.: Landslide susceptibility modeling using Reduced Error Pruning Trees and different ensemble techniques: hybrid machine learning approaches. Catena 175, 203–218 (2019)

    Article  Google Scholar 

  59. Zhang, X., Mahadevan, S.: Ensemble machine learning models for aviation incident risk prediction. Decis. Support Syst. 116, 48–63 (2019)

    Article  Google Scholar 

  60. Jaiswal, A., Malhotra, R.: Software reliability prediction using machine learning techniques. Int. J. Syst. Assur. Eng. Manag. 9(1), 230–244 (2018)

    Article  Google Scholar 

  61. Khagi, B., Kwon, G.R., Lama, R.: Comparative analysis of Alzheimer’s disease classification by CDR level using CNN, feature selection, and machine-learning techniques. Int. J. Imaging Syst. Technol. (2019)

    Google Scholar 

  62. Kumari, M., et al.: Comparative analysis of machine learning based QSAR models and molecular docking studies to screen potential anti-tubercular inhibitors against InhA of mycobacterium tuberculosis. Int. J. Comput. Biol. Drug Des. 11(3), 209–235 (2018)

    Article  Google Scholar 

  63. Bataineh, A.A.: A comparative analysis of nonlinear machine learning algorithms for breast cancer detection. Int. J. Mach. Learn. Comput. 9(3), 248–254 (2019)

    Article  MathSciNet  Google Scholar 

  64. Manzoor, S.I., Singla, J.: A comparative analysis of machine learning techniques for spam detection. Int. J. Adv. Trends Comput. Sci. Eng. 8(3), 810–814 (2019)

    Article  Google Scholar 

  65. Odugu, K., Rajasekar, B.: Comparative analysis on supervised machine learning models for future wireless communication networks. Int. J. Innov. Technol. Explor. Eng. 8(6), 721–723 (2019)

    Google Scholar 

  66. Hou Q, et al.: An adaptive hybrid model for short-term urban traffic flow prediction. Phys. A Stat. Mech. Appl. 527 (2019)

    Article  Google Scholar 

  67. Du, P., et al.: A novel hybrid model for short-term wind power forecasting. Appl. Soft Comput. J. 80, 93–106 (2019)

    Article  Google Scholar 

  68. Zhang, W., He, H., Zhang, S.: A novel multi-stage hybrid model with enhanced multi-population niche genetic algorithm: An application in credit scoring. Expert Syst. Appl. 121, 221–232 (2019)

    Article  Google Scholar 

  69. Pham, B.T., Prakash, I.: A novel hybrid model of Bagging-based Naïve Bayes Trees for landslide susceptibility assessment. Bull. Eng. Geol. Env. 78(3), 1911–1925 (2019)

    Article  Google Scholar 

  70. Wu, J., et al.: A new hybrid model to predict the electrical load in five states of Australia. Energy, 598–609 (2019)

    Google Scholar 

  71. Albalawi, F., et al.: Hybrid model for efficient prediction of poly(A) signals in human genomic DNA. Methods (2019)

    Google Scholar 

  72. Gorczyca, M.T., Toscano, N.C., Cheng, J.D.: The trauma severity model: an ensemble machine learning approach to risk prediction. Comput. Biol. Med. 108, 9–19 (2019)

    Article  Google Scholar 

  73. Wang, Q.F., Xu, M., Hussain, A.: Large-scale ensemble model for customer churn prediction in search ads. Cogn. Comput. 11(2), 262–270 (2019)

    Article  Google Scholar 

  74. Naghibi, S.A., et al.: Application of rotation forest with decision trees as base classifier and a novel ensemble model in spatial modeling of groundwater potential. Environ. Monit. Assess. 191(4) (2019)

    Google Scholar 

  75. Ali, M., Prasad, R.: Significant wave height forecasting via an extreme learning machine model integrated with improved complete ensemble empirical mode decomposition. Renew. Sustain. Energy Rev. 281–295 (2019)

    Article  Google Scholar 

  76. Yamanaka, A., Maeda, Y., Sasaki, K.: Ensemble Kalman filter-based data assimilation for three-dimensional multi-phase-field model: Estimation of anisotropic grain boundary properties. Mater. Des. 165 (2019)

    Article  Google Scholar 

  77. Yadav, D.C., Pal, S.: To generate an ensemble model for women thyroid prediction using data mining techniques. Asian Pac. J. Cancer Prev. 20(4), 1275–1281 (2019)

    Article  Google Scholar 

  78. Ardabili, S., Mosavi, A., Mahmoudi, A., Mesri Gundoshmian, T., Nosratabadi, S., Var-konyi-Koczy, A.: Modelling temperature variation of mushroom growing hall using artificial neural networks (2019)

    Google Scholar 

  79. Mesri Gundoshmian, T., Ardabili, S., Mosavi, A., Varkonyi-Koczy, A.: Prediction of combine harvester performance using hybrid machine learning modeling and response surface methodology (2019)

    Google Scholar 

  80. Ardabili, S., Mosavi, A., Varkonyi-Koczy, A.: Systematic review of deep learning and machine learning models in biofuels research (2019)

    Google Scholar 

  81. Ardabili, S., Mosavi, A., Varkonyi-Koczy, A.: Advances in machine learning model-ing reviewing hybrid and ensemble methods (2019)

    Google Scholar 

  82. Ardabili, S., Mosavi, A., Varkonyi-Koczy, A.: Building Energy information: demand and consumption prediction with Machine Learning models for sustainable and smart cities (2019)

    Google Scholar 

  83. Ardabili, S., Mosavi, A., Dehghani, M., Varkonyi-Koczy, A.: Deep learning and machine learning in hydrological processes climate change and earth systems a systematic review (2019)

    Google Scholar 

  84. Mohammadzadeh D., Karballaeezadeh, N., Mohemmi, M., Mosavi, A., Várkonyi-Kóczy A.: Urban train soil-structure interaction modeling and analysis (2019)

    Google Scholar 

  85. Mosavi, A., Ardabili, S., Varkonyi-Koczy, A.: List of deep learning models (2019)

    Google Scholar 

  86. Nosratabadi, S., Mosavi, A., Keivani, R., Ardabili, S., Aram, F., State of the art survey of deep learning and machine learning models for smart cities and urban sustainability (2019)

    Google Scholar 

Download references

Acknowledgments

This publication has been supported by the Project: “Support of research and development activities of the J. Selye University in the field of Digital Slovakia and creative industry” of the Research & Innovation Operational Programme (ITMS code: NFP313010T504) co-funded by the European Regional Development Fund.

Author information

Authors and Affiliations

  1. Institute of Advanced Studies Koszeg, Koszeg, Hungary

    Sina Ardabili

  2. Kalman Kando Faculty of Electrical Engineering, Obuda University, Budapest, Hungary

    Amir Mosavi & Annamária R. Várkonyi-Kóczy

  3. School of the Built Environment, Oxford Brookes University, Oxford, OX3 0BP, UK

    Amir Mosavi

  4. Department of Mathematics and Informatics, J. Selye University, Komarno, Slovakia

    Annamária R. Várkonyi-Kóczy

Authors
  1. Sina Ardabili
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amir Mosavi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Annamária R. Várkonyi-Kóczy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amir Mosavi .

Editor information

Editors and Affiliations

  1. Obuda University, Budapest, Hungary

    Annamária R. Várkonyi-Kóczy

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ardabili, S., Mosavi, A., Várkonyi-Kóczy, A.R. (2020). Advances in Machine Learning Modeling Reviewing Hybrid and Ensemble Methods. In: Várkonyi-Kóczy, A. (eds) Engineering for Sustainable Future. INTER-ACADEMIA 2019. Lecture Notes in Networks and Systems, vol 101. Springer, Cham. https://doi.org/10.1007/978-3-030-36841-8_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-36841-8_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-36840-1

  • Online ISBN: 978-3-030-36841-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation