Advertisement

List of Deep Learning Models

  • Amir MosaviEmail author
  • Sina Ardabili
  • Annamária R. Várkonyi-Kóczy
Conference paper
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 101)

Abstract

Deep learning (DL) algorithms have recently emerged from machine learning and soft computing techniques. Since then, several deep learning (DL) algorithms have been recently introduced to scientific communities and are applied in various application domains. Today the usage of DL has become essential due to their intelligence, efficient learning, accuracy and robustness in model building. However, in the scientific literature, a comprehensive list of DL algorithms has not been introduced yet. This paper provides a list of the most popular DL algorithms, along with their applications domains.

Keywords

Deep learning Machine learning Convolutional neural networks (CNN) Recurrent neural networks (RNN) Denoising autoencoder (DAE) Deep belief networks (DBNs) Long short-term memory (LSTM) 

Notes

Acknowledgements

This publication has been supported by the Project: “Support of research and development activities of the J. Selye University in the field of Digital Slovakia and creative industry” of the Research & Innovation Operational Programme (ITMS code: NFP313010T504) co-funded by the European Regional Development Fund.

References

  1. 1.
    Diamant, A., et al.: Deep learning in head & neck cancer outcome prediction. Scientific Reports 9(1) (2019)Google Scholar
  2. 2.
    Dong, Y., et al.: Bandgap prediction by deep learning in configurationally hybridized graphene and boron nitride. npj Comput. Mater. 5(1) (2019)Google Scholar
  3. 3.
    Liu, Y.: Novel volatility forecasting using deep learning–long short term memory recurrent neural networks. Expert Syst. Appl. 132, 99–109 (2019)CrossRefGoogle Scholar
  4. 4.
    Ludwiczak, J., et al.: PiPred – a deep-learning method for prediction of π-helices in protein sequences. Scientific Reports 9(1) (2019)Google Scholar
  5. 5.
    Matin, R., Hansen, C., Mølgaard, P.: Predicting distresses using deep learning of text segments in annual reports. Expert Syst. Appl. 132, 199–208 (2019)CrossRefGoogle Scholar
  6. 6.
    Nguyen, D., et al.: A feasibility study for predicting optimal radiation therapy dose distributions of prostate cancer patients from patient anatomy using deep learning. Scientific Reports 9(1) (2019)Google Scholar
  7. 7.
    Shickel, B., et al.: DeepSOFA: a continuous acuity score for critically ill patients using clinically interpretable deep learning. Scientific Reports 9(1) (2019)Google Scholar
  8. 8.
    Wang, K., Qi, X., Liu, H.: A comparison of day-ahead photovoltaic power forecasting models based on deep learning neural network. Appl. Energy 251 (2019)CrossRefGoogle Scholar
  9. 9.
    Aram, F., et al.: Design and validation of a computational program for analysing mental maps: Aram mental map analyzer. Sustainability (Switzerland) 11(14) (2019)CrossRefGoogle Scholar
  10. 10.
    Asadi, E., et al.: Groundwater quality assessment for drinking and agricultural purposes in Tabriz Aquifer, Iran (2019)Google Scholar
  11. 11.
    Asghar, M.Z., Subhan, F., Imran, M., Kundi, F.M., Shamshirband, S., Mosavi, A., Csiba, P., Várkonyi-Kóczy, A.R.: Performance evaluation of supervised machine learning techniques for efficient detection of emotions from online content. Preprints 2019, 2019080019  https://doi.org/10.20944/preprints201908.0019.v1
  12. 12.
    Bemani, A., Baghban, A., Shamshirband, S., Mosavi, A., Csiba, P., Várkonyi-Kóczy, A.R.: Applying ANN, ANFIS, and LSSVM Models for Estimation of Acid Solvent Solubility in Supercritical CO2. Preprints 2019, 2019060055  https://doi.org/10.20944/preprints201906.0055.v2
  13. 13.
    Choubin, B., et al.: Snow avalanche hazard prediction using machine learning methods. J. Hydrol. 577 (2019)CrossRefGoogle Scholar
  14. 14.
    Choubin, B., et al.: An ensemble prediction of flood susceptibility using multivariate discriminant analysis, classification and regression trees, and support vector machines. Sci. Total Environ. 651, 2087–2096 (2019)CrossRefGoogle Scholar
  15. 15.
    Dehghani, M., et al.: Prediction of hydropower generation using Grey wolf optimization adaptive neuro-fuzzy inference system. Energies 12(2), 289 (2019)CrossRefGoogle Scholar
  16. 16.
    Dineva, A., et al.: Review of soft computing models in design and control of rotating electrical machines. Energies 12(6) (2019)CrossRefGoogle Scholar
  17. 17.
    Dineva, A., et al.: Multi-label classification for fault diagnosis of rotating electrical machines (2019)Google Scholar
  18. 18.
    Farzaneh-Gord, M., et al.: Numerical simulation of pressure pulsation effects of a snubber in a CNG station for increasing measurement accuracy. Eng. Appl. Comput. Fluid Mech. 13(1), 642–663 (2019)Google Scholar
  19. 19.
    Ghalandari, M., et al.: Investigation of submerged structures’ flexibility on sloshing frequency using a boundary element method and finite element analysis. Eng. Appl. Comput.Fluid Mech. 13(1), 519–528 (2019)Google Scholar
  20. 20.
    Ghalandari, M., et al.: Flutter speed estimation using presented differential quadrature method formulation. Eng. Appl. Comput. Fluid Mech. 13(1), 804–810 (2019)Google Scholar
  21. 21.
    Karballaeezadeh, N., et al.: Prediction of remaining service life of pavement using an optimized support vector machine (case study of Semnan-Firuzkuh road). Eng. Appl. Comput. Fluid Mech. 13(1), 188–198 (2019)Google Scholar
  22. 22.
    Menad, N.A., et al.: Modeling temperature dependency of oil—water relative permeability in thermal enhanced oil recovery processes using group method of data handling and gene expression programming. Eng. Appl. Comput. Fluid Mech. 13(1), 724–743 (2019)MathSciNetGoogle Scholar
  23. 23.
    Mohammadzadeh, S., et al.: Prediction of compression index of fine-grained soils using a gene expression programming model. Infrastructures 4(2), 26 (2019)CrossRefGoogle Scholar
  24. 24.
    Mosavi, A., Edalatifar, M.: A Hybrid Neuro-Fuzzy Algorithm for Prediction of Reference Evapotranspiration, in Lecture Notes in Networks and Systems, pp. 235–243. Springer (2019)Google Scholar
  25. 25.
    Mosavi, A., Lopez, A., Várkonyi-Kóczy, A.R.: Industrial applications of big data: state of the art survey, D. Luca, L. Sirghi, and C. Costin, Editors, pp. 225–232. Springer (2018)Google Scholar
  26. 26.
    Mosavi, A., Ozturk, P., Chau, K.W.: Flood prediction using machine learning models: literature review. Water (Switzerland) 10(11) (2018)Google Scholar
  27. 27.
    Mosavi, A., Rabczuk, T.: Learning and intelligent optimization for material design innovation, D.E. Kvasov, et al., Editors, pp. 358–363. Springer (2017)Google Scholar
  28. 28.
    Mosavi, A., Rabczuk, T., Várkonyi-Kóczy, A.R.: Reviewing the novel machine learning tools for materials design, D. Luca, L. Sirghi, and C. Costin, Editors, pp. 50–58. Springer (2018)Google Scholar
  29. 29.
    Mosavi, A., et al.: State of the art of machine learning models in energy systems, a systematic review. Energies 12(7) (2019)CrossRefGoogle Scholar
  30. 30.
    Mosavi, A., et al.: Prediction of multi-inputs bubble column reactor using a novel hybrid model of computational fluid dynamics and machine learning. Eng. Appl. Comput. Fluid Mech. 13(1), 482–492 (2019)Google Scholar
  31. 31.
    Mosavi, A., Várkonyi-Kóczy, A.R.: Integration of machine learning and optimization for robot learning, R. Jablonski and R. Szewczyk, Editors, pp. 349–355. Springer (2017)Google Scholar
  32. 32.
    Nosratabadi, S., et al.: Sustainable business models: a review. Sustainability (Switzerland) 11(6) (2019)CrossRefGoogle Scholar
  33. 33.
    Qasem, S.N., et al.: Estimating daily dew point temperature using machine learning algorithms. Water (Switzerland) 11(3) (2019)CrossRefGoogle Scholar
  34. 34.
    Rezakazemi, M., Mosavi, A., Shirazian, S.: ANFIS pattern for molecular membranes separation optimization. J. Mol. Liq. 274, 470–476 (2019)CrossRefGoogle Scholar
  35. 35.
    Riahi-Madvar, H., et al.: Comp. Anal. Soft Comput. Techn. RBF, MLP, ANFIS with MLR and MNLR Predict. Grade-control Scour Hole Geometry. Eng. Appl. Comput. Fluid Mech. 13(1), 529–550 (2019)Google Scholar
  36. 36.
    Shabani, S., Samadianfard, S., Taghi Sattari, M., Shamshirband, S., Mosavi, A., Kmet, T., Várkonyi-Kóczy, A.R.: Modeling daily pan evaporation in humid climates using gaussian process regression. Preprints 2019, 2019070351  https://doi.org/10.20944/preprints201907.0351.v1
  37. 37.
    Shamshirband, S., Hadipoor, M., Baghban, A., Mosavi, A., Bukor J., Várkonyi-Kóczy, A.R.: Developing an ANFIS-PSO model to predict mercury emissions in combustion flue gases. Preprints 2019, 2019070165  https://doi.org/10.20944/preprints201907.0165.v1
  38. 38.
    Shamshirband, S., et al.: Ensemble models with uncertainty analysis for multi-day ahead forecasting of chlorophyll a concentration in coastal waters. Eng. Appl. Comput. Fluid Mech. 13(1), 91–101 (2019)Google Scholar
  39. 39.
    Shamshirband, S., Mosavi, A., Rabczuk, T.: Particle swarm optimization model to predict scour depth around bridge pier. arXiv preprint arXiv:1906.08863 (2019)
  40. 40.
    Taherei Ghazvinei, P., et al.: Sugarcane growth prediction based on meteorological parameters using extreme learning machine and artificial neural network. Eng. Appl. Comput. Fluid Mech. 12(1), 738–749 (2018)Google Scholar
  41. 41.
    Torabi, M., et al.: A Hybrid clustering and classification technique for forecasting short-term energy consumption. Environ. Progr. Sustain. Energy 38(1), 66–76 (2019)CrossRefGoogle Scholar
  42. 42.
    Torabi, M., et al.: A Hybrid Machine Learning Approach for Daily Prediction of Solar Radiation, in Lecture Notes in Networks and Systems, pp. 266–274. Springer (2019)Google Scholar
  43. 43.
    Biswas, M., et al.: State-of-the-art review on deep learning in medical imaging. Front. Biosci. Landmark 24(3), 392–426 (2019)CrossRefGoogle Scholar
  44. 44.
    Bote-Curiel, L., et al.: Deep learning and big data in healthcare: a double review for critical beginners. Appl. Sci. (Switzerland) 9(11) (2019)CrossRefGoogle Scholar
  45. 45.
    Feng, Y., Teh, H.S., Cai, Y.: Deep learning for chest radiology: a review. Curr. Radiol. Reports 7(8) (2019)Google Scholar
  46. 46.
    Griffiths, D., Boehm, J.: A Review on deep learning techniques for 3D sensed data classification. Remote Sens. 11(12) (2019)CrossRefGoogle Scholar
  47. 47.
    Gupta, A., et al.: Deep learning in image cytometry: a review. Cytom. Part A 95(4), 366–380 (2019)CrossRefGoogle Scholar
  48. 48.
    Ha, V.K., et al.: Deep learning based single image super-resolution: a survey. Int. J. Autom. Comput. 16(4), 413–426 (2019)CrossRefGoogle Scholar
  49. 49.
    Jiang, W., Zhang, C.S., Yin, X.C.: Deep learning based scene text detection: a survey. Tien Tzu Hsueh Pao/Acta Electronica Sinica 47(5), 1152–1161 (2019)Google Scholar
  50. 50.
    Johnsirani Venkatesan, N., Nam, C., Shin, D.R.: Deep learning frameworks on apache spark: a review. IETE Techn. Rev. (Inst. Electron. Telecommun. Eng., India) 36(2), 164–177 (2019)CrossRefGoogle Scholar
  51. 51.
    Li, X., He, Y., Jing, X.: A survey of deep learning-based human activity recognition in radar. Remote Sens. 11(9) (2019)CrossRefGoogle Scholar
  52. 52.
    Litjens, G., et al.: State-of-the-art deep learning in cardiovascular image analysis. JACC: Cardiovasc. Imaging 12(8P1), 1549–1565 (2019)Google Scholar
  53. 53.
    Liu, S., et al.: Deep learning in medical ultrasound analysis: a review. Engineering 5(2), 261–275 (2019)MathSciNetCrossRefGoogle Scholar
  54. 54.
    Mazurowski, M.A., et al.: Deep learning in radiology: an overview of the concepts and a survey of the state of the art with focus on MRI. J. Magn. Reson. Imaging 49(4), 939–954 (2019)CrossRefGoogle Scholar
  55. 55.
    Narendra, G., Sivakumar, D.: Deep learning based hyperspectral image analysis-a survey. J. Comput. Theor. Nanosci. 16(4), 1528–1535 (2019)CrossRefGoogle Scholar
  56. 56.
    Wang, H., et al.: A review of deep learning for renewable energy forecasting. Energy Convers. Manag. 198 (2019)CrossRefGoogle Scholar
  57. 57.
    Wang, Y., et al.: Enhancing transportation systems via deep learning: a survey. Transp. Res. Part C: Emerg. Technol. 99, 144–163 (2019)CrossRefGoogle Scholar
  58. 58.
    Zhang, W., et al.: Deep learning-based multimedia analytics: a review. ACM Trans. Multimed. Comput. Commun. Appl. 15(1s) (2019)Google Scholar
  59. 59.
    Zhou, J., et al.: Deep learning for aspect-level sentiment classification: survey, vision, and challenges. IEEE Access 7, 78454–78483 (2019)CrossRefGoogle Scholar
  60. 60.
    Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: European Conference on Computer Vision. Springer (2014)Google Scholar
  61. 61.
    Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  62. 62.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  63. 63.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (2012)Google Scholar
  64. 64.
    He, K., et al.: Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  65. 65.
    Kong, Z., et al.: Condition monitoring of wind turbines based on spatio-temporal fusion of SCADA data by convolutional neural networks and gated recurrent units. Renew. Energy 146, 760–768 (2020)CrossRefGoogle Scholar
  66. 66.
    Lossau, T., et al.: Motion estimation and correction in cardiac CT angiography images using convolutional neural networks. Computerized Med. Imaging Graphics 76 (2019)Google Scholar
  67. 67.
    Bhatnagar, S., et al.: Prediction of aerodynamic flow fields using convolutional neural networks. Comput. Mech. 64(2), 525–545 (2019)MathSciNetzbMATHCrossRefGoogle Scholar
  68. 68.
    Nevavuori, P., Narra, N., Lipping, T.: Crop yield prediction with deep convolutional neural networks. Comput. Electron. Agric. 163 (2019)CrossRefGoogle Scholar
  69. 69.
    Ajami, A., et al.: Identifying a slums’ degree of deprivation from VHR images using convolutional neural networks. Remote Sens. 11(11) (2019)CrossRefGoogle Scholar
  70. 70.
    Min, S., Lee, B., Yoon, S.: Deep learning in bioinformatics. Brief. Bioinform. 18(5), 851–869 (2017)Google Scholar
  71. 71.
    Zhu, S., et al.: Gaussian mixture model coupled recurrent neural networks for wind speed interval forecast. Energy Convers. Manag. 198 (2019)CrossRefGoogle Scholar
  72. 72.
    Pan, B., Xu, X., Shi, Z.: Tropical cyclone intensity prediction based on recurrent neural networks. Electron. Lett. 55(7), 413–415 (2019)CrossRefGoogle Scholar
  73. 73.
    Bisharad, D., Laskar, R.H.: Music genre recognition using convolutional recurrent neural network architecture. Expert Syst. (2019)Google Scholar
  74. 74.
    Zhong, C., et al.: Inland ship trajectory restoration by recurrent neural network. J. Navig. (2019)Google Scholar
  75. 75.
    Jarrah, M., Salim, N.: A recurrent neural network and a discrete wavelet transform to predict the Saudi stock price trends. Int. J. Adv. Comput. Sci. Appl. 10(4), 155–162 (2019)Google Scholar
  76. 76.
    Al Rahhal, M.M., et al.: Deep learning approach for active classification of electrocardiogram signals. Inf. Sci. 345, 340–354 (2016)CrossRefGoogle Scholar
  77. 77.
    Yin, Z., Zhang, J.: Cross-session classification of mental workload levels using EEG and an adaptive deep learning model. Biomed. Signal Process. Control 33, 30–47 (2017)CrossRefGoogle Scholar
  78. 78.
    Sun, W., Zheng, B., Qian, W.: Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis. Comput. Biol. Med. 89, 530–539 (2017)CrossRefGoogle Scholar
  79. 79.
    Chen, Y., et al.: Indoor location method of interference source based on deep learning of spectrum fingerprint features in Smart Cyber-Physical systems. Eurasip J. Wirel. Commun. Netw. 2019(1) (2019)Google Scholar
  80. 80.
    Liu, P., Zheng, P., Chen, Z.: Deep learning with stacked denoising auto-encoder for short-term electric load forecasting. Energies 12(12) (2019)CrossRefGoogle Scholar
  81. 81.
    Nicolai, A., Hollinger, G.A.: Denoising autoencoders for laser-based scan registration. IEEE Robot. Autom. Lett. 3(4), 4391–4398 (2018)CrossRefGoogle Scholar
  82. 82.
    Yue, L., et al.: Multiple auxiliary information based deep model for collaborative filtering. J. Comput. Sci. Technol. 33(4), 668–681 (2018)CrossRefGoogle Scholar
  83. 83.
    Roy, S.S., Ahmed, M., Akhand, M.A.H.: Noisy image classification using hybrid deep learning methods. J. Inf.Commun. Technol. 17(2), 233–269 (2018)Google Scholar
  84. 84.
    Tan, Z., et al.: Denoised senone i-vectors for robust speaker verification. IEEE/ACM Trans. Audio Speech Lang. Process. 26(4), 820–830 (2018)CrossRefGoogle Scholar
  85. 85.
    Zhang, Q., et al.: Deep learning based classification of breast tumors with shear-wave elastography. Ultrasonics 72, 150–157 (2016)CrossRefGoogle Scholar
  86. 86.
    Wulsin, D., et al.: Modeling electroencephalography waveforms with semi-supervised deep belief nets: fast classification and anomaly measurement. J. Neural Eng. 8(3), 036015 (2011)CrossRefGoogle Scholar
  87. 87.
    Patterson, J., Gibson, A.: Deep Learning: A Practitioner’s Approach. “ O’Reilly Media, Inc.” (2017)Google Scholar
  88. 88.
    Vieira, S., Pinaya, W.H., Mechelli, A.: Using deep learning to investigate the neuroimaging correlates of psychiatric and neurological disorders: methods and applications. Neurosci. Biobehav. Rev. 74, 58–75 (2017)CrossRefGoogle Scholar
  89. 89.
    Hassan, M.M., et al.: Human emotion recognition using deep belief network architecture. Inf. Fusion 51, 10–18 (2019)CrossRefGoogle Scholar
  90. 90.
    Cheng, Y., et al.: Deep belief network for meteorological time series prediction in the internet of things. IEEE Int. Things J. 6(3), 4369–4376 (2019)CrossRefGoogle Scholar
  91. 91.
    Yu, Y., et al.: Forecasting a short-term wind speed using a deep belief network combined with a local predictor. IEEJ Trans. Electr. Electron. Eng. 14(2), 238–244 (2019)CrossRefGoogle Scholar
  92. 92.
    Zheng, J., Fu, X., Zhang, G.: Research on exchange rate forecasting based on deep belief network. Neural Comput. Appl. 31, 573–582 (2019)CrossRefGoogle Scholar
  93. 93.
    Ahmad, M., et al.: Deep belief network modeling for automatic liver segmentation. IEEE Access 7, 20585–20595 (2019)CrossRefGoogle Scholar
  94. 94.
    Ronoud, S., Asadi, S.: An evolutionary deep belief network extreme learning-based for breast cancer diagnosis. Soft Comput. (2019)Google Scholar
  95. 95.
    Ghimire, S., et al.: Deep solar radiation forecasting with convolutional neural network and long short-term memory network algorithms. Appl. Energy (2019)Google Scholar
  96. 96.
    Hong, J., Wang, Z., Yao, Y.: Fault prognosis of battery system based on accurate voltage abnormity prognosis using long short-term memory neural networks. Appl. Energy (2019)Google Scholar
  97. 97.
    Krishan, M., et al.: Air quality modelling using long short-term memory (LSTM) over NCT-Delhi, India. Air Qual. Atmos. Health 12(8), 899–908 (2019)CrossRefGoogle Scholar
  98. 98.
    Zhang, R., et al.: Deep long short-term memory networks for nonlinear structural seismic response prediction. Comput. Struct. 220, 55–68 (2019)CrossRefGoogle Scholar
  99. 99.
    Hua, Y., et al.: Deep learning with long short-term memory for time series prediction. IEEE Commun. Mag. 57(6), 114–119 (2019)CrossRefGoogle Scholar
  100. 100.
    Zhang, J., et al.: Short-term forecasting and uncertainty analysis of wind turbine power based on long short-term memory network and Gaussian mixture model. Appl. Energy, 229–244 (2019)CrossRefGoogle Scholar
  101. 101.
    Vardaan, K., et al.: Earthquake trend prediction using long short-term memory RNN. Int. J. Electr. Comput. Eng. 9(2), 1304–1312 (2019)Google Scholar
  102. 102.
    Mesri Gundoshmian, T., Ardabili, S., Mosavi, A., Varkonyi-Koczy, A.: Prediction of combine harvester performance using hybrid machine learning modeling and re-sponse surface methodology, Preprints 2019Google Scholar
  103. 103.
    Ardabili, S., Mosavi, A., Varkonyi-Koczy, A.: Systematic review of deep learning and machine learning models in biofuels research, Preprints 2019Google Scholar
  104. 104.
    Ardabili, S., Mosavi, A., Varkonyi-Koczy, A.: Advances in machine learning modeling reviewing hybrid and ensemble methods, Preprints 2019Google Scholar
  105. 105.
    Ardabili, S., Mosavi, A., Varkonyi-Koczy, A.: Building Energy information: demand and consumption prediction with Machine Learning models for sustainable and smart cities, Preprints 2019Google Scholar
  106. 106.
    Ardabili, S., Mosavi, A., Dehghani, M., Varkonyi-Koczy, A., Deep learning and machine learning in hydrological processes climate change and earth systems a systematic review, Preprints 2019Google Scholar
  107. 107.
    Mohammadzadeh, D., Karballaeezadeh, N., Mohemmi, M., Mosavi, A., Várkonyi-Kóczy A.: Urban train soil-structure interaction modeling and analysis, Preprints 2019Google Scholar
  108. 108.
    Mosavi, A., Ardabili, S., Varkonyi-Koczy, A.: List of deep learning models, Preprints 2019Google Scholar
  109. 109.
    Nosratabadi, S., Mosavi, A., Keivani, R., Ardabili, S., Aram, F.: State of the art survey of deep learning and machine learning models for smart cities and urban sustainability, Preprints 2019Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Institue of Automation, Kalman Kando Faculty of Electrical EngineeringObuda UniversityBudapestHungary
  2. 2.School of the Built EnvironmentOxford Brookes UniversityOxfordUK
  3. 3.Institute of Advanced Studies KoszegKoszegHungary
  4. 4.Department of Mathematics and InformaticsJ. Selye UniversityKomarnoSlovakia

Personalised recommendations