Skip to main content
SpringerLink
Log in

A comprehensive review of deep neuro-fuzzy system architectures and their optimization methods

Neural Computing and Applications Aims and scope Submit manuscript

Cite this article

Abstract

Deep neuro-fuzzy systems (DNFSs) have been successfully applied to real-world problems using the efficient learning process of deep neural networks (DNNs) and reasoning aptitude from fuzzy inference systems (FIS). This study provides a comprehensive review of DNFS dividing it into two essential parts. The first part aims to provide a thorough understanding of DNFS and its architectural representation, whereas the second part reviews DNFS optimization methods. This study aims to assist researchers in understanding the various ways DNFS models are developed by hybridizing DNN and FIS, as well as gradient (derivative)-based methods and metaheuristics (derivative-free) optimization, as discussed in the literature. This study revealed that the proposed DNFS architectures performed 11.6% better than non-fuzzy models, with an overall accuracy of 81.4%. The investigation based on optimization methods revealed that DNFS with metaheuristics optimization methods has shown an overall accuracy of 93.56%, which is 21.10% higher than the DNFS models using gradient-based methods. Additionally, this study showed that DNFS networks presented in the literature have integrated DNN with typical FIS, although more satisfactory results can be obtained using a new generation of FIS termed fractional FIS (FFIS) and Mamdani complex FIS (M-CFIS). Besides, dynamic neural networks are suggested in the replacement of static DNNs to facilitate dynamic learning. Some studies have also demonstrated the optimization of DNFS using classical gradient-based approaches that can affect network performance when solving highly nonlinear problems. This study suggests implementing optimization methods with new and improvised metaheuristics to improve the training and performance of the models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

References

  1. Sharmadha S, Shivani K, Shruthi K, Bharathi B, & Kavitha S (2020) Automatic speech recognition using deep neural network. Advances in Intelligent Systems and Computing, vol 1118. Springer, Singapore. https://doi.org/10.1007/978-981-15-2475-2_33

  2. Widiastuti NI (2019) Convolution neural network for text mining and natural language processing. IOP Conf Series Mater Sci Eng 662:052010. https://doi.org/10.1088/1757-899x/662/5/052010

    Article  Google Scholar 

  3. Guo J, Fan Y, Pang L, Yang L, Ai Q, Zamani H, Cheng X (2020) A Deep Look into neural ranking models for information retrieval. Inf Process Manage 57(6):102067. https://doi.org/10.1016/j.ipm.2019.102067

    Article  Google Scholar 

  4. Nishani E, Çiço B (2017) Computer vision approaches based on deep learning and neural networks: deep neural networks for video analysis of human pose estimation. 2017 6th Mediterranean Conference on Embedded Computing (MECO), pp. 1–4. https://doi.org/10.1109/MECO.2017.7977207

  5. Wainberg M, Merico D, Delong A, Frey BJ (2018) Deep learning in biomedicine. Nat Biotechnol 36(9):829–838. https://doi.org/10.1038/nbt.4233

    Article  Google Scholar 

  6. Romascanu A, Ker H, Sieber R, Greenidge S, Lumley S, Bush D, Morgan S, Zhao R, Brunila M (2020) Using deep learning and social network analysis to understand and manage extreme flooding. J Conting Crisis Manag 28(3):251–261. https://doi.org/10.1111/1468-5973.12311

    Article  Google Scholar 

  7. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444. https://doi.org/10.1038/nature14539

    Article  Google Scholar 

  8. Schaedler M, Blümm C, Kuschnerov M, Pittalà F, Calabrò S, Pachnicke S (2019) Deep neural network equalization for optical short reach communication. Appl Sci 9:4675. https://doi.org/10.3390/app9214675

    Article  Google Scholar 

  9. Altwaijry N, Al-Turaiki I (2021) Arabic handwriting recognition system using convolutional neural network. Neural Comput Appl 33(7):2249–2261. https://doi.org/10.1007/s00521-020-05070-8

    Article  Google Scholar 

  10. Bullinaria JA (2013) Recurrent neural networks. Neural Comput: Lecture, 12

  11. Abdullah MHA, Othman M, Kasim S, Mohamed SA (2019) Evolving spiking neural networks methods for classification problem: a case study in flood events risk assessment. Indonesian J Electr Eng Computer Sci 16:222–229. https://doi.org/10.11591/ijeecs.v16.i1.pp222-229

    Article  Google Scholar 

  12. Said J, Jadid Abdulkadir S, Alhussian H, Nazmi M, Elsheikh A (2018) Long short term memory recurrent network for standard and poor’s 500 index modelling. Int J Eng Technol 7:25–29. https://doi.org/10.14419/ijet.v7i4.15.21365

    Article  Google Scholar 

  13. Qiu Y, Dai Y (2019) A stacked auto-encoder based fault diagnosis model for chemical process. In Computer Aided Chemical Engineering (Vol. 46, pp. 1303–1308): Elsevier. https://doi.org/10.1016/B978-0-12-818634-3.50218-6

  14. Hua Y, Guo J, Zhao H (2015) Deep belief networks and deep learning. Proceedings of 2015 International Conference on Intelligent Computing and Internet of Things, 2015, pp. 1–4, https://doi.org/10.1109/ICAIOT.2015.7111524.

  15. Zhang N, Ding S, Zhang J, Xue Y (2018) An overview on restricted Boltzmann machines. Neurocomputing 275:1186–1199. https://doi.org/10.1016/j.neucom.2017.09.065

    Article  Google Scholar 

  16. Bonanno D, Nock K, Smith L, Elmore P, Petry F (2017) An approach to explainable deep learning using fuzzy inference (Vol. 10207): SPIE. https://doi.org/10.1117/12.2268001

  17. Hayashi Y (2020) Black Box Nature of Deep Learning for Digital Pathology: Beyond Quantitative to Qualitative Algorithmic Performances. In Artificial Intelligence and Machine Learning for Digital Pathology (pp. 95–101): Springer. https://doi.org/10.1007/978-3-030-50402-1_6

  18. Buhrmester V, Münch D, Arens M (2019) Analysis of explainers of black box deep neural networks for computer vision: A survey. arXiv preprint arXiv: 1911.12116

  19. Aviles AI, Alsaleh SM, Montseny E, Sobrevilla P, Casals A (2016) A Deep-Neuro-Fuzzy approach for estimating the interaction forces in Robotic surgery. 2016 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 1113–1119. https://doi.org/10.1109/FUZZ-IEEE.2016.7737812

  20. Zheng Y, Sheng W, Sun X, Chen S (2017) Airline passenger profiling based on fuzzy deep machine learning. IEEE Trans Neural Netw Learn Syst 28(12):2911–2923. https://doi.org/10.1109/TNNLS.2016.2609437

    Article  MathSciNet  Google Scholar 

  21. El Hatri C, Boumhidi J (2018) Fuzzy deep learning based urban traffic incident detection. Cogn Syst Res 50:206–213. https://doi.org/10.1016/j.cogsys.2017.12.002

    Article  Google Scholar 

  22. Ramasamy B, Hameed AZ (2019) Classification of healthcare data using hybridised fuzzy and convolutional neural network. Healthcare Technol Letters 6(3):59–63. https://doi.org/10.1049/htl.2018.5046

    Article  Google Scholar 

  23. Aye YY, Thiha K, Pyu MMM, Watanabe K (2019) A deep neural network based human following robot with fuzzy control. IEEE Int Confn Robotics Biomimetics (ROBIO) 2019:720–725. https://doi.org/10.1109/ROBIO49542.2019.8961577

    Article  Google Scholar 

  24. Chopade HA, Narvekar M (2017) Hybrid auto text summarization using deep neural network and fuzzy logic system. Int Conf Inventive Comput Inf (ICICI) 2017:52–56. https://doi.org/10.1109/ICICI.2017.8365192

    Article  Google Scholar 

  25. Zhang L, Zhu Y, Shi X, Li X (2020) A Situation Assessment Method with an Improved Fuzzy Deep Neural Network for Multiple UAVs. Information. https://doi.org/10.3390/info11040194

    Article  Google Scholar 

  26. Liao P, Xu M, Yang C (2020) A fuzzy ensemble method with deep learning for multi-robot system. IEEE Access 8:220352–220363. https://doi.org/10.1109/ACCESS.2020.3042439

    Article  Google Scholar 

  27. Yin P, Dou G, Lin X, Liu L (2020) A hybrid method for forecasting new product sales based on fuzzy clustering and deep learning. Kybernetes 49(12):3099–3118. https://doi.org/10.1108/K-10-2019-0688

    Article  Google Scholar 

  28. Asghar MZ, Subhan F, Ahmad H, Khan WZ, Hakak S, Gadekallu TR, Alazab M (2021) Senti-eSystem: a sentiment-based eSystem-using hybridized fuzzy and deep neural network for measuring customer satisfaction. Softw Practice Exp 51(3):571–594. https://doi.org/10.1002/spe.2853

    Article  Google Scholar 

  29. Bedi P, & Khurana P (2020) Sentiment Analysis Using Fuzzy-Deep Learning. Proceedings of ICETIT 2019, Cham. https://doi.org/10.1007/978-3-030-30577-2_21

  30. Shalaginov A, Franke K (2017) A deep neuro-fuzzy method for multi-label malware classification and fuzzy rules extraction. IEEE Symposium Series Comput Intell (SSCI) 2017:1–8. https://doi.org/10.1109/SSCI.2017.8280788

    Article  Google Scholar 

  31. Chen D, Zhang X, Wang LL, Han Z (2019) Prediction of cloud resources demand based on hierarchical pythagorean fuzzy deep neural network. IEEE Trans Serv Comput. https://doi.org/10.1109/TSC.2019.2906901

    Article  Google Scholar 

  32. Monsefi AK, Zakeri B, Samsam S, & Khashehchi M (2019) Performing software test oracle based on deep neural network with fuzzy inference system. International Congress on High-Performance Computing and Big Data Analysis, 406–417. Springer, Cham, 2019. https://doi.org/10.1007/978-3-030-33495-6_31

  33. Nguyen T-L, Kavuri S, Lee M (2019) A multimodal convolutional neuro-fuzzy network for emotion understanding of movie clips. Neural Netw 118:208–219. https://doi.org/10.1016/j.neunet.2019.06.010

    Article  Google Scholar 

  34. Greeshma MS, Bindu VR (2017) Single image super resolution using fuzzy deep convolutional networks. Int Conf Technol Adv Power Energy (TAP Energy) 2017:1–6. https://doi.org/10.1109/TAPENERGY.2017.8397224

    Article  Google Scholar 

  35. Guan C, Wang S, Liew AW (2020) Lip image segmentation based on a fuzzy convolutional neural network. IEEE Trans Fuzzy Syst 28(7):1242–1251. https://doi.org/10.1109/TFUZZ.2019.2957708

    Article  Google Scholar 

  36. Nguyen T-L, Kavuri S, Lee M (2018) A fuzzy convolutional neural network for text sentiment analysis. J Intell Fuzzy Syst 35(6):6025–6034. https://doi.org/10.3233/JIFS-169843

    Article  Google Scholar 

  37. Deng Y, Ren Z, Kong Y, Bao F, Dai Q (2017) A hierarchical fused fuzzy deep neural network for data classification. IEEE Trans Fuzzy Syst 25(4):1006–1012. https://doi.org/10.1109/TFUZZ.2016.2574915

    Article  Google Scholar 

  38. Lima S, Terán L, Portmann E (2020) A proposal for an explainable fuzzy-based deep learning system for skin cancer prediction. Seventh Int Conf eDemocracy eGovernment (ICEDEG) 2020:29–35. https://doi.org/10.1109/ICEDEG48599.2020.9096799

    Article  Google Scholar 

  39. Yang CH, Moi SH, Hou MF, Chuang LY, Lin YD (2020) Applications of deep learning and fuzzy systems to detect cancer mortality in next-generation genomic data. IEEE Trans Fuzzy Syst. https://doi.org/10.1109/TFUZZ.2020.3028909

    Article  Google Scholar 

  40. Shen T, Wang J, Gou C, Wang FY (2020) Hierarchical fused model with deep learning and type-2 fuzzy learning for breast cancer diagnosis. IEEE Trans Fuzzy Syst 28(12):3204–3218. https://doi.org/10.1109/TFUZZ.2020.3013681

    Article  Google Scholar 

  41. Zhang R, Shen F, Zhao J (2014) A model with fuzzy granulation and deep belief networks for exchange rate forecasting. Int Joint Conf Neural Netw (IJCNN) 2014:366–373. https://doi.org/10.1109/IJCNN.2014.6889448

    Article  Google Scholar 

  42. Chen W, An J, Li R, Fu L, Xie G, Bhuiyan MZA, Li K (2018) A novel fuzzy deep-learning approach to traffic flow prediction with uncertain spatial–temporal data features. Futur Gener Comput Syst 89:78–88. https://doi.org/10.1016/j.future.2018.06.021

    Article  Google Scholar 

  43. Van ND, & Kim G (2018) Fuzzy Logic and Deep Steering Control based Recommendation System for Self-Driving Car. 2018 18th International Conference on Control, Automation and Systems (ICCAS), 1107–1110

  44. Zheng Y-J, Sheng W-G, Sun X-M, Chen S-Y (2016) Airline passenger profiling based on fuzzy deep machine learning. IEEE Trans Neural Netw Learn Syst 28(12):2911–2923. https://doi.org/10.1109/TNNLS.2016.2609437

    Article  MathSciNet  Google Scholar 

  45. Wang L-X (2019) Fast training algorithms for deep convolutional fuzzy systems with application to stock index prediction. IEEE Trans Fuzzy Syst 28(7):1301–1314. https://doi.org/10.1109/TFUZZ.2019.2930488

    Article  Google Scholar 

  46. Chen X, Rajan D, Quek C (2020) A deep hybrid fuzzy neural Hammerstein-Wiener network for stock price prediction. Int Conf Artificial Intell Inf Commun (ICAIIC) 2020:288–293. https://doi.org/10.1109/ICAIIC48513.2020.9065269

    Article  Google Scholar 

  47. Chandrasekar R (2020) Fuzzy crow search algorithm-based deep LSTM for bitcoin prediction. Int J Distributed Syst Technol (IJDST) 11(4):53–71. https://doi.org/10.4018/IJDST.2020100104

    Article  MathSciNet  Google Scholar 

  48. Xiao P (2020) Information management of E-Commerce platform based on neural networks and fuzzy deep learning models. Int Conf Smart Electron Commun (ICOSEC) 2020:532–535. https://doi.org/10.1109/ICOSEC49089.2020.9215235

    Article  Google Scholar 

  49. Elavarasan D, Vincent PMDR (2021) Fuzzy deep learning-based crop yield prediction model for sustainable agronomical frameworks. Neural Comput Appl. https://doi.org/10.1007/s00521-021-05950-7

    Article  Google Scholar 

  50. Fan T, Xu J (2020) Image classification of crop diseases and pests based on deep learning and fuzzy system. Int J Data Warehousing Mining (IJDWM) 16(2):34–47. https://doi.org/10.4018/IJDWM.2020040103

    Article  MathSciNet  Google Scholar 

  51. Zheng Y, Chen S, Xue Y, Xue J (2017) A pythagorean-type fuzzy deep denoising autoencoder for industrial accident early warning. IEEE Trans Fuzzy Syst 25(6):1561–1575. https://doi.org/10.1109/TFUZZ.2017.2738605

    Article  Google Scholar 

  52. Tian Z, Fong S (2016) Survey of meta-heuristic algorithms for deep learning training. Optim Algorithms-Methods Appl. https://doi.org/10.5772/63785

    Article  Google Scholar 

  53. Rere L, Fanany MI, Arymurthy AM (2016) Metaheuristic algorithms for convolution neural network. Comput Intell Neurosci. https://doi.org/10.1155/2016/1537325

    Article  Google Scholar 

  54. Akay B, Karaboga D, Akay R (2021) A comprehensive survey on optimizing deep learning models by metaheuristics. Artif Intell Rev. https://doi.org/10.1007/s10462-021-09992-0

    Article  MATH  Google Scholar 

  55. Altundogan TG, Karakose M (2019) Multiple object tracking with dynamic fuzzy cognitive maps using deep learning. Int Artif Intell Data Process Symposium (IDAP) 2019:1–5. https://doi.org/10.1109/IDAP.2019.8875871

    Article  Google Scholar 

  56. Velliangiri S, Pandey HM (2020) Fuzzy-taylor-elephant herd optimization inspired deep belief network for DDoS attack detection and comparison with state-of-the-arts algorithms. Futur Gener Comput Syst 110:80–90. https://doi.org/10.1016/j.future.2020.03.049

    Article  Google Scholar 

  57. Siva Raja PM, Rani AV (2020) Brain tumor classification using a hybrid deep autoencoder with Bayesian fuzzy clustering-based segmentation approach. Biocybernetics Biomed Eng 40(1):440–453. https://doi.org/10.1016/j.bbe.2020.01.006

    Article  Google Scholar 

  58. Ravi C (2020) Image classification using deep learning and fuzzy systems. Intelligent Systems Design and Applications, vol 941. Springer, Cham. https://doi.org/10.1007/978-3-030-16660-1_50

  59. Chimatapu R, Hagras H, Starkey A, & Owusu G (2018) Interval Type-2 Fuzzy Logic Based Stacked Autoencoder Deep Neural Network For Generating Explainable AI Models in Workforce Optimization. 2018 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 1–8. https://doi.org/10.1109/FUZZ-IEEE.2018.8491679

  60. Singh G, Pal M, Yadav Y, Singla T (2020) Deep neural network-based predictive modeling of road accidents. Neural Comput Appl. https://doi.org/10.1007/s00521-019-04695-8

    Article  Google Scholar 

  61. Najafabadi MM, Villanustre F, Khoshgoftaar TM, Seliya N, Wald R, Muharemagic E (2015) Deep learning applications and challenges in big data analytics. Journal of Big Data 2(1):1. https://doi.org/10.1186/s40537-014-0007-7

    Article  Google Scholar 

  62. Talpur N, Abdulkadir SJ, Hasan MH (2020) A deep learning based neuro-fuzzy approach for solving classification problems. Int Conf Comput Intell (ICCI) 2020:167–172. https://doi.org/10.1109/ICCI51257.2020.9247639

    Article  Google Scholar 

  63. Jadid Abdulkadir S, Shamsuddin SM, Sallehuddin R (2012) Moisture Prediction in maize using three term back propagation neural network. Int J Environ Sci Dev. https://doi.org/10.7763/IJESD.2012.V3.215

    Article  Google Scholar 

  64. Khamparia A, Singh KM (2019) A systematic review on deep learning architectures and applications. Expert Syst 36(3):e12400. https://doi.org/10.1111/exsy.12400

    Article  Google Scholar 

  65. Suto J, Oniga S (2019) Efficiency investigation from shallow to deep neural network techniques in human activity recognition. Cogn Syst Res 54:37–49. https://doi.org/10.1016/j.cogsys.2018.11.009

    Article  Google Scholar 

  66. Zhou X-H, Zhang M-X, Xu Z-G, Cai C-Y, Huang Y-J, Zheng Y-J (2019) Shallow and deep neural network training by water wave optimization. Swarm Evol Comput 50:100561. https://doi.org/10.1016/j.swevo.2019.100561

    Article  Google Scholar 

  67. Lozano-Diez A, Zazo R, Toledano DT, Gonzalez-Rodriguez J (2017) An analysis of the influence of deep neural network (DNN) topology in bottleneck feature based language recognition. PLoS ONE 12(8):e0182580. https://doi.org/10.1371/journal.pone.0182580

    Article  Google Scholar 

  68. Rengasamy D, Jafari M, Rothwell B, Chen X, Figueredo GP (2020) Deep learning with dynamically weighted loss function for sensor-based prognostics and health management. Sensors. https://doi.org/10.3390/s20030723

    Article  Google Scholar 

  69. Dernoncourt F (2013) Introduction to fuzzy logic. Massachusetts Institute of Technology, 21

  70. Dorzhigulov A, & James AP (2020) Deep Neuro-Fuzzy Architectures. In A. P. James (Ed.), Deep Learning Classifiers with Memristive Networks: Theory and Applications (pp. 195–213). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-14524-8_15

  71. Walia N, Singh H, Sharma A (2015) ANFIS: Adaptive neuro-fuzzy inference system-a survey. Int J Comput Appl. https://doi.org/10.5120/ijca2015905635

    Article  Google Scholar 

  72. Korshunova KP (2018) A Convolutional Fuzzy Neural Network for Image Classification. 2018 3rd Russian-Pacific Conference on Computer Technology and Applications (RPC), 1–4. https://doi.org/10.1109/RPC.2018.8482211

  73. Zhang L, Zhu Y, Shi X, Li X (2020) A situation assessment method with an improved fuzzy deep neural network for multiple UAVs. Information 11:194. https://doi.org/10.3390/info11040194

    Article  Google Scholar 

  74. Guha D, Roy PK, & Banerjee S (2018) Robust Optimization Algorithms for Solving Automatic Generation Control of Multi-Constrained Power System: Robustness Study of AGC Problem in Power System. In Handbook of research on power and energy system optimization (pp. 75–114): IGI Global. https://doi.org/10.4018/978-1-5225-3935-3.CH003

  75. Jadid Abdulkadir S, Yong S (2013) Variants of particle swarm optimization in enhancing artificial neural networks. Aust J Basic Appl Sci 7:388–400

    Google Scholar 

  76. Hussain K, Mohd Salleh MN, Cheng S, Shi Y (2019) Metaheuristic research: a comprehensive survey. Artif Intell Rev 52(4):2191–2233. https://doi.org/10.1007/s10462-017-9605-z

    Article  Google Scholar 

  77. Sun R-Y (2020) Optimization for deep learning: an overview. J Oper Res Soc China 8(2):249–294. https://doi.org/10.1007/s40305-020-00309-6

    Article  MathSciNet  MATH  Google Scholar 

  78. Yang X-S (2012) Nature-inspired mateheuristic algorithms: success and new challenges. J Comput Eng Inf Technol 1(1):1–3. https://doi.org/10.4172/2324-9307.1000e101

    Article  Google Scholar 

  79. Yang X-S, & Deb S (2015) Cuckoo search for optimization and computational intelligence. In Encyclopedia of Information Science and Technology, Third Edition (pp. 133–142): IGI global. http://doi.org/https://doi.org/10.4018/978-1-4666-5888-2.ch014

  80. Sweke R, Wilde F, Meyer JJ, Schuld M, Fährmann PK, Meynard-Piganeau B, & Eisert JJQ (2020) Stochastic gradient descent for hybrid quantum-classical optimization. 4, 314. https://doi.org/10.22331/q-2020-08-31-314

  81. Jiawei Z (2019) Gradient Descent based Optimization Algorithms for Deep Learning Models Training. ArXiv, abs/1903.03614

  82. Sun S, Cao Z, Zhu H, Zhao J (2020) A survey of optimization methods from a machine learning perspective. IEEE Trans Cybernetics 50(8):3668–3681. https://doi.org/10.1109/TCYB.2019.2950779

    Article  Google Scholar 

  83. Yeganejou M, & Dick S (2018) Classification via Deep Fuzzy c-Means Clustering. 2018 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 1–6. https://doi.org/10.1109/FUZZ-IEEE.2018.8491461

  84. Price SR, Price SR, & Anderson DT (2019) Introducing fuzzy layers for deep learning. 2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 1–6. https://doi.org/10.1109/FUZZ-IEEE.2019.8858790

  85. Kesse M, Buah E, Handroos H, Ayetor GK (2020) Development of an artificial intelligence powered TIG welding algorithm for the prediction of bead geometry for TIG welding processes using hybrid deep learning. Metals 10(4):451. https://doi.org/10.3390/met10040451

    Article  Google Scholar 

  86. Sharma D, Singh Aujla G, Bajaj R (2021) Deep neuro-fuzzy approach for risk and severity prediction using recommendation systems in connected health care. Trans Emerg Telecommun Technol 32(7):e4159. https://doi.org/10.1002/ett.4159

    Article  Google Scholar 

  87. Usman M, Carie A, Marapelli B, Bedru HD, Biswas K (2021) A human-in-the-loop probabilistic CNN-fuzzy logic framework for accident prediction in vehicular networks. IEEE Sens J 21(14):15496–15503. https://doi.org/10.1109/JSEN.2020.3023661

    Article  Google Scholar 

  88. Feng Q, Chen L, Chen CLP, Guo L (2020) Deep fuzzy clustering—a representation learning approach. IEEE Trans Fuzzy Syst 28(7):1420–1433. https://doi.org/10.1109/TFUZZ.2020.2966173

    Article  Google Scholar 

  89. Feng S, Chen CLP, Zhang C (2020) A fuzzy deep model based on fuzzy restricted boltzmann machines for high-dimensional data classification. IEEE Trans Fuzzy Syst 28(7):1344–1355. https://doi.org/10.1109/TFUZZ.2019.2902111

    Article  Google Scholar 

  90. Hare W (2020) A discussion on variational analysis in derivative-free optimization. Set-Valued Variational Anal. https://doi.org/10.1007/s11228-020-00556-y

    Article  MathSciNet  MATH  Google Scholar 

  91. Abd Elaziz M, Dahou A, Abualigah L, Yu L, Alshinwan M, Khasawneh AM, Lu S (2021) Advanced metaheuristic optimization techniques in applications of deep neural networks: a review. Neural Comput Appl. https://doi.org/10.1007/s00521-021-05960-5

    Article  Google Scholar 

  92. Hashim FA, Hussain K, Houssein EH, Mabrouk MS, Al-Atabany W (2021) Archimedes optimization algorithm: a new metaheuristic algorithm for solving optimization problems. Appl Intell 51(3):1531–1551. https://doi.org/10.1007/s10489-020-01893-z

    Article  MATH  Google Scholar 

  93. Holland JH (1992) Genetic algorithms. Sci Am 267(1):66–73

    Article  Google Scholar 

  94. Eberhart R, & Kennedy J (1995) A new optimizer using particle swarm theory. MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science, 39–43. https://doi.org/10.1109/MHS.1995.494215

  95. Dorigo M, Birattari M, Stutzle T (2006) Ant colony optimization. IEEE Comput Intell Mag 1(4):28–39. https://doi.org/10.1109/MCI.2006.329691

    Article  Google Scholar 

  96. Yang X-S, Deb S (2014) Cuckoo search: recent advances and applications. Neural Comput Appl 24(1):169–174. https://doi.org/10.1007/s00521-013-1367-1

    Article  Google Scholar 

  97. Yang XS, He X (2013) Firefly algorithm: recent advances and applications. Int J Swarm Intell 1(1):36–50. https://doi.org/10.1504/IJSI.2013.055801

    Article  Google Scholar 

  98. Turabieh H, Mafarja M, Li X (2019) Iterated feature selection algorithms with layered recurrent neural network for software fault prediction. Expert Syst Appl 122:27–42. https://doi.org/10.1016/j.eswa.2018.12.033

    Article  Google Scholar 

  99. Wang G-G, Deb S, & Coelho LDS (2015) Elephant herding optimization. 2015 3rd International Symposium on Computational and Business Intelligence (ISCBI), 1–5. https://doi.org/10.1109/ISCBI.2015.8

  100. Li J, Lei H, Alavi AH, Wang G-G (2020) Elephant herding optimization: variants, hybrids, and applications. Mathematics 8(9):1415. https://doi.org/10.3390/math8091415

    Article  Google Scholar 

  101. Gupta S, Singh V, Singh S, Prakash T, Rathore N (2016) Elephant herding optimization based PID controller tuning. Int J Adv Technol Eng Exploration 3(24):194. https://doi.org/10.19101/IJATEE.2016.324005

    Article  Google Scholar 

  102. Tuba E, & Stanimirovic Z (2017) Elephant herding optimization algorithm for support vector machine parameters tuning. 2017 9th International Conference on Electronics, Computers and Artificial Intelligence (ECAI), 1–4. https://doi.org/10.1109/ECAI.2017.8166464

  103. Rao R (2016) Jaya: a simple and new optimization algorithm for solving constrained and unconstrained optimization problems. Int J Ind Eng Comput 7(1):19–34. https://doi.org/10.5267/j.ijiec.2015.8.004

    Article  Google Scholar 

  104. Pandey HM (2016) Jaya a novel optimization algorithm: What, how and why? 2016 6th International Conference-Cloud System and Big Data Engineering (Confluence), 728–730. https://doi.org/10.1109/CONFLUENCE.2016.7508215

  105. Rao RV, Saroj A (2017) A self-adaptive multi-population based Jaya algorithm for engineering optimization. Swarm Evol Comput 37:1–26. https://doi.org/10.1016/j.swevo.2017.04.008

    Article  Google Scholar 

  106. Dede T (2018) Jaya algorithm to solve single objective size optimization problem for steel grillage structures. Steel and Composite Structures 26(2):163–170. https://doi.org/10.12989/scs.2018.26.2.163

    Article  MathSciNet  Google Scholar 

  107. Khatir S, Boutchicha D, Le Thanh C, Tran-Ngoc H, Nguyen TN, Abdel-Wahab M (2020) Improved ANN technique combined with Jaya algorithm for crack identification in plates using XIGA and experimental analysis. Theoret Appl Fract Mech 107:102554. https://doi.org/10.1016/j.tafmec.2020.102554

    Article  Google Scholar 

  108. Jothi G, Inbarani HH, Azar AT, Devi KR (2019) Rough set theory with Jaya optimization for acute lymphoblastic leukemia classification. Neural Comput Appl 31(9):5175–5194. https://doi.org/10.1007/s00521-018-3359-7

    Article  Google Scholar 

  109. Shi Y (2011) Brain storm optimization algorithm. A dvances in Swarm Intelligence. ICSI 2011. Lecture Notes in Computer Science, vol. 6728. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21515-5_36

  110. Cheng S, Qin Q, Chen J, Shi Y (2016) Brain storm optimization algorithm: a review. Artif Intell Rev 46(4):445–458. https://doi.org/10.1007/s10462-016-9471-0

    Article  Google Scholar 

  111. Talpur N, Salleh MNM, Hussain K (2017) An investigation of membership functions on performance of ANFIS for solving classification problems. IOP Conf Series Mater Sci Eng 226:012103. https://doi.org/10.1088/1757-899x/226/1/012103

    Article  Google Scholar 

  112. Nossier SA, Wall J, Moniri M, Glackin C, Cannings N (2021) An experimental analysis of deep learning architectures for supervised speech enhancement. Electronics. https://doi.org/10.3390/electronics10010017

    Article  Google Scholar 

  113. Han Y, Huang G, Song S, Yang L, Wang H, Wang Y (2021) Dynamic neural networks: A survey. arXiv preprint arXiv: 2102.04906.

  114. Mazandarani M, Li X (2020) Fractional fuzzy inference system: the new generation of fuzzy inference systems. IEEE Access 8:126066–126082. https://doi.org/10.1109/ACCESS.2020.3008064

    Article  Google Scholar 

  115. Selvachandran G, Quek SG, Lan LTH, Son LH, Giang NL, Ding W, Albuquerque VHCd (2021) A new design of mamdani complex fuzzy inference system for multiattribute decision making problems. IEEE Trans Fuzzy Syst 29(4):716–730. https://doi.org/10.1109/TFUZZ.2019.2961350

    Article  Google Scholar 

  116. Lan LTH, Tuan TM, Ngan TT, Giang NL, Ngoc VTN, Van Hai P (2020) A new complex fuzzy inference system with fuzzy knowledge graph and extensions in decision making. IEEE Access 8:164899–164921. https://doi.org/10.1109/ACCESS.2020.3021097

    Article  Google Scholar 

  117. Wu J, & Feng S (2017) Improved biogeography-based optimization for the traveling salesman problem. 2017 2nd IEEE International Conference on Computational Intelligence and Applications (ICCIA), 166–171. https://doi.org/10.1109/CIAPP.2017.8167201

  118. Biradar S, Hote YV (2016) Accelerated modified big bang big crunch optimization based on evolution of universe. 2016 11th International Conference on Industrial and Information Systems (ICIIS), 698–703. https://doi.org/10.1109/ICIINFS.2016.8263028

  119. Meena NK, Parashar S, Swarnkar A, Gupta N, Niazi KR (2017) Improved elephant herding optimization for multiobjective DER accommodation in distribution systems. IEEE Trans Industr Inf 14(3):1029–1039. https://doi.org/10.1109/TII.2017.2748220

    Article  Google Scholar 

  120. Barakat AF, El-Sehiemy RA, Elsayd MI, Osman E (2019) An enhanced Jaya optimization algorithm (EJOA) for solving multi-objective ORPD problem. Int Conf Innovative Trends Comput Eng (ITCE) 2019:479–484. https://doi.org/10.1109/ITCE.2019.8646363

    Article  Google Scholar 

  121. El-Abd M (2017) Global-best brain storm optimization algorithm. Swarm Evol Comput 37:27–44. https://doi.org/10.1016/j.swevo.2017.05.001

    Article  Google Scholar 

  122. Wang G-G, Deb S, Coelho LDS (2018) Earthworm optimisation algorithm: a bio-inspired metaheuristic algorithm for global optimisation problems. Int J Bio-Inspired Comput 12(1):1–22. https://doi.org/10.1504/IJBIC.2018.093328

    Article  Google Scholar 

  123. Mirjalili S, Lewis A (2016) The whale optimization algorithm. Adv Eng Softw 95:51–67. https://doi.org/10.1016/j.advengsoft.2016.01.008

    Article  Google Scholar 

  124. Wang G-G, Deb S, Cui Z (2019) Monarch butterfly optimization. Neural Comput Appl 31(7):1995–2014. https://doi.org/10.1007/s00521-015-1923-y

    Article  Google Scholar 

  125. Shabani A, Asgarian B, Gharebaghi SA, Salido MA, Giret A (2019) A New optimization algorithm based on search and rescue operations. Math Probl Eng 2019:2482543. https://doi.org/10.1155/2019/2482543

    Article  Google Scholar 

  126. Faramarzi A, Heidarinejad M, Mirjalili S, Gandomi AH (2020) Marine predators algorithm: a nature-inspired metaheuristic. Expert Syst Appl 152:113377. https://doi.org/10.1016/j.eswa.2020.113377

    Article  Google Scholar 

  127. Heidari AA, Mirjalili S, Faris H, Aljarah I, Mafarja M, Chen H (2019) Harris hawks optimization: algorithm and applications. Futur Gener Comput Syst 97:849–872. https://doi.org/10.1016/j.future.2019.02.028

    Article  Google Scholar 

  128. Kaveh A, & Eslamlou AD (2020) Water strider algorithm: A new metaheuristic and applications. In Structures (Vol. 25, pp. 520–541). Elsevier. https://doi.org/10.1016/j.istruc.2020.03.033

  129. Tejani GG, Pholdee N, Bureerat S, Prayogo D (2018) Multiobjective adaptive symbiotic organisms search for truss optimization problems. Knowl-Based Syst 161:398–414. https://doi.org/10.1016/j.knosys.2018.08.005

    Article  Google Scholar 

  130. Kumar S, Tejani GG, Pholdee N, Bureerat S (2021) Multi-Objective Passing Vehicle Search algorithm for structure optimization. Expert Syst Appl 169:114511. https://doi.org/10.1016/j.eswa.2020.114511

    Article  Google Scholar 

  131. Nematollahi AF, Rahiminejad A, Vahidi B (2019) A novel multi-objective optimization algorithm based on lightning attachment procedure optimization algorithm. Appl Soft Comput 75:404–427. https://doi.org/10.1016/j.asoc.2018.11.032

    Article  Google Scholar 

  132. Abdel-Basset M, Mohamed R, Mirjalili S (2021) A novel Whale Optimization Algorithm integrated with Nelder-Mead simplex for multi-objective optimization problems. Knowl-Based Syst 212:106619. https://doi.org/10.1016/j.knosys.2020.106619

    Article  Google Scholar 

  133. Aljarah I, Habib M, Faris H, Al-Madi N, Heidari AA, Mafarja M et al (2020) A dynamic locality multi-objective salp swarm algorithm for feature selection. Comput Ind Eng 147:106628. https://doi.org/10.1016/j.cie.2020.106628

    Article  Google Scholar 

  134. Kumar S, Tejani GG, Pholdee N, Bureerat S, Mehta P (2021) Hybrid heat transfer search and passing vehicle search optimizer for multi-objective structural optimization. Knowl-Based Syst 212:106556. https://doi.org/10.1016/j.knosys.2020.106556

    Article  Google Scholar 

Download references

Acknowledgements

Research reported in this publication was supported by Fundamental Research Grant Project (FRGS) from the Ministry of Education Malaysia (FRGS/1/2018/ICT02/UTP/03/1) under UTP grant number 015MA0-013.

Author information

Authors and Affiliations

  1. Centre for Research in Data Science, Computer Information Science Department, Universiti Teknologi PETRONAS, Seri Iskandar, Perak, Malaysia

    Noureen Talpur, Said Jadid Abdulkadir, Hitham Alhussian, ·Mohd Hilmi Hasan & Norshakirah Aziz

  2. Department of Computer Sciences, College of Computing, Umm Al Qura University Makkah, Al Qunfudhah, Saudi Arabia

    Alwi Bamhdi

Authors
  1. Noureen Talpur
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Said Jadid Abdulkadir
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hitham Alhussian
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. ·Mohd Hilmi Hasan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Norshakirah Aziz
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alwi Bamhdi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Noureen Talpur.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix

Appendix A: List of Abbreviations Used in this Manuscript

Abbreviation

Definition

A

Accuracy

ACO

Ant Colony Optimization

AGFS

Adaptive Genetic Fuzzy System

ANFIS

Adaptive Neuro-Fuzzy Inference Systems

ANN

Artificial Neural Networks

AOA

Archimedes Optimization Algorithm

BB-BC

Big Bang—Big Crunch

BBO

Biogeography-Based Optimization

BFC

Bayesian Fuzzy Clustering

BP

Back Propagation

BSO

Brain Storm Optimization

CFNN

Convolutional Fuzzy Neural Network

CG

Conjugate Gradient

CNN

Convolutional Neural Networks

CS

Cuckoo Search

DBN

Deep Belief Network

DDAE

Deep Denoising Auto-Encoder

DDoS

Distributed Denial-Of-Service

DFCM

Dynamic Fuzzy Cognitive Maps

DNFS

Deep Neuro-Fuzzy Systems

DNN

Deep Neural Networks

DT

Decision Trees

EHO

Elephant Herd Optimization

EJOA

Enhanced Jaya optimization algorithm

EOA

Earthworm Optimization Algorithm

F

Fallout

FCM

Fuzzy C-Means

FCNN

Fuzzy Convolutional Neural Network

FDBM

Fuzzy Deep Boltzmann Machine

FDDAE

Fuzzy Deep Denoising Auto-Encoder

FDNN

Fuzzy Deep Neural Network

FF

Firefly Algorithm

FFIS

Fractional Fuzzy Inference System

FIS

Fuzzy Inference System

FL

Fuzzy Logic

FSA

Fuzzy Stacked Autoencoder

FT-EHO-DBN

Fuzzy and Taylor-Elephant Herd Optimization Deep Belief Network

GA

Genetic Algorithms

GA

Genetic Algorithms

GBSO

Global-best Brain Storm Optimization

GD

Gradient Descent

GLW

Greedy Layer Wise

HF

Hessian-Free

HHO

Harris Hawks Optimization

IHABBO

Improved Hybridization of Adaptive Biogeography-Based Optimization

IMOEHO

Improved and Multi-Objective Elephant Herd Optimization

IT2FLS

Interval Type-2 Fuzzy Logic Systems

JOA

Jaya Optimization Algorithms

KNN

K-Nearest Neighbor

LAPO

Lightning Attachment Procedure Optimization

LSTMRN

Long Short Term Memory Recurrent Network

MBBBC

Modified Big Bang-Big Crunch

MBO

Monarch Butterfly Optimization

M-CFIS

Mamdani Complex Fuzzy Inference System

MIWOA

Multi-objective Improved Whale Optimization Algorithm

MOASOS

Multi-Objective Adaptive Symbiotic Organisms Search

MOHHTS–PVS

Multi-Objective Hybrid Heat Transfer Search and Passing Vehicle Search

MOPVS

Multi-Objective Passing Vehicle Search

MOSSA

Multi-objective Salp Swarm Algorithm

MPA

Marine Predators Algorithm

MRMSE

Mean of Root Mean Square Error

MSE

Mean Squared Error

NB

Naïve Bayes

P

Precision

PFDBM

Pythagorean Fuzzy Deep Boltzmann Machine

PSO

Particle Swarm Optimization

R/S

Recall/Sensitivity

RBM

Restricted Boltzmann Machine

RF

Random Forest

RMSE

Root Mean Squared Error

RNN

Recurrent Neural Network

S

Specificity

SA

Situation Assessment

SAE

Stacked Auto-Encoder

SAR

Search And Rescue optimization algorithm

SDRMSE

Standard Deviation of Root Mean Square Error

SGD

Stochastic Gradient Descent

SVM

Support Vector Machine

UAV

Unmanned Aerial Vehicle

WOA

Whale Optimization Algorithm

WSA

Water Strider Algorithm

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Talpur, N., Abdulkadir, S.J., Alhussian, H. et al. A comprehensive review of deep neuro-fuzzy system architectures and their optimization methods. Neural Comput & Applic 34, 1837–1875 (2022). https://doi.org/10.1007/s00521-021-06807-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06807-9

Keywords

search

Navigation