Skip to main content
Log in

A comprehensive review of deep neuro-fuzzy system architectures and their optimization methods

Neural Computing and Applications Aims and scope Submit manuscript

Cite this article


Deep neuro-fuzzy systems (DNFSs) have been successfully applied to real-world problems using the efficient learning process of deep neural networks (DNNs) and reasoning aptitude from fuzzy inference systems (FIS). This study provides a comprehensive review of DNFS dividing it into two essential parts. The first part aims to provide a thorough understanding of DNFS and its architectural representation, whereas the second part reviews DNFS optimization methods. This study aims to assist researchers in understanding the various ways DNFS models are developed by hybridizing DNN and FIS, as well as gradient (derivative)-based methods and metaheuristics (derivative-free) optimization, as discussed in the literature. This study revealed that the proposed DNFS architectures performed 11.6% better than non-fuzzy models, with an overall accuracy of 81.4%. The investigation based on optimization methods revealed that DNFS with metaheuristics optimization methods has shown an overall accuracy of 93.56%, which is 21.10% higher than the DNFS models using gradient-based methods. Additionally, this study showed that DNFS networks presented in the literature have integrated DNN with typical FIS, although more satisfactory results can be obtained using a new generation of FIS termed fractional FIS (FFIS) and Mamdani complex FIS (M-CFIS). Besides, dynamic neural networks are suggested in the replacement of static DNNs to facilitate dynamic learning. Some studies have also demonstrated the optimization of DNFS using classical gradient-based approaches that can affect network performance when solving highly nonlinear problems. This study suggests implementing optimization methods with new and improvised metaheuristics to improve the training and performance of the models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18


  1. Sharmadha S, Shivani K, Shruthi K, Bharathi B, & Kavitha S (2020) Automatic speech recognition using deep neural network. Advances in Intelligent Systems and Computing, vol 1118. Springer, Singapore.

  2. Widiastuti NI (2019) Convolution neural network for text mining and natural language processing. IOP Conf Series Mater Sci Eng 662:052010.

    Article  Google Scholar 

  3. Guo J, Fan Y, Pang L, Yang L, Ai Q, Zamani H, Cheng X (2020) A Deep Look into neural ranking models for information retrieval. Inf Process Manage 57(6):102067.

    Article  Google Scholar 

  4. Nishani E, Çiço B (2017) Computer vision approaches based on deep learning and neural networks: deep neural networks for video analysis of human pose estimation. 2017 6th Mediterranean Conference on Embedded Computing (MECO), pp. 1–4.

  5. Wainberg M, Merico D, Delong A, Frey BJ (2018) Deep learning in biomedicine. Nat Biotechnol 36(9):829–838.

    Article  Google Scholar 

  6. Romascanu A, Ker H, Sieber R, Greenidge S, Lumley S, Bush D, Morgan S, Zhao R, Brunila M (2020) Using deep learning and social network analysis to understand and manage extreme flooding. J Conting Crisis Manag 28(3):251–261.

    Article  Google Scholar 

  7. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444.

    Article  Google Scholar 

  8. Schaedler M, Blümm C, Kuschnerov M, Pittalà F, Calabrò S, Pachnicke S (2019) Deep neural network equalization for optical short reach communication. Appl Sci 9:4675.

    Article  Google Scholar 

  9. Altwaijry N, Al-Turaiki I (2021) Arabic handwriting recognition system using convolutional neural network. Neural Comput Appl 33(7):2249–2261.

    Article  Google Scholar 

  10. Bullinaria JA (2013) Recurrent neural networks. Neural Comput: Lecture, 12

  11. Abdullah MHA, Othman M, Kasim S, Mohamed SA (2019) Evolving spiking neural networks methods for classification problem: a case study in flood events risk assessment. Indonesian J Electr Eng Computer Sci 16:222–229.

    Article  Google Scholar 

  12. Said J, Jadid Abdulkadir S, Alhussian H, Nazmi M, Elsheikh A (2018) Long short term memory recurrent network for standard and poor’s 500 index modelling. Int J Eng Technol 7:25–29.

    Article  Google Scholar 

  13. Qiu Y, Dai Y (2019) A stacked auto-encoder based fault diagnosis model for chemical process. In Computer Aided Chemical Engineering (Vol. 46, pp. 1303–1308): Elsevier.

  14. Hua Y, Guo J, Zhao H (2015) Deep belief networks and deep learning. Proceedings of 2015 International Conference on Intelligent Computing and Internet of Things, 2015, pp. 1–4,

  15. Zhang N, Ding S, Zhang J, Xue Y (2018) An overview on restricted Boltzmann machines. Neurocomputing 275:1186–1199.

    Article  Google Scholar 

  16. Bonanno D, Nock K, Smith L, Elmore P, Petry F (2017) An approach to explainable deep learning using fuzzy inference (Vol. 10207): SPIE.

  17. Hayashi Y (2020) Black Box Nature of Deep Learning for Digital Pathology: Beyond Quantitative to Qualitative Algorithmic Performances. In Artificial Intelligence and Machine Learning for Digital Pathology (pp. 95–101): Springer.

  18. Buhrmester V, Münch D, Arens M (2019) Analysis of explainers of black box deep neural networks for computer vision: A survey. arXiv preprint arXiv: 1911.12116

  19. Aviles AI, Alsaleh SM, Montseny E, Sobrevilla P, Casals A (2016) A Deep-Neuro-Fuzzy approach for estimating the interaction forces in Robotic surgery. 2016 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 1113–1119.

  20. Zheng Y, Sheng W, Sun X, Chen S (2017) Airline passenger profiling based on fuzzy deep machine learning. IEEE Trans Neural Netw Learn Syst 28(12):2911–2923.

    Article  MathSciNet  Google Scholar 

  21. El Hatri C, Boumhidi J (2018) Fuzzy deep learning based urban traffic incident detection. Cogn Syst Res 50:206–213.

    Article  Google Scholar 

  22. Ramasamy B, Hameed AZ (2019) Classification of healthcare data using hybridised fuzzy and convolutional neural network. Healthcare Technol Letters 6(3):59–63.

    Article  Google Scholar 

  23. Aye YY, Thiha K, Pyu MMM, Watanabe K (2019) A deep neural network based human following robot with fuzzy control. IEEE Int Confn Robotics Biomimetics (ROBIO) 2019:720–725.

    Article  Google Scholar 

  24. Chopade HA, Narvekar M (2017) Hybrid auto text summarization using deep neural network and fuzzy logic system. Int Conf Inventive Comput Inf (ICICI) 2017:52–56.

    Article  Google Scholar 

  25. Zhang L, Zhu Y, Shi X, Li X (2020) A Situation Assessment Method with an Improved Fuzzy Deep Neural Network for Multiple UAVs. Information.

    Article  Google Scholar 

  26. Liao P, Xu M, Yang C (2020) A fuzzy ensemble method with deep learning for multi-robot system. IEEE Access 8:220352–220363.

    Article  Google Scholar 

  27. Yin P, Dou G, Lin X, Liu L (2020) A hybrid method for forecasting new product sales based on fuzzy clustering and deep learning. Kybernetes 49(12):3099–3118.

    Article  Google Scholar 

  28. Asghar MZ, Subhan F, Ahmad H, Khan WZ, Hakak S, Gadekallu TR, Alazab M (2021) Senti-eSystem: a sentiment-based eSystem-using hybridized fuzzy and deep neural network for measuring customer satisfaction. Softw Practice Exp 51(3):571–594.

    Article  Google Scholar 

  29. Bedi P, & Khurana P (2020) Sentiment Analysis Using Fuzzy-Deep Learning. Proceedings of ICETIT 2019, Cham.

  30. Shalaginov A, Franke K (2017) A deep neuro-fuzzy method for multi-label malware classification and fuzzy rules extraction. IEEE Symposium Series Comput Intell (SSCI) 2017:1–8.

    Article  Google Scholar 

  31. Chen D, Zhang X, Wang LL, Han Z (2019) Prediction of cloud resources demand based on hierarchical pythagorean fuzzy deep neural network. IEEE Trans Serv Comput.

    Article  Google Scholar 

  32. Monsefi AK, Zakeri B, Samsam S, & Khashehchi M (2019) Performing software test oracle based on deep neural network with fuzzy inference system. International Congress on High-Performance Computing and Big Data Analysis, 406–417. Springer, Cham, 2019.

  33. Nguyen T-L, Kavuri S, Lee M (2019) A multimodal convolutional neuro-fuzzy network for emotion understanding of movie clips. Neural Netw 118:208–219.

    Article  Google Scholar 

  34. Greeshma MS, Bindu VR (2017) Single image super resolution using fuzzy deep convolutional networks. Int Conf Technol Adv Power Energy (TAP Energy) 2017:1–6.

    Article  Google Scholar 

  35. Guan C, Wang S, Liew AW (2020) Lip image segmentation based on a fuzzy convolutional neural network. IEEE Trans Fuzzy Syst 28(7):1242–1251.

    Article  Google Scholar 

  36. Nguyen T-L, Kavuri S, Lee M (2018) A fuzzy convolutional neural network for text sentiment analysis. J Intell Fuzzy Syst 35(6):6025–6034.

    Article  Google Scholar 

  37. Deng Y, Ren Z, Kong Y, Bao F, Dai Q (2017) A hierarchical fused fuzzy deep neural network for data classification. IEEE Trans Fuzzy Syst 25(4):1006–1012.

    Article  Google Scholar 

  38. Lima S, Terán L, Portmann E (2020) A proposal for an explainable fuzzy-based deep learning system for skin cancer prediction. Seventh Int Conf eDemocracy eGovernment (ICEDEG) 2020:29–35.

    Article  Google Scholar 

  39. Yang CH, Moi SH, Hou MF, Chuang LY, Lin YD (2020) Applications of deep learning and fuzzy systems to detect cancer mortality in next-generation genomic data. IEEE Trans Fuzzy Syst.

    Article  Google Scholar 

  40. Shen T, Wang J, Gou C, Wang FY (2020) Hierarchical fused model with deep learning and type-2 fuzzy learning for breast cancer diagnosis. IEEE Trans Fuzzy Syst 28(12):3204–3218.

    Article  Google Scholar 

  41. Zhang R, Shen F, Zhao J (2014) A model with fuzzy granulation and deep belief networks for exchange rate forecasting. Int Joint Conf Neural Netw (IJCNN) 2014:366–373.

    Article  Google Scholar 

  42. Chen W, An J, Li R, Fu L, Xie G, Bhuiyan MZA, Li K (2018) A novel fuzzy deep-learning approach to traffic flow prediction with uncertain spatial–temporal data features. Futur Gener Comput Syst 89:78–88.

    Article  Google Scholar 

  43. Van ND, & Kim G (2018) Fuzzy Logic and Deep Steering Control based Recommendation System for Self-Driving Car. 2018 18th International Conference on Control, Automation and Systems (ICCAS), 1107–1110

  44. Zheng Y-J, Sheng W-G, Sun X-M, Chen S-Y (2016) Airline passenger profiling based on fuzzy deep machine learning. IEEE Trans Neural Netw Learn Syst 28(12):2911–2923.

    Article  MathSciNet  Google Scholar 

  45. Wang L-X (2019) Fast training algorithms for deep convolutional fuzzy systems with application to stock index prediction. IEEE Trans Fuzzy Syst 28(7):1301–1314.

    Article  Google Scholar 

  46. Chen X, Rajan D, Quek C (2020) A deep hybrid fuzzy neural Hammerstein-Wiener network for stock price prediction. Int Conf Artificial Intell Inf Commun (ICAIIC) 2020:288–293.

    Article  Google Scholar 

  47. Chandrasekar R (2020) Fuzzy crow search algorithm-based deep LSTM for bitcoin prediction. Int J Distributed Syst Technol (IJDST) 11(4):53–71.

    Article  MathSciNet  Google Scholar 

  48. Xiao P (2020) Information management of E-Commerce platform based on neural networks and fuzzy deep learning models. Int Conf Smart Electron Commun (ICOSEC) 2020:532–535.

    Article  Google Scholar 

  49. Elavarasan D, Vincent PMDR (2021) Fuzzy deep learning-based crop yield prediction model for sustainable agronomical frameworks. Neural Comput Appl.

    Article  Google Scholar 

  50. Fan T, Xu J (2020) Image classification of crop diseases and pests based on deep learning and fuzzy system. Int J Data Warehousing Mining (IJDWM) 16(2):34–47.

    Article  MathSciNet  Google Scholar 

  51. Zheng Y, Chen S, Xue Y, Xue J (2017) A pythagorean-type fuzzy deep denoising autoencoder for industrial accident early warning. IEEE Trans Fuzzy Syst 25(6):1561–1575.

    Article  Google Scholar 

  52. Tian Z, Fong S (2016) Survey of meta-heuristic algorithms for deep learning training. Optim Algorithms-Methods Appl.

    Article  Google Scholar 

  53. Rere L, Fanany MI, Arymurthy AM (2016) Metaheuristic algorithms for convolution neural network. Comput Intell Neurosci.

    Article  Google Scholar 

  54. Akay B, Karaboga D, Akay R (2021) A comprehensive survey on optimizing deep learning models by metaheuristics. Artif Intell Rev.

    Article  MATH  Google Scholar 

  55. Altundogan TG, Karakose M (2019) Multiple object tracking with dynamic fuzzy cognitive maps using deep learning. Int Artif Intell Data Process Symposium (IDAP) 2019:1–5.

    Article  Google Scholar 

  56. Velliangiri S, Pandey HM (2020) Fuzzy-taylor-elephant herd optimization inspired deep belief network for DDoS attack detection and comparison with state-of-the-arts algorithms. Futur Gener Comput Syst 110:80–90.

    Article  Google Scholar 

  57. Siva Raja PM, Rani AV (2020) Brain tumor classification using a hybrid deep autoencoder with Bayesian fuzzy clustering-based segmentation approach. Biocybernetics Biomed Eng 40(1):440–453.

    Article  Google Scholar 

  58. Ravi C (2020) Image classification using deep learning and fuzzy systems. Intelligent Systems Design and Applications, vol 941. Springer, Cham.

  59. Chimatapu R, Hagras H, Starkey A, & Owusu G (2018) Interval Type-2 Fuzzy Logic Based Stacked Autoencoder Deep Neural Network For Generating Explainable AI Models in Workforce Optimization. 2018 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 1–8.

  60. Singh G, Pal M, Yadav Y, Singla T (2020) Deep neural network-based predictive modeling of road accidents. Neural Comput Appl.

    Article  Google Scholar 

  61. Najafabadi MM, Villanustre F, Khoshgoftaar TM, Seliya N, Wald R, Muharemagic E (2015) Deep learning applications and challenges in big data analytics. Journal of Big Data 2(1):1.

    Article  Google Scholar 

  62. Talpur N, Abdulkadir SJ, Hasan MH (2020) A deep learning based neuro-fuzzy approach for solving classification problems. Int Conf Comput Intell (ICCI) 2020:167–172.

    Article  Google Scholar 

  63. Jadid Abdulkadir S, Shamsuddin SM, Sallehuddin R (2012) Moisture Prediction in maize using three term back propagation neural network. Int J Environ Sci Dev.

    Article  Google Scholar 

  64. Khamparia A, Singh KM (2019) A systematic review on deep learning architectures and applications. Expert Syst 36(3):e12400.

    Article  Google Scholar 

  65. Suto J, Oniga S (2019) Efficiency investigation from shallow to deep neural network techniques in human activity recognition. Cogn Syst Res 54:37–49.

    Article  Google Scholar 

  66. Zhou X-H, Zhang M-X, Xu Z-G, Cai C-Y, Huang Y-J, Zheng Y-J (2019) Shallow and deep neural network training by water wave optimization. Swarm Evol Comput 50:100561.

    Article  Google Scholar 

  67. Lozano-Diez A, Zazo R, Toledano DT, Gonzalez-Rodriguez J (2017) An analysis of the influence of deep neural network (DNN) topology in bottleneck feature based language recognition. PLoS ONE 12(8):e0182580.

    Article  Google Scholar 

  68. Rengasamy D, Jafari M, Rothwell B, Chen X, Figueredo GP (2020) Deep learning with dynamically weighted loss function for sensor-based prognostics and health management. Sensors.

    Article  Google Scholar 

  69. Dernoncourt F (2013) Introduction to fuzzy logic. Massachusetts Institute of Technology, 21

  70. Dorzhigulov A, & James AP (2020) Deep Neuro-Fuzzy Architectures. In A. P. James (Ed.), Deep Learning Classifiers with Memristive Networks: Theory and Applications (pp. 195–213). Cham: Springer International Publishing.

  71. Walia N, Singh H, Sharma A (2015) ANFIS: Adaptive neuro-fuzzy inference system-a survey. Int J Comput Appl.

    Article  Google Scholar 

  72. Korshunova KP (2018) A Convolutional Fuzzy Neural Network for Image Classification. 2018 3rd Russian-Pacific Conference on Computer Technology and Applications (RPC), 1–4.

  73. Zhang L, Zhu Y, Shi X, Li X (2020) A situation assessment method with an improved fuzzy deep neural network for multiple UAVs. Information 11:194.

    Article  Google Scholar 

  74. Guha D, Roy PK, & Banerjee S (2018) Robust Optimization Algorithms for Solving Automatic Generation Control of Multi-Constrained Power System: Robustness Study of AGC Problem in Power System. In Handbook of research on power and energy system optimization (pp. 75–114): IGI Global.

  75. Jadid Abdulkadir S, Yong S (2013) Variants of particle swarm optimization in enhancing artificial neural networks. Aust J Basic Appl Sci 7:388–400

    Google Scholar 

  76. Hussain K, Mohd Salleh MN, Cheng S, Shi Y (2019) Metaheuristic research: a comprehensive survey. Artif Intell Rev 52(4):2191–2233.

    Article  Google Scholar 

  77. Sun R-Y (2020) Optimization for deep learning: an overview. J Oper Res Soc China 8(2):249–294.

    Article  MathSciNet  MATH  Google Scholar 

  78. Yang X-S (2012) Nature-inspired mateheuristic algorithms: success and new challenges. J Comput Eng Inf Technol 1(1):1–3.

    Article  Google Scholar 

  79. Yang X-S, & Deb S (2015) Cuckoo search for optimization and computational intelligence. In Encyclopedia of Information Science and Technology, Third Edition (pp. 133–142): IGI global.

  80. Sweke R, Wilde F, Meyer JJ, Schuld M, Fährmann PK, Meynard-Piganeau B, & Eisert JJQ (2020) Stochastic gradient descent for hybrid quantum-classical optimization. 4, 314.

  81. Jiawei Z (2019) Gradient Descent based Optimization Algorithms for Deep Learning Models Training. ArXiv, abs/1903.03614

  82. Sun S, Cao Z, Zhu H, Zhao J (2020) A survey of optimization methods from a machine learning perspective. IEEE Trans Cybernetics 50(8):3668–3681.

    Article  Google Scholar 

  83. Yeganejou M, & Dick S (2018) Classification via Deep Fuzzy c-Means Clustering. 2018 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 1–6.

  84. Price SR, Price SR, & Anderson DT (2019) Introducing fuzzy layers for deep learning. 2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 1–6.

  85. Kesse M, Buah E, Handroos H, Ayetor GK (2020) Development of an artificial intelligence powered TIG welding algorithm for the prediction of bead geometry for TIG welding processes using hybrid deep learning. Metals 10(4):451.

    Article  Google Scholar 

  86. Sharma D, Singh Aujla G, Bajaj R (2021) Deep neuro-fuzzy approach for risk and severity prediction using recommendation systems in connected health care. Trans Emerg Telecommun Technol 32(7):e4159.

    Article  Google Scholar 

  87. Usman M, Carie A, Marapelli B, Bedru HD, Biswas K (2021) A human-in-the-loop probabilistic CNN-fuzzy logic framework for accident prediction in vehicular networks. IEEE Sens J 21(14):15496–15503.

    Article  Google Scholar 

  88. Feng Q, Chen L, Chen CLP, Guo L (2020) Deep fuzzy clustering—a representation learning approach. IEEE Trans Fuzzy Syst 28(7):1420–1433.

    Article  Google Scholar 

  89. Feng S, Chen CLP, Zhang C (2020) A fuzzy deep model based on fuzzy restricted boltzmann machines for high-dimensional data classification. IEEE Trans Fuzzy Syst 28(7):1344–1355.

    Article  Google Scholar 

  90. Hare W (2020) A discussion on variational analysis in derivative-free optimization. Set-Valued Variational Anal.

    Article  MathSciNet  MATH  Google Scholar 

  91. Abd Elaziz M, Dahou A, Abualigah L, Yu L, Alshinwan M, Khasawneh AM, Lu S (2021) Advanced metaheuristic optimization techniques in applications of deep neural networks: a review. Neural Comput Appl.

    Article  Google Scholar 

  92. Hashim FA, Hussain K, Houssein EH, Mabrouk MS, Al-Atabany W (2021) Archimedes optimization algorithm: a new metaheuristic algorithm for solving optimization problems. Appl Intell 51(3):1531–1551.

    Article  MATH  Google Scholar 

  93. Holland JH (1992) Genetic algorithms. Sci Am 267(1):66–73

    Article  Google Scholar 

  94. Eberhart R, & Kennedy J (1995) A new optimizer using particle swarm theory. MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science, 39–43.

  95. Dorigo M, Birattari M, Stutzle T (2006) Ant colony optimization. IEEE Comput Intell Mag 1(4):28–39.

    Article  Google Scholar 

  96. Yang X-S, Deb S (2014) Cuckoo search: recent advances and applications. Neural Comput Appl 24(1):169–174.

    Article  Google Scholar 

  97. Yang XS, He X (2013) Firefly algorithm: recent advances and applications. Int J Swarm Intell 1(1):36–50.

    Article  Google Scholar 

  98. Turabieh H, Mafarja M, Li X (2019) Iterated feature selection algorithms with layered recurrent neural network for software fault prediction. Expert Syst Appl 122:27–42.

    Article  Google Scholar 

  99. Wang G-G, Deb S, & Coelho LDS (2015) Elephant herding optimization. 2015 3rd International Symposium on Computational and Business Intelligence (ISCBI), 1–5.

  100. Li J, Lei H, Alavi AH, Wang G-G (2020) Elephant herding optimization: variants, hybrids, and applications. Mathematics 8(9):1415.

    Article  Google Scholar 

  101. Gupta S, Singh V, Singh S, Prakash T, Rathore N (2016) Elephant herding optimization based PID controller tuning. Int J Adv Technol Eng Exploration 3(24):194.

    Article  Google Scholar 

  102. Tuba E, & Stanimirovic Z (2017) Elephant herding optimization algorithm for support vector machine parameters tuning. 2017 9th International Conference on Electronics, Computers and Artificial Intelligence (ECAI), 1–4.

  103. Rao R (2016) Jaya: a simple and new optimization algorithm for solving constrained and unconstrained optimization problems. Int J Ind Eng Comput 7(1):19–34.

    Article  Google Scholar 

  104. Pandey HM (2016) Jaya a novel optimization algorithm: What, how and why? 2016 6th International Conference-Cloud System and Big Data Engineering (Confluence), 728–730.

  105. Rao RV, Saroj A (2017) A self-adaptive multi-population based Jaya algorithm for engineering optimization. Swarm Evol Comput 37:1–26.

    Article  Google Scholar 

  106. Dede T (2018) Jaya algorithm to solve single objective size optimization problem for steel grillage structures. Steel and Composite Structures 26(2):163–170.

    Article  MathSciNet  Google Scholar 

  107. Khatir S, Boutchicha D, Le Thanh C, Tran-Ngoc H, Nguyen TN, Abdel-Wahab M (2020) Improved ANN technique combined with Jaya algorithm for crack identification in plates using XIGA and experimental analysis. Theoret Appl Fract Mech 107:102554.

    Article  Google Scholar 

  108. Jothi G, Inbarani HH, Azar AT, Devi KR (2019) Rough set theory with Jaya optimization for acute lymphoblastic leukemia classification. Neural Comput Appl 31(9):5175–5194.

    Article  Google Scholar 

  109. Shi Y (2011) Brain storm optimization algorithm. A dvances in Swarm Intelligence. ICSI 2011. Lecture Notes in Computer Science, vol. 6728. Springer, Berlin, Heidelberg.

  110. Cheng S, Qin Q, Chen J, Shi Y (2016) Brain storm optimization algorithm: a review. Artif Intell Rev 46(4):445–458.

    Article  Google Scholar 

  111. Talpur N, Salleh MNM, Hussain K (2017) An investigation of membership functions on performance of ANFIS for solving classification problems. IOP Conf Series Mater Sci Eng 226:012103.

    Article  Google Scholar 

  112. Nossier SA, Wall J, Moniri M, Glackin C, Cannings N (2021) An experimental analysis of deep learning architectures for supervised speech enhancement. Electronics.

    Article  Google Scholar 

  113. Han Y, Huang G, Song S, Yang L, Wang H, Wang Y (2021) Dynamic neural networks: A survey. arXiv preprint arXiv: 2102.04906.

  114. Mazandarani M, Li X (2020) Fractional fuzzy inference system: the new generation of fuzzy inference systems. IEEE Access 8:126066–126082.

    Article  Google Scholar 

  115. Selvachandran G, Quek SG, Lan LTH, Son LH, Giang NL, Ding W, Albuquerque VHCd (2021) A new design of mamdani complex fuzzy inference system for multiattribute decision making problems. IEEE Trans Fuzzy Syst 29(4):716–730.

    Article  Google Scholar 

  116. Lan LTH, Tuan TM, Ngan TT, Giang NL, Ngoc VTN, Van Hai P (2020) A new complex fuzzy inference system with fuzzy knowledge graph and extensions in decision making. IEEE Access 8:164899–164921.

    Article  Google Scholar 

  117. Wu J, & Feng S (2017) Improved biogeography-based optimization for the traveling salesman problem. 2017 2nd IEEE International Conference on Computational Intelligence and Applications (ICCIA), 166–171.

  118. Biradar S, Hote YV (2016) Accelerated modified big bang big crunch optimization based on evolution of universe. 2016 11th International Conference on Industrial and Information Systems (ICIIS), 698–703.

  119. Meena NK, Parashar S, Swarnkar A, Gupta N, Niazi KR (2017) Improved elephant herding optimization for multiobjective DER accommodation in distribution systems. IEEE Trans Industr Inf 14(3):1029–1039.

    Article  Google Scholar 

  120. Barakat AF, El-Sehiemy RA, Elsayd MI, Osman E (2019) An enhanced Jaya optimization algorithm (EJOA) for solving multi-objective ORPD problem. Int Conf Innovative Trends Comput Eng (ITCE) 2019:479–484.

    Article  Google Scholar 

  121. El-Abd M (2017) Global-best brain storm optimization algorithm. Swarm Evol Comput 37:27–44.

    Article  Google Scholar 

  122. Wang G-G, Deb S, Coelho LDS (2018) Earthworm optimisation algorithm: a bio-inspired metaheuristic algorithm for global optimisation problems. Int J Bio-Inspired Comput 12(1):1–22.

    Article  Google Scholar 

  123. Mirjalili S, Lewis A (2016) The whale optimization algorithm. Adv Eng Softw 95:51–67.

    Article  Google Scholar 

  124. Wang G-G, Deb S, Cui Z (2019) Monarch butterfly optimization. Neural Comput Appl 31(7):1995–2014.

    Article  Google Scholar 

  125. Shabani A, Asgarian B, Gharebaghi SA, Salido MA, Giret A (2019) A New optimization algorithm based on search and rescue operations. Math Probl Eng 2019:2482543.

    Article  Google Scholar 

  126. Faramarzi A, Heidarinejad M, Mirjalili S, Gandomi AH (2020) Marine predators algorithm: a nature-inspired metaheuristic. Expert Syst Appl 152:113377.

    Article  Google Scholar 

  127. Heidari AA, Mirjalili S, Faris H, Aljarah I, Mafarja M, Chen H (2019) Harris hawks optimization: algorithm and applications. Futur Gener Comput Syst 97:849–872.

    Article  Google Scholar 

  128. Kaveh A, & Eslamlou AD (2020) Water strider algorithm: A new metaheuristic and applications. In Structures (Vol. 25, pp. 520–541). Elsevier.

  129. Tejani GG, Pholdee N, Bureerat S, Prayogo D (2018) Multiobjective adaptive symbiotic organisms search for truss optimization problems. Knowl-Based Syst 161:398–414.

    Article  Google Scholar 

  130. Kumar S, Tejani GG, Pholdee N, Bureerat S (2021) Multi-Objective Passing Vehicle Search algorithm for structure optimization. Expert Syst Appl 169:114511.

    Article  Google Scholar 

  131. Nematollahi AF, Rahiminejad A, Vahidi B (2019) A novel multi-objective optimization algorithm based on lightning attachment procedure optimization algorithm. Appl Soft Comput 75:404–427.

    Article  Google Scholar 

  132. Abdel-Basset M, Mohamed R, Mirjalili S (2021) A novel Whale Optimization Algorithm integrated with Nelder-Mead simplex for multi-objective optimization problems. Knowl-Based Syst 212:106619.

    Article  Google Scholar 

  133. Aljarah I, Habib M, Faris H, Al-Madi N, Heidari AA, Mafarja M et al (2020) A dynamic locality multi-objective salp swarm algorithm for feature selection. Comput Ind Eng 147:106628.

    Article  Google Scholar 

  134. Kumar S, Tejani GG, Pholdee N, Bureerat S, Mehta P (2021) Hybrid heat transfer search and passing vehicle search optimizer for multi-objective structural optimization. Knowl-Based Syst 212:106556.

    Article  Google Scholar 

Download references


Research reported in this publication was supported by Fundamental Research Grant Project (FRGS) from the Ministry of Education Malaysia (FRGS/1/2018/ICT02/UTP/03/1) under UTP grant number 015MA0-013.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Noureen Talpur.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.



Appendix A: List of Abbreviations Used in this Manuscript






Ant Colony Optimization


Adaptive Genetic Fuzzy System


Adaptive Neuro-Fuzzy Inference Systems


Artificial Neural Networks


Archimedes Optimization Algorithm


Big Bang—Big Crunch


Biogeography-Based Optimization


Bayesian Fuzzy Clustering


Back Propagation


Brain Storm Optimization


Convolutional Fuzzy Neural Network


Conjugate Gradient


Convolutional Neural Networks


Cuckoo Search


Deep Belief Network


Deep Denoising Auto-Encoder


Distributed Denial-Of-Service


Dynamic Fuzzy Cognitive Maps


Deep Neuro-Fuzzy Systems


Deep Neural Networks


Decision Trees


Elephant Herd Optimization


Enhanced Jaya optimization algorithm


Earthworm Optimization Algorithm




Fuzzy C-Means


Fuzzy Convolutional Neural Network


Fuzzy Deep Boltzmann Machine


Fuzzy Deep Denoising Auto-Encoder


Fuzzy Deep Neural Network


Firefly Algorithm


Fractional Fuzzy Inference System


Fuzzy Inference System


Fuzzy Logic


Fuzzy Stacked Autoencoder


Fuzzy and Taylor-Elephant Herd Optimization Deep Belief Network


Genetic Algorithms


Genetic Algorithms


Global-best Brain Storm Optimization


Gradient Descent


Greedy Layer Wise




Harris Hawks Optimization


Improved Hybridization of Adaptive Biogeography-Based Optimization


Improved and Multi-Objective Elephant Herd Optimization


Interval Type-2 Fuzzy Logic Systems


Jaya Optimization Algorithms


K-Nearest Neighbor


Lightning Attachment Procedure Optimization


Long Short Term Memory Recurrent Network


Modified Big Bang-Big Crunch


Monarch Butterfly Optimization


Mamdani Complex Fuzzy Inference System


Multi-objective Improved Whale Optimization Algorithm


Multi-Objective Adaptive Symbiotic Organisms Search


Multi-Objective Hybrid Heat Transfer Search and Passing Vehicle Search


Multi-Objective Passing Vehicle Search


Multi-objective Salp Swarm Algorithm


Marine Predators Algorithm


Mean of Root Mean Square Error


Mean Squared Error


Naïve Bayes




Pythagorean Fuzzy Deep Boltzmann Machine


Particle Swarm Optimization




Restricted Boltzmann Machine


Random Forest


Root Mean Squared Error


Recurrent Neural Network




Situation Assessment


Stacked Auto-Encoder


Search And Rescue optimization algorithm


Standard Deviation of Root Mean Square Error


Stochastic Gradient Descent


Support Vector Machine


Unmanned Aerial Vehicle


Whale Optimization Algorithm


Water Strider Algorithm

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Talpur, N., Abdulkadir, S.J., Alhussian, H. et al. A comprehensive review of deep neuro-fuzzy system architectures and their optimization methods. Neural Comput & Applic 34, 1837–1875 (2022).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: