Skip to main content
SpringerLink
Log in

A Comprehensive Review on the Advancement of High-Dimensional Neural Networks in Quaternionic Domain with Relevant Applications

  • Review article
  • Published:
Archives of Computational Methods in Engineering Aims and scope Submit manuscript

Abstract

The neurocomputing communities have focused much interest on quaternionic-valued neural networks (QVNNs) due to the natural extension in quaternionic signals, learning of inter and spatial relationships between the features, and remarkable improvement against real-valued neural networks (RVNNs) and complex-valued neural networks (CVNNs). The excellent learning capability of QVNN inspired the researchers working on various applications in image processing, signal processing, computer vision, and robotic control system. Apart from its applications, many researchers have proposed new structures of quaternionic neurons and extended the architecture of QVNN for specific applications containing high-dimensional information. These networks have revealed their performance with a lesser number of parameters over conventional RVNNs. This paper focuses on past and recent studies of simple and deep QVNNs architectures and their applications. This paper provides the future directions to prospective researchers to establish new architectures and to extend the existing architecture of high-dimensional neural networks with the help of quaternion, octonion, or sedenion for appropriate applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Mikolov T, Karafiát M, Burget L, Cernockỳ J, Khudanpur S (2010) Recurrent neural network based language model. In: Interspeech, vol. 2, pp. 1045–1048. Makuhari

  2. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  3. Graves A, Mohamed A-r, Hinton G (2013) Speech recognition with deep recurrent neural networks. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 6645–6649. Ieee

  4. Hinton GE, Osindero S, Teh Y-W (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    MathSciNet  MATH  Google Scholar 

  5. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    MathSciNet  MATH  Google Scholar 

  6. Huang FJ, LeCun Y (2006) Large-scale learning with svm and convolutional for generic object categorization. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 1, pp. 284–291. IEEE

  7. Schuster M, Paliwal KK (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673–2681

    Google Scholar 

  8. Matsui N, Isokawa T, Kusamichi H, Peper F, Nishimura H (2004) Quaternion neural network with geometrical operators. J Intell Fuzzy Syst 15(3–4):149–164

    MATH  Google Scholar 

  9. Hirose A (2012) Complex-Valued Neural Networks. Studies in Computational Intelligence. Springer. https://books.google.co.in/books?id=0wI8IL8T3p0C

  10. Trabelsi C, Bilaniuk O, Zhang Y, Serdyuk D, Subramanian S, Santos J, Mehri S, Rostamzadeh N, Bengio Y, Pal C (2017) Deep complex networks. arxiv preprint arXiv: 1705:09792

  11. Mandic DP, Goh VSL (2009) Complex valued nonlinear adaptive filters: noncircularity, widely linear and neural models. Wiley, New York

    Google Scholar 

  12. Aizenberg I, Gonzalez A (2018) Image recognition using MLMVN and frequency domain features. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp 1–8. IEEE

  13. Nitta T et al (2006) Three-dimensional vector valued neural network and its generalization ability. Neural Inf Process 10(10):237–242

    Google Scholar 

  14. Gaudet CJ, Maida AS (2018) Deep quaternion networks. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp 1–8. IEEE

  15. Parcollet T, Ravanelli M, Morchid M, Linarès G, Trabelsi C, De Mori R, Bengio, Y (2018) Quaternion recurrent neural networks. arXiv preprint arXiv:1806.04418

  16. Takahashi K, Isaka A, Fudaba T, Hashimoto M (2017) Remarks on quaternion neural network-based controller trained by feedback error learning. In: 2017 IEEE/SICE International Symposium on System Integration (SII), pp 875–880. IEEE

  17. Ogawa T (2016) Neural network inversion for multilayer quaternion neural networks. Comput Technol Appl 7:73–82

    Google Scholar 

  18. Bayro-Corrochano E, Lechuga-Gutiérrez L, Garza-Burgos M (2018) Geometric techniques for robotics and hmi: interpolation and haptics in conformal geometric algebra and control using quaternion spike neural networks. Robot Auton Syst 104:72–84

    Google Scholar 

  19. Arena P, Fortuna L, Occhipinti L, Xibilia MG (1994) Neural networks for quaternion-valued function approximation. In: Proceedings of IEEE International Symposium on Circuits and Systems-ISCAS’94, vol. 6, pp 307–310. IEEE

  20. De Leo S, Rotelli P (1997) Local hypercomplex analyticity. arXiv preprint arXiv:funct-an/9703002

  21. Nitta T (1995) A quaternary version of the back-propagation algorithm. In: Proceedings of ICNN’95-International Conference on Neural Networks, vol. 5, pp 2753–2756. IEEE

  22. http://www.eis.t.u-tokyo.ac.jp/news/NNTC_CVNN/

  23. https://sites.google.com/uniroma1.it/icassp2019-hypercomplex/home

  24. Saoud LS, Ghorbani R, Rahmoune F (2017) Cognitive quaternion valued neural network and some applications. Neurocomputing 221:85–93

    Google Scholar 

  25. Tokuda K, Zen H, Kitamura T (2003) Trajectory modeling based on HMMs with the explicit relationship between static and dynamic features. In: Eighth European Conference on Speech Communication and Technology. Citeseer

  26. Isokawa T, Kusakabe T, Matsui N, Peper F (2003) Quaternion neural network and its application. In: International Conference on Knowledge-based and Intelligent Information and Engineering Systems, pp 318–324. Springer

  27. Hamilton WR (1844) Ii. on quaternions; or on a new system of imaginaries in algebra. Lond Edinb Dublin Philos Mag J Sci 25(163):10–13

    Google Scholar 

  28. Diebel J (2006) Representing attitude: Euler angles, unit quaternions, and rotation vectors. Matrix 58(15–16):1–35

    Google Scholar 

  29. Dam EB, Koch M, Lillholm M (1998) Quaternions, interpolation and animation, vol 2. Citeseer, Princeton

    Google Scholar 

  30. Kumar S, Tripathi BK (2019) On the learning machine with quaternionic domain neural network and its high-dimensional applications. J Intell Fuzzy Syst 36(6):5189–5202

    Google Scholar 

  31. Cariow A, Cariowa G (2020) Fast algorithms for quaternion-valued convolutional neural networks. IEEE Trans Neural Netw Learn Syst 32(1):457–462

    MathSciNet  Google Scholar 

  32. Fortuna L, Muscato G, Xibilia MG (2001) A comparison between HMLP and HRBF for attitude control. IEEE Trans Neural Netw 12(2):318–328

    Google Scholar 

  33. Shoemake K (1985) Animating rotation with quaternion curves. In: Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques, pp 245–254

  34. Pletinckx D (1989) Quaternion calculus as a basic tool in computer graphics. Vis Comput 5(1):2–13

    MATH  Google Scholar 

  35. Yun X, Bachmann ER (2006) Design, implementation, and experimental results of a quaternion-based kalman filter for human body motion tracking. IEEE Trans Rob 22(6):1216–1227

    Google Scholar 

  36. Sangwine SJ (1996) Fourier transforms of colour images using quaternion or hypercomplex, numbers. Electron Lett 32(21):1979–1980

    Google Scholar 

  37. Dornaika F, Horaud R (1998) Simultaneous robot-world and hand-eye calibration. IEEE Trans Robot Autom 14(4):617–622

    Google Scholar 

  38. Chou JC (1992) Quaternion kinematic and dynamic differential equations. IEEE Trans Robot Autom 8(1):53–64

    Google Scholar 

  39. Karney CF (2007) Quaternions in molecular modeling. J Mol Graph Model 25(5):595–604

    Google Scholar 

  40. Vince J (2011) Vince: quaternions for computer graphics. Springer, New York

    MATH  Google Scholar 

  41. Ben-Ari M (2014) A tutorial on euler angles and quaternions, vol 524. Weizmann Institute of Science, Israel

  42. Subakan ÖN, Vemuri BC (2011) A quaternion framework for color image smoothing and segmentation. Int J Comput Vision 91(3):233–250

    MathSciNet  MATH  Google Scholar 

  43. Wang Y, Kou KI, Zou C, Tang YY (2021) Robust sparse representation in quaternion space. IEEE Trans Image Process 30:3637–3649

    MathSciNet  Google Scholar 

  44. Hu C, Meng MQ-h, Mandal M, Liu PX (2006) Robot rotation decomposition using quaternions. In: 2006 International Conference on Mechatronics and Automation, pp 1158–1163. IEEE

  45. Özgür E, Mezouar Y (2016) Kinematic modeling and control of a robot arm using unit dual quaternions. Robot Auton Syst 77:66–73

    Google Scholar 

  46. Condurache D, Martinusi V (2010) Quaternionic exact solution to the relative orbital motion problem. J Guid Control Dyn 33(4):1035–1047

    Google Scholar 

  47. Chelnokov YN (2022) Quaternion methods and models of regular celestial mechanics and astrodynamics. Appl Math Mech 43(1):21–80

    MathSciNet  MATH  Google Scholar 

  48. Mason J, Schuh C (2008) Hyperspherical harmonics for the representation of crystallographic texture. Acta Mater 56(20):6141–6155

    Google Scholar 

  49. Bayro-Corrochano E (2021) A survey on quaternion algebra and geometric algebra applications in engineering and computer science 1995–2020. IEEE Access 9:104326–104355

    Google Scholar 

  50. Nitta T, Garis H (1992) A 3D vector version of the back-propagation algorithm. In: Proceedings of International Joint Conference on Neural Networks, pp 511–516

  51. Tripathi BK, Kalra PK (2011) On the learning machine for three dimensional mapping. Neural Comput Appl 20(1):105–111

    Google Scholar 

  52. Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. JMLR Workshop and Conference Proceedings, pp 249–256

  53. Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: Icml

  54. Arena P, Fortuna L, Re R, Xibilia MG (1993) On the capability of neural networks with complex neurons in complex valued functions approximation. In: 1993 IEEE International Symposium on Circuits and Systems, pp 2168–2171. IEEE

  55. Arena P, Fortuna L, Muscato G, Xibilia MG (1997) Multilayer perceptrons to approximate quaternion valued functions. Neural Netw 10(2):335–342

    Google Scholar 

  56. Isokawa T, Nishimura H, Matsui N (2012) Quaternionic multilayer perceptron with local analyticity. Information 3(4):756–770

    Google Scholar 

  57. Ujang BC, Took CC, Mandic DP (2011) Quaternion-valued nonlinear adaptive filtering. IEEE Trans Neural Netw 22(8):1193–1206

    Google Scholar 

  58. Mandic DP, Jahanchahi C, Took CC (2010) A quaternion gradient operator and its applications. IEEE Signal Process Lett 18(1):47–50

    Google Scholar 

  59. Ujang BC, Jahanchahi C, Took CC, Mandic D (2010) Quaternion valued neural networks and nonlinear adaptive filters. IEEE Trans. Neural Netw

  60. Nitta T (2004) A solution to the 4-bit parity problem with a single quaternary neuron. Neural Inf Process Lett Rev 5(2):33–39

    Google Scholar 

  61. McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5(4):115–133

    MathSciNet  MATH  Google Scholar 

  62. Nitta T (1997) An extension of the back-propagation algorithm to complex numbers. Neural Netw 10(8):1391–1415

    Google Scholar 

  63. Bassey J, Qian L, Li X (2021) A survey of complex-valued neural networks. arXiv preprint arXiv:2101.12249

  64. Suresh S, Sundararajan N, Savitha R (2013) Supervised learning with complex-valued neural networks. Springer, New York

    Google Scholar 

  65. Tripathi BK (2016) High dimensional neurocomputing. Springer, New York

    Google Scholar 

  66. Kumar S, Tripathi BK (2019) Root-power mean aggregation-based neuron in quaternionic domain. IETE J Res 65(4):557–575

    Google Scholar 

  67. Yadav RN, Kumar N, Kalra PK, John J (2006) Learning with generalized-mean neuron model. Neurocomputing 69(16–18):2026–2032

    Google Scholar 

  68. Yadav RN, Kalra PK, John J (2006) Neural network learning with generalized-mean based neuron model. Soft Comput 10(3):257–263

    Google Scholar 

  69. Tripathi BK, Chandra B, Singh M, Kalra PK (2011) Complex generalized-mean neuron model and its applications. Appl Soft Comput 11(1):768–777

    Google Scholar 

  70. Tripathi BK, Kalra PK (2011) On efficient learning machine with root-power mean neuron in complex domain. IEEE Trans Neural Netw 22(5):727–738

    Google Scholar 

  71. Kolmogorov AN, Castelnuovo G (1930) Sur la Notion de la Moyenne. G. Bardi, tip. della R. Accad. dei Lincei, ???

  72. Nagumo M (1930) Über eine klasse der mittelwerte. Jpn J Math 7:71–79

    MATH  Google Scholar 

  73. Cavanaugh JE, Neath AA (2019) The akaike information criterion: background, derivation, properties, application, interpretation, and refinements. Wiley Interdiscip Rev Comput Stat 11(3):1460

    MathSciNet  Google Scholar 

  74. Kumar S, Tripathi BK (2019) On the learning machine with compensatory aggregation based neurons in quaternionic domain. J Comput Design Eng 6(1):33–48

    Google Scholar 

  75. Tripathi BK, Kalra PK (2010) The novel aggregation function-based neuron models in complex domain. Soft Comput 14(10):1069–1081

    MATH  Google Scholar 

  76. Tripathi BK (2017) On the complex domain deep machine learning for face recognition. Appl Intell 47(2):382–396

    Google Scholar 

  77. Lee C-C, Chung P-C, Tsai J-R, Chang C-I (1999) Robust radial basis function neural networks. IEEE Trans Syst Man Cybern Part B 29(6):674–685

    Google Scholar 

  78. Kumar S, Singh RK, Chaudhary A (2022) A novel non-linear neuron model based on multiplicative aggregation in quaternionic domain. Complex Intell Syst 1–23

  79. Kumar S, Singh RK, Chaudhary A (2020) On the learning machine with amplificatory neuron in complex domain. Arab J Sci Eng 45(12):10287–10309

    Google Scholar 

  80. Yoshida M, Kuroe Y, Mori T (2005) Models of hopfield-type quaternion neural networks and their energy functions. Int J Neural Syst 15(01n02):129–135

    Google Scholar 

  81. Buchholz S, Le Bihan N (2006) Optimal separation of polarized signals by quaternionic neural networks. In: 2006 14th European Signal Processing Conference, pp 1–5. IEEE

  82. Parcollet T, Morchid M, Bousquet P-M, Dufour R, Linarès G, De Mori R (2016) Quaternion neural networks for spoken language understanding. In: 2016 IEEE Spoken Language Technology Workshop (SLT), pp 362–368. IEEE

  83. Popa C-A (2018) Learning algorithms for quaternion-valued neural networks. Neural Process Lett 47(3):949–973

    Google Scholar 

  84. De Boer P-T, Kroese DP, Mannor S, Rubinstein RY (2005) A tutorial on the cross-entropy method. Ann Oper Res 134(1):19–67

    MathSciNet  MATH  Google Scholar 

  85. Platt J et al (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv Large Margin Classif 10(3):61–74

    Google Scholar 

  86. Willmott CJ, Matsuura K (2005) Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Res 30(1):79–82

    Google Scholar 

  87. Riedmiller M, Braun H (1993) A direct adaptive method for faster backpropagation learning: The rprop algorithm. In: IEEE International Conference on Neural Networks, pp 586–591. IEEE

  88. Igel C, Hüsken M (2003) Empirical evaluation of the improved rprop learning algorithms. Neurocomputing 50:105–123

    MATH  Google Scholar 

  89. Kumar S, Tripathi BK (2018) High-dimensional information processing through resilient propagation in quaternionic domain. J Ind Inf Integr 11:41–49

    Google Scholar 

  90. Xu D, Jahanchahi C, Took CC, Mandic DP (2015) Enabling quaternion derivatives: the generalized HR calculus. R Soc Open Sci 2(8):150255

    MathSciNet  Google Scholar 

  91. Xu D, Zhang L, Zhang H (2017) Learning algorithms in quaternion neural networks using ghr calculus. Neural Netw World 27(3):271

    Google Scholar 

  92. Singh S, Tripathi B (2022) Pneumonia classification using quaternion deep learning. Multimed Tools Appl 81(2):1743–1764

    Google Scholar 

  93. Granero MA, Hernández CX, Valle ME (2021) Quaternion-valued convolutional neural network applied for acute lymphoblastic leukemia diagnosis. In: Intelligent Systems: 10th Brazilian Conference, BRACIS 2021, Virtual Event, November 29–December 3, 2021, Proceedings, Part II, pp 280–293. Springer

  94. Singh S, Tripathi B, Rawat SS (2023) Deep quaternion convolutional neural networks for breast cancer classification. Multimedia Tools and Applications, 1–24

  95. Shang F, Hirose A (2013) Quaternion neural-network-based PolSAR land classification in poincare-sphere-parameter space. IEEE Trans Geosci Remote Sens 52(9):5693–5703

    Google Scholar 

  96. Muppidi A, Radfar M (2021) Speech emotion recognition using quaternion convolutional neural networks. In: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp 6309–6313. IEEE

  97. Pavllo D, Feichtenhofer C, Auli M, Grangier D (2020) Modeling human motion with quaternion-based neural networks. Int J Comput Vision 128:855–872

    MATH  Google Scholar 

  98. El Alami A, Lakhili Z, Mesbah A, Berrahou A, Qjidaa H (2019) Color face recognition by using quaternion and deep neural networks. In: 2019 International Conference on Wireless Technologies, Embedded and Intelligent Systems (WITS), pp 1–5. IEEE

  99. Yin Q, Wang J, Luo X, Zhai J, Jha SK, Shi Y-Q (2019) Quaternion convolutional neural network for color image classification and forensics. IEEE Access 7:20293–20301

    Google Scholar 

  100. Zhou Y, Jin L, Liu H, Song E (2020) Color facial expression recognition by quaternion convolutional neural network with gabor attention. IEEE Trans Cogn Dev Syst 13(4):969–983

    Google Scholar 

  101. Buchholz S (2000) Quaternionic spinor mlp, 377–382

  102. Parcollet T (2019) Quaternion neural networks. PhD thesis, Université d’Avignon

  103. Kusamichi H, Isokawa T, Matsui N, Ogawa Y, Maeda K (2004) A new scheme for color night vision by quaternion neural network. In: Proceedings of the 2nd International Conference on Autonomous Robots and Agents, vol 1315

  104. Greenblatt A, Mosquera-Lopez C, Agaian S (2013) Quaternion neural networks applied to prostate cancer gleason grading. In: 2013 IEEE International Conference on Systems, Man, and Cybernetics, pp 1144–1149. IEEE

  105. Soulard R, Carré P (2011) Quaternionic wavelets for texture classification. Pattern Recogn Lett 32(13):1669–1678

    Google Scholar 

  106. Takahashi K, Takahashi S, Cui Y, Hashimoto M (2014) Remarks on computational facial expression recognition from HOG features using quaternion multi-layer neural network. In: International Conference on Engineering Applications of Neural Networks, pp 15–24. Springer

  107. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, pp 886–893. IEEE

  108. Jolliffe I, Lovric M (2011) International encyclopedia of statistical science. Principal component analysis. Springer, New York, pp 1094–1096

    Google Scholar 

  109. Kinugawa K, Shang F, Usami N, Hirose A et al (2018) Isotropization of quaternion-neural-network-based polsar adaptive land classification in poincare-sphere parameter space. IEEE Geosci Remote Sens Lett 15(8):1234–1238

    Google Scholar 

  110. Bechet F, Maza B, Bigouroux N, Bazillon T, El-Beze M, De Mori R, Arbillot E (2012) Decoda: a call-centre human-human spoken conversation corpus. In: Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pp 1343–1347

  111. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  112. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp 448–456. PMLR

  113. Parcollet T, Morchid M, Linarès G (2017) Quaternion denoising encoder-decoder for theme identification of telephone conversations. In: Interspeech 2017, pp 3325–3328. ISCA

  114. Vincent P, Larochelle H, Bengio Y, Manzagol P-A (2008) Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, pp 1096–1103

  115. Hinton G, Deng L, Yu D, Dahl GE, Mohamed A-R, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath TN et al (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29(6):82–97

    Google Scholar 

  116. Parcollet T, Morchid M, Linares G (2017) Deep quaternion neural networks for spoken language understanding. In: 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pp 504–511. IEEE

  117. Cui Y, Takahashi K, Hashimoto M (2013) Design of control systems using quaternion neural network and its application to inverse kinematics of robot manipulator. In: Proceedings of the 2013 IEEE/SICE International Symposium on System Integration, pp 527–532. IEEE

  118. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25

  119. Higham NJ (1990) Analysis of the Cholesky decomposition of a semi-definite matrix

  120. Geiger A, Lenz P, Stiller C, Urtasun R (2013) Vision meets robotics: the kitti dataset. Int J Robot Res 32(11):1231–1237

    Google Scholar 

  121. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 770–778

  122. Parcollet T, Zhang Y, Morchid M, Trabelsi C, Linarès G, De Mori R, Bengio Y (2018) Quaternion convolutional neural networks for end-to-end automatic speech recognition. arXiv preprint arXiv:1806.07789

  123. Garofolo JS, Lamel LF, Fisher WM, Fiscus JG, Pallett DS (1993) DARPA TIMIT acoustic-phonetic continous speech corpus CD-ROM. NIST speech disc 1-1.1. NASA STI/Recon technical report n 93, 27403

  124. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1026–1034

  125. Comminiello D, Lella M, Scardapane S, Uncini A (2019) Quaternion convolutional neural networks for detection and localization of 3D sound events. In: ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp 8533–8537. IEEE

  126. Adavanne S, Politis A, Nikunen J, Virtanen T (2018) Sound event localization and detection of overlapping sources using convolutional recurrent neural networks. IEEE J Sel Topics Signal Process 13(1):34–48

    Google Scholar 

  127. Hopfield JJ, Tank DW (1985) Neural computation of decisions in optimization problems. Biol Cybern 52(3):141–152

    MATH  Google Scholar 

  128. Isokawa T, Nishimura H, Kamiura N, Matsui N (2006) Fundamental properties of quaternionic Hopfield neural network. In: The 2006 IEEE International Joint Conference on Neural Network Proceedings, pp 218–223. IEEE

  129. Isokawa T, Nishimura H, Kamiura N, Matsui N (2008) Associative memory in quaternionic hopfield neural network. Int J Neural Syst 18(02):135–145

    Google Scholar 

  130. Kobayashi M (2015) Hybrid quaternionic Hopfield neural network. IEICE Trans Fundam Electron Commun Comput Sci 98(7):1512–1518

    Google Scholar 

  131. Kobayashi M, Nakajima A (2012) Twisted quaternary neural networks. IEEJ Trans Electr Electron Eng 7(4):397–401

    Google Scholar 

  132. Valle ME, Castro FZ (2017) On the dynamics of Hopfield neural networks on unit quaternions. IEEE Trans Neural Netw Learn Syst 29(6):2464–2471

    Google Scholar 

  133. Lin J-S, Cheng K-S, Mao C-W (1996) A fuzzy Hopfield neural network for medical image segmentation. IEEE Trans Nucl Sci 43(4):2389–2398

    Google Scholar 

  134. Pavllo D, Grangier D, Auli M (2018) Quaternet: A quaternion-based recurrent model for human motion. arXiv preprint arXiv:1805.06485

  135. Pascanu R, Mikolov T, Bengio Y (2013) On the difficulty of training recurrent neural networks. In: International Conference on Machine Learning, pp 1310–1318. PMLR

  136. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Google Scholar 

  137. Goodfellow I (2016) Nips 2016 tutorial: Generative adversarial networks. arXiv preprint arXiv:1701.00160

  138. Sfikas G, Giotis AP, Retsinas G, Nikou C (2021) Quaternion generative adversarial networks for inscription detection in byzantine monuments. In: International Conference on Pattern Recognition, pp 171–184. Springer

  139. Zhang H, Zhang Z, Odena A, Lee H (2019) Consistency regularization for generative adversarial networks. arXiv preprint arXiv:1910.12027

  140. Zhou Z, Liang J, Song Y, Yu L, Wang H, Zhang W, Yu Y, Zhang Z (2019) Lipschitz generative adversarial nets. In: International Conference on Machine Learning, pp 7584–7593. PMLR

  141. Miyato T, Kataoka T, Koyama M, Yoshida Y (2018) Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957

  142. Brock A, Donahue J, Simonyan K (2018) Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096

  143. Grassucci E, Cicero E, Comminiello D (2022) Quaternion generative adversarial networks. In: Generative Adversarial Learning: Architectures and Applications, pp 57–86. Springer, New York

  144. Schmidhuber J (2020) Generative adversarial networks are special cases of artificial curiosity (1990) and also closely related to predictability minimization (1991). Neural Netw 127:58–66

    MATH  Google Scholar 

  145. Wu Z, Pan S, Chen F, Long G, Zhang C, Philip SY (2020) A comprehensive survey on graph neural networks. IEEE Trans Neural Netw Learn Syst 32(1):4–24

    MathSciNet  Google Scholar 

  146. Hamilton WL, Ying R, Leskovec J (2017) Representation learning on graphs: Methods and applications. arXiv preprint arXiv:1709.05584

  147. NGUYEN DQ (2021) Representation learning for graph-structured data. PhD thesis, Monash University

  148. Zhang D, Yin J, Zhu X, Zhang C (2018) Network representation learning: a survey. IEEE Trans Big Data 6(1):3–28

    Google Scholar 

  149. Brody S, Alon U, Yahav E (2021) How attentive are graph attention networks? arXiv preprint arXiv:2105.14491

  150. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907

  151. Xu K, Hu W, Leskovec J, Jegelka S (2019) How powerful are graph neural networks? ICLR

  152. Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. Advances in neural information processing systems 30

  153. Nguyen TD, Phung D et al (2021) Quaternion graph neural networks. In: Asian Conference on Machine Learning, pp 236–251. PMLR

  154. Wang C, Li L, Zhang H, Li D (2022) Quaternion-based knowledge graph neural network for social recommendation. Knowledge-Based Systems, 109940

  155. Zhu X, Xu Y, Xu H, Chen C (2018) Quaternion convolutional neural networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 631–647

  156. Tay Y, Zhang A, Tuan LA, Rao J, Zhang S, Wang S, Fu J, Hui SC (2019) Lightweight and efficient neural natural language processing with quaternion networks. arXiv preprint arXiv:1906.04393

  157. Nguyen DQ, Vu T, Nguyen TD, Phung D (2022) Quatre: Relation-aware quaternions for knowledge graph embeddings. In: Companion Proceedings of the Web Conference 2022, pp 189–192

  158. Chen H, Li G, Jiang W, Sun Y (2022) Dynamic dual quaternion knowledge graph embedding. Applied Intelligence, 1–11

  159. Sen P, Namata G, Bilgic M, Getoor L, Galligher B, Eliassi-Rad T (2008) Collective classification in network data. AI Mag 29(3):93–93

    Google Scholar 

  160. Namata G, London B, Getoor L, Huang B, Edu U (2012) Query-driven active surveying for collective classification. In: 10th International Workshop on Mining and Learning with Graphs, vol 8, p 1

  161. Casanova PVGCA, Lio ARP, Bengio Y (2018) Graph attention networks. ICLR

  162. Chami I, Ying Z, Ré C, Leskovec J (2019) Hyperbolic graph convolutional neural networks. Advances in neural information processing systems 32

  163. Pei H, Wei B, Chang KC-C, Lei Y, Yang B (2020) Geom-gcn: Geometric graph convolutional networks. arXiv preprint arXiv:2002.05287

  164. Niepert M, Ahmed M, Kutzkov K (2016) Learning convolutional neural networks for graphs. In: International Conference on Machine Learning, pp 2014–2023. PMLR

  165. Verma S, Zhang Z-L (2018) Graph capsule convolutional neural networks. arXiv preprint arXiv:1805.08090

  166. Zhang M, Cui Z, Neumann M, Chen Y (2018) An end-to-end deep learning architecture for graph classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 32

  167. Xinyi Z, Chen L (2019) Capsule graph neural network. In: International Conference on Learning Representations

  168. Haggai M, Heli B-H, Nadav S, Yaron L (2019) Invariant and equivariant graph networks. In: The International Conference on Learning Representations (ICLR’19)

  169. Maron H, Ben-Hamu H, Serviansky H, Lipman Y (2019) Provably powerful graph networks. Advances in neural information processing systems 32

  170. Chen T, Bian S, Sun Y (2019) Are powerful graph neural nets necessary? a dissection on graph classification. arXiv preprint arXiv:1905.04579

  171. Yanardag P, Vishwanathan S (2015) Deep graph kernels. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 1365–1374

  172. Tombe R, Viriri S (2022) Effective processing of convolutional neural networks for computer vision: a tutorial and survey. IETE Tech Rev 39(1):49–62

    Google Scholar 

  173. Li Z, Liu F, Yang W, Peng S, Zhou J (2021) A survey of convolutional neural networks: analysis, applications, and prospects. IEEE Transactions on Neural Networks and Learning Systems

  174. Khan A, Sohail A, Zahoora U, Qureshi AS (2020) A survey of the recent architectures of deep convolutional neural networks. Artif Intell Rev 53(8):5455–5516

    Google Scholar 

  175. Li Y, Hao Z, Lei H (2016) Survey of convolutional neural network. J Comput Appl 36(9):2508

    Google Scholar 

  176. Hinton GE, Krizhevsky A, Wang SD (2011) Transforming auto-encoders. In: International Conference on Artificial Neural Networks, pp 44–51. Springer

  177. Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. Advances in neural information processing systems 30

  178. Hinton GE, Sabour S, Frosst N (2018) Matrix capsules with em routing. In: International Conference on Learning Representations

  179. LaLonde R, Xu Z, Irmakci I, Jain S, Bagci U (2021) Capsules for biomedical image segmentation. Med Image Anal 68:101889

    Google Scholar 

  180. Saif A, Imtiaz T, Rifat S, Shahnaz C, Zhu W-P, Ahmad MO (2021) Capscovnet: a modified capsule network to diagnose covid-19 from multimodal medical imaging. IEEE Trans Artif Intell 2(6):608–617

    Google Scholar 

  181. Venkatanaresh M, Kullayamma I (2022) A new approach for crop type mapping in satellite images using hybrid deep capsule auto encoder. Knowl-Based Syst 256:109881

    Google Scholar 

  182. Gu J, Tresp V, Hu H (2021) Capsule network is not more robust than convolutional network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 14309–14317

  183. Byerly A, Kalganova T, Dear I (2021) No routing needed between capsules. Neurocomputing 463:545–553

    Google Scholar 

  184. Zhou H, Zhang C, Zhang X, Ma Q (2022) Image classification based on quaternion-valued capsule network. Applied Intelligence, 1–20

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, KIET Group of Institutions, Delhi-NCR, Ghaziabad, Uttar Pradesh, 201206, India

    Sushil Kumar & Umang Rastogi

Authors
  1. Sushil Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Umang Rastogi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sushil Kumar.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare. All authors contributed equally to this review paper and agreed to the submission. No financial interest have reported.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, S., Rastogi, U. A Comprehensive Review on the Advancement of High-Dimensional Neural Networks in Quaternionic Domain with Relevant Applications. Arch Computat Methods Eng 30, 3941–3968 (2023). https://doi.org/10.1007/s11831-023-09925-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11831-023-09925-w

Search

Navigation