Advertisement

Neural Computing and Applications

, Volume 31, Issue 12, pp 8681–8692 | Cite as

A random-weighted plane-Gaussian artificial neural network

  • Xubing YangEmail author
  • Hongxin Yang
  • Fuquan Zhang
  • Xijian Fan
  • Qiaolin Ye
  • Zhe Feng
Original Article
  • 76 Downloads

Abstract

Multilayer perceptron (MLP) and radial basis function network (RBFN) have received considerable attentions in data classification and regression. As a bridge between MLP and RBFN, plane-Gaussian (PG) network is capable of exhibiting globality and locality simultaneously by so-called PG activation function. Due to tuning network weight values by back propagation or clustering method in the training phase, they all confront with slow convergence rate, time-consuming, and easily dropping in local minima. To speed training networks, random projection technologies, for instance, extreme learning machine (ELM), have brightened up in recent decades. In this paper, we propose a random-weighted PG network, termed as RwPG. Instead of plane clustering in PG network, our RwPG adopts random values as network weight, and then analytically calculates network output by matrix inversion. Compared to PG and ELM, the advantages of the proposed RwPG list in fourfold: (1) It will be proved that the RwPG is also a universal approximator. (2) It inherits the geometrical interpretation of PG network, and is also suitable for capturing linearity in data, especially for plane distribution cases. (3) It has comparable training speed for ELM, but significantly faster than that of PG network. (4) Owing to random-weighted technology, RwPG is probably capable of breaking through local extremum problems. Finally, experiments on artificial and benchmark datasets will show its superiorities.

Keywords

Matrix-generalized inverse Plane-Gaussian artificial neural network Random weight 

Notes

Acknowledgements

We would thank the anonymous editors and reviewers for their valuable comments and suggestions. We would thank Dr. Liyong Fu, the professor of Chinese Academy of Forestry, for his academic advice about deep networks in our revisions. This research was supported in part by the Central Public-interest Scientific Institution Basal Research Fund (Grant No. CAFYBB2019QD003), Natural Science Foundation of China under Grant 31670554 and 61871444, the Jiangsu Science Foundation under Grant BK20161527 and BK20171453, and Postgraduate Research and Practice Innovation Program of Jiangsu Province (SJKY19_0907).

Author contribution

XY proposed learning method and wrote manuscript. HY and ZF designed experiments. XF, FZ, and QY analyzed experimental results and gave some advice for manuscript.

Compliance with ethical standards

Conflict of interest

The authors declared that they have no conflicts of interest to this work.

References

  1. 1.
    Lu K, An X, Li J et al (2017) Efficient deep network for vision-based object detection in robotic applications. Neurocomputing 245:31–45CrossRefGoogle Scholar
  2. 2.
    Cox DD, Dean T (2014) Neural networks and neuroscience-inspired computer vision. Curr Biol 24(18):921–929CrossRefGoogle Scholar
  3. 3.
    Siniscalchi SM, Svendsen T, Lee C (2014) An artificial neural network approach to automatic speech processing. Neurocomputing 140(22):326–338CrossRefGoogle Scholar
  4. 4.
    Wu Y, Schuster M, Chen Z et al (2016) Google’s neural machine translation system: bridging the gap between human and machine translation. CoRR. Technical report, available at http://arxiv.org/abs/1609.08144
  5. 5.
    Varshney D, kumar S, Gupta V (2017) Predicting information diffusion probabilities in social networks: a Bayesian networks based approach. Knowl-Based Syst 133:66–76CrossRefGoogle Scholar
  6. 6.
    Mishra J, Anguera JA, Gazzaley A (2016) Video games for neuro-cognitive optimization. Neuron 90(2):214–218CrossRefGoogle Scholar
  7. 7.
    Yang X, Chen S, Chen B (2012) Plane-Gaussian artificial neural network. Neural Comput Appl 21(2):305–317CrossRefGoogle Scholar
  8. 8.
    Bradley PS, Mangasarian OL (2000) k-Plane Clustering. J Glob Optim 16(1):23–32MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Bengio Y. (2012) Practical recommendations for gradient-based training of deep architectures. In: LNCS neural networks: tricks of the trade 2nd ed. Springer, Berlin, pp 437–478Google Scholar
  10. 10.
    Bengio Y, Courville A, Vincent P (2012) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35(8):1798–1828CrossRefGoogle Scholar
  11. 11.
    Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. In: Proceeding of the 27th international conference on machine learning (ICML). Omnipress, Haifa, Israel, Wahpeton, ND, USA, June 21–24, pp 807–814Google Scholar
  12. 12.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. NIPS 25:1097–1105Google Scholar
  13. 13.
    Srivastava RK, Greff K, Schmidhuber J (2015) Training very deep networks. NIPS 28:2377–2385Google Scholar
  14. 14.
    He KM, Zhang XY, Ren SQ et al (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR). IEEE Press, Las Vegas, NV, pp 770–778.  https://doi.org/10.1109/CVPR.2016.90
  15. 15.
    He KM, Zhang XY, Ren SQ et al (2016) Identity mapping in deep residual networks. In: Proceedings of European conference on computer vision (ECCV), Amsterdam, Netherlands, October 8–16, pp 630–645. arXiv:1603.05027v3
  16. 16.
    Guo P, Zhou XL, Wang K (2018) PILAE: a non-gradient descent learning scheme for deep feedforward neural networks. arXiv:1811.01545
  17. 17.
    Pang S, Yang X (2016) Deep convolutional extreme learning machine and its application in handwritten digit classification. Comput Intell Neurosci.  https://doi.org/10.1155/2016/3049632 CrossRefGoogle Scholar
  18. 18.
    Michel M, Abel G, Wellington P (2019) Deep convolutional extreme learning machines: filters combination and error model validation. Neurocomputing 329:359–369CrossRefGoogle Scholar
  19. 19.
    Tissera M, McDonnell M (2016) Deep extreme learning machines: supervised autoencoding architecture for classification. Neurocomputing 174(Part A):42–49CrossRefGoogle Scholar
  20. 20.
    Duan M, Li K, Yang C et al (2018) A hybrid deep learning CNN–ELM for age and gender classification. Neurocompuing 275:448–461CrossRefGoogle Scholar
  21. 21.
    Li J, Zhao X, Li Y et al (2018) Classification of hyperspectral imagery using a new fully convolutional neural network. IEEE Geosci Remote Sens Lett 99:1–5Google Scholar
  22. 22.
    Cao W, Wang X, Ming Z et al (2018) A review on neural networks with random weights. Neurocomputing 275:278–287CrossRefGoogle Scholar
  23. 23.
    Schmidt W, Kraaijveld M, Duin R (1992) Feedforward neural networks with random weights. In: Proceedings of 11th IAPR international conference on pattern recognition methodology and systems, vol 2, pp 1–4Google Scholar
  24. 24.
    Deng C, Huang G, Xu J, Tang J (2015) Extreme learning machines: new trends and applications. Sci China Inf Sci 58(2):1–16CrossRefGoogle Scholar
  25. 25.
    Huang G, Chen L, Siew C (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892CrossRefGoogle Scholar
  26. 26.
    Huang G (2015) What are extreme learning machines? Filling the gap between Frank Rosenblatt’s dream and John von Neumann’s puzzle. Cognit Comput 7:263–278CrossRefGoogle Scholar
  27. 27.
    Igelnik B, Pao Y-H (1995) Stochastic choice of basis functions in adaptive function approximation and the functional-link net. IEEE Trans Neural Netw 6(6):1320–1329CrossRefGoogle Scholar
  28. 28.
    Li JY, Chow W, Igenik B, Pao YH (1997) Comments on “Stochastic choice of basis functions in adaptive function approximation and the functional-link net”. IEEE Trans Neural Netw 8(2):452–454Google Scholar
  29. 29.
    Kasun L, Zhou H, Huang G-B, Vong CM (2013) Representational learning with extreme learning machine for big data. IEEE Intell Syst 28(6):31–34Google Scholar
  30. 30.
    Huang G, Zhou H, Ding X et al (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B 42(2):513–529CrossRefGoogle Scholar
  31. 31.
    Dua D, Taniskidou EK (2017) UCI machine learning repository (http://archive.ics.uci.edu/ml). University of California, School of Information and Computer Science, Irvine
  32. 32.
    Moore AW, Crogan ML (2005) Discriminators for use in flow-based classification. Research reports: RR-05-13, Department of Computer Science, Queen Mary, University of LondonGoogle Scholar
  33. 33.
    Mygdalis V, Iosifidis A, Tefas A et al (2018) Semi-supervised subclass support vector data description for image and video classification. Neurocomputing 278:51–61CrossRefGoogle Scholar
  34. 34.
    Maronidis A, Tefas A, Pitas I (2015) Subclass graph embedding and a marginal fisher analysis paradigm. Pattern Recognit 48(12):4024–4035zbMATHCrossRefGoogle Scholar
  35. 35.
    Wan H, Wang H, Guo G et al (2018) Seperability-oriented subclass discriminant analysis. IEEE Trans Pattern Anal Mach Intell 40(2):409–422CrossRefGoogle Scholar
  36. 36.
    Baum EB, Haussler D (2014) What size net gives valid generalization? Neural Comput 1(1):151–160CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.College of Information Science and TechnologyNanjing Forestry UniversityNanjingPeople’s Republic of China

Personalised recommendations