Skip to main content
SpringerLink
Log in

An efficient multilayer RBF neural network and its application to regression problems

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

By combining multilayer perceptrons (MLPs) and radial basis function neural networks (RBF-NNs), an efficient multilayer RBF network is proposed in this work for regression problems. As an extension to the existing multilayer RBF network (RBF-MLP-I), the new multilayer RBF network (RBF-MLP-II) first nonlinearly transforms the multi-dimensional input data by adopting a set of multivariate basis functions. Then, linear weighted sums of these basis functions, i.e., the RBF approximations, are computed in the first hidden layer and used as the features of this layer. Subsequently, in the following hidden layers, each feature of the preceding hidden layer is fed into a univariate RBF characterized by the trainable scalar center and width, and then, RBF approximations are also applied to these basis functions. Finally, the features of the last hidden layer are linearly transformed to approximate the target output data. RBF-MLP-II reduces the number of parameters in basis functions and thus the network complexity of RBF-MLP-I. Verified by four regression problems, it is demonstrated that the proposed RBF-MLP-II exhibits the best approximation accuracy and fastest training convergence compared to conventional MLPs, RBF-NNs, and RBF-MLP-I.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Draper NR, Smith H (1998) Applied regression analysis. John Wiley & Sons, Hoboken

    Book  Google Scholar 

  2. Weisberg S (2005) Applied linear regression. John Wiley & Sons, Hoboken

    Book  Google Scholar 

  3. Bates DM, Watts DG (1988) Nonlinear regression analysis and its applications. Wiley, New York

    Book  Google Scholar 

  4. Montgomery DC, Peck EA, Vining GG (2012) Introduction to linear regression analysis. John Wiley & Sons, Hoboken

    MATH  Google Scholar 

  5. Theil H (1992) A rank-invariant method of linear and polynomial regression analysis. Springer, Berlin, pp 345–381

    Google Scholar 

  6. Hosmer DW Jr, Lemeshow S, Sturdivant RX (2013) Applied logistic regression. John Wiley & Sons, Hoboken

    Book  Google Scholar 

  7. Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12:55–67

    Article  Google Scholar 

  8. Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B 58:267–288

    MathSciNet  MATH  Google Scholar 

  9. Box GEP, Tiao GC (2011) Bayesian inference in statistical analysis. John Wiley & Sons, Hoboken

    MATH  Google Scholar 

  10. Goodfellow I, Bengio Y, Courville A, Bengio Y (2016) Deep learning. MIT Press, Cambridge

    MATH  Google Scholar 

  11. Pal SK, Mitra S (1992) Multilayer perceptron, fuzzy sets, and classification. IEEE Trans Neural Netw 3:683–697

    Article  Google Scholar 

  12. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444

    Article  Google Scholar 

  13. Gardner MW, Dorling SR (1998) Artificial neural networks (the multilayer perceptron)—a review of applications in the atmospheric sciences. Atmos Environ 32:2627–2636

    Article  Google Scholar 

  14. Kůrková V (1992) Kolmogorov’s theorem and multilayer neural networks. Neural Netw 5:501–506. https://doi.org/10.1016/0893-6080(92)90012-8

    Article  Google Scholar 

  15. Chen T, Chen H (1995) Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Trans Neural Netw 6:911–917

    Article  Google Scholar 

  16. Chaudhuri BB, Bhattacharya U (2000) Efficient training and improved performance of multilayer perceptron in pattern classification. Neurocomputing 34:11–27

    Article  Google Scholar 

  17. Mazroua AA, Salama MMA, Bartnikas R (1993) PD pattern recognition with neural networks using the multilayer perceptron technique. IEEE Trans Electr Insul 28:1082–1089

    Article  Google Scholar 

  18. Ahmed NK, Atiya AF, El GN, El-Shishiny H (2010) An empirical comparison of machine learning models for time series forecasting. Econom Rev 29:594–621

    Article  MathSciNet  Google Scholar 

  19. Koskela T, Lehtokangas M, Saarinen J, Kaski K (1996) Time series prediction with multilayer perceptron, FIR and Elman neural networks. In: Proceedings of the world congress on neural networks. Citeseer, pp 491–496

  20. Shedeed HA, Issa MF, El-Sayed SM (2013) Brain EEG signal processing for controlling a robotic arm. In: Proceedings—2013 8th international conference on computer engineering and systems, ICCES 2013. IEEE, pp 152–157

  21. Nielsen JLG, Holmgaard S, Jiang N et al (2009) Enhanced EMG signal processing for simultaneous and proportional myoelectric control. In: Proceedings of the 31st annual international conference of the IEEE engineering in medicine and biology society: engineering the future of biomedicine, EMBC 2009. IEEE, pp 4335–4338

  22. Sezer OB, Ozbayoglu AM, Dogdu E (2017) An artificial neural network-based stock trading system using technical analysis and big data framework. In: arXiv, pp 223–226

  23. Moon T, Hong S, Choi HY et al (2019) Interpolation of greenhouse environment data using multilayer perceptron. Comput Electron Agric 166:105023

    Article  Google Scholar 

  24. Orr MJL (1996) Introduction to radial basis function networks. University of Edinburgh, Edinburgh, pp 1–67

    Google Scholar 

  25. Broomhead DS, Lowe D (1988) Radial basis functions, multi-variable functional interpolation and adaptive networks

  26. Walczak B, Massart DL (1996) The radial basis functions—partial least squares approach as a flexible non-linear regression technique. Anal Chim Acta 331:177–185

    Article  Google Scholar 

  27. Shu C, Ding H, Yeo KS (2003) Local radial basis function-based differential quadrature method and its application to solve two-dimensional incompressible Navier–Stokes equations. Comput Methods Appl Mech Eng 192:941–954

    Article  Google Scholar 

  28. Shu C, Ding H, Yeo KS (2004) Solution of partial differential equations by a global radial basis function-based differential quadrature method. Eng Anal Bound Elem 28:1217–1226

    Article  Google Scholar 

  29. Ding H, Shu C, Yeo KS, Lu ZL (2005) Simulation of natural convection in eccentric annuli between a square outer cylinder and a circular inner cylinder using local MQ-DQ method. Numer Heat Transf Part A Appl 47:291–313

    Article  Google Scholar 

  30. Harpham C, Dawson CW (2006) The effect of different basis functions on a radial basis function network for time series prediction: a comparative study. Neurocomputing 69:2161–2170

    Article  Google Scholar 

  31. Schwenker F, Kestler HA, Palm G (2001) Three learning phases for radial-basis-function networks. Neural Netw 14:439–458. https://doi.org/10.1016/S0893-6080(01)00027-2

    Article  MATH  Google Scholar 

  32. Na S, Xumin L, Yong G (2010) Research on k-means clustering algorithm: an improved k-means clustering algorithm. In: IEEE, pp 63–67

  33. Broomhead DS, Lowe D (1988) Multivariable functional interpolation and adaptive networks, complex systems, vol 2

  34. Chen F-C (1990) Back-propagation neural networks for nonlinear self-tuning adaptive control. IEEE Control Syst Mag 10:44–48

    Article  Google Scholar 

  35. Hecht-Nielsen R (1992) Theory of the backpropagation neural network. Elsevier, New York, pp 65–93

    Google Scholar 

  36. Sibi P, Jones SA, Siddarth P (2013) Analysis of different activation functions using back propagation neural networks. J Theor Appl Inf Technol 47:1264–1268

    Google Scholar 

  37. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv14126980

  38. Chao J, Hoshino M, Kitamura T, Masuda T (2001) A multilayer RBF network and its supervised learning. In: Proceedings of the international joint conference on neural networks. IEEE, pp 1995–2000

  39. Mhaskar H, Liao Q, Poggio T (2017) When and why are deep networks better than shallow ones? In: Proceedings of the AAAI conference on artificial intelligence

  40. Bottou L (2010) Large-scale machine learning with stochastic gradient descent. In: Proceedings of COMPSTAT’2010. Springer, pp 177–186

  41. Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In: Journal of Machine Learning Research, pp 249–256

  42. Chollet F (2018) Deep Learning mit Python und Keras: Das Praxis-Handbuch vom Entwickler der Keras-Bibliothek. MITP-Verlags GmbH & Co. KG

  43. Abadi M, Agarwal A, Barham P et al (2016) TensorFlow: large-scale machine learning on heterogeneous distributed systems

  44. Jagtap AD, Kawaguchi K, Karniadakis GE (2020) Adaptive activation functions accelerate convergence in deep and physics-informed neural networks. J Comput Phys 404:109136

    Article  MathSciNet  Google Scholar 

  45. Jang J-S (1993) ANFIS: adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 23:665–685

    Article  Google Scholar 

  46. Chen Y, Yang B, Dong J (2006) Time-series prediction using a local linear wavelet neural network. Neurocomputing 69:449–465

    Article  Google Scholar 

  47. De Vito S, Massera E, Piga M et al (2008) On field calibration of an electronic nose for benzene estimation in an urban pollution monitoring scenario. Sens Actuators B Chem 129:750–757

    Article  Google Scholar 

  48. http://archive.ics.uci.edu/ml/datasets/Air+Quality

Download references

Acknowledgements

This work was partially supported by the research grant of the National University of Singapore (NUS), Ministry of Education. In addition, the first author is thankful to NUS for ring-fenced scholarship.

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, National University of Singapore, 10 Kent Ridge Crescent, Singapore, 117576, Singapore

    Qinghua Jiang, Lailai Zhu, Chang Shu & Vinothkumar Sekar

Authors
  1. Qinghua Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lailai Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chang Shu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vinothkumar Sekar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chang Shu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest to this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, Q., Zhu, L., Shu, C. et al. An efficient multilayer RBF neural network and its application to regression problems. Neural Comput & Applic 34, 4133–4150 (2022). https://doi.org/10.1007/s00521-021-06373-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06373-0

Keywords

Search

Navigation