Skip to main content
SpringerLink
Log in

Griffiths’ Variable Learning Rate Online Sequential Learning Algorithm for Feed-Forward Neural Networks

  • Published:
Automatic Control and Computer Sciences Aims and scope Submit manuscript

Abstract

For online sequential training of deep neural networks, where the training data set is chaotic in nature, it becomes quite challenging for choosing a proper learning rate. This paper presents Griffiths’ variable learning rate algorithm for improved performance of online sequential learning of feed-forward neural networks used for chaotic time-series prediction. Here, the learning rate is varied based on Griffiths’ cross-correlation between input training data and squared error, which facilitates better tracking of time-series data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.

Similar content being viewed by others

REFERENCES

  1. Haykin, S., Neural Networks: A Comprehensive Foundation, Upper Saddle River, N.J.: Prentice-Hall, 1999.

    MATH  Google Scholar 

  2. Widrow, B., Winter, R.G., and Baxter, R.A., Layered neural nets for pattern recognition, IEEE Trans. Acoust., Speech, Signal Process., 1988, vol. 36, no. 7, pp. 1109–1118.  https://doi.org/10.1109/29.1638

    Article  MATH  Google Scholar 

  3. Cheng, B. and Titterington, D.M., Neural networks: A review from a statistical perspective, Stat. Sci., 1994, vol. 9, no. 1, pp. 2–30.  https://doi.org/10.1214/ss/1177010638

    Article  MathSciNet  MATH  Google Scholar 

  4. Wang, J., Belatreche, A., Maguire, and McGinnity, M., Online versus offline learning for spiking neural networks: A review and new strategies, IEEE 9th Int. Conf. on Cybernetic Intelligent Systems, Reading, 2010, IEEE, 2010, pp. 1–6.  https://doi.org/10.1109/UKRICIS.2010.5898113

  5. Huang, G.-B., Chen, Y.-Q., and Babri, H.A., Classification ability of single hidden layer feedforward neural networks, IEEE Trans. Neural Networks, 2000, vol. 11, no. 3, pp. 799–801.  https://doi.org/10.1109/72.846750

    Article  Google Scholar 

  6. Smale, S. and Yao, Y., Online learning algorithms, Found. Comput. Math., 2006, vol. 6, no. 2, pp. 145–170.  https://doi.org/10.1007/s10208-004-0160-z

    Article  MathSciNet  MATH  Google Scholar 

  7. Bharath, Y.K., Veena, S., Nagalakshmi, K.V., Darshan, M., and Nagapadma, R., Development of robust VAD schemes for Voice Operated Switch application in aircrafts: Comparison of real-time VAD schemes which are based on Linear Energy-based Detector, Fuzzy Logic and Artificial Neural Networks, 2nd Int. Conf. on Applied and Theoretical Computing and Communication Technology (iCATccT), Bangalore, India, 2016, IEEE, 2016, pp. 191–195.  https://doi.org/10.1109/ICATCCT.2016.7911990

  8. Amrutha, K.N., Bharath, Y.K., and Jayanthi, J., Aircraft engine fuel flow parameter prediction and health monitoring system, 4th Int. Conf. on Recent Trends on Electronics, Information, Communication & Technology (RTEICT), Bangalore, India, 2019, IEEE, 2019, pp. 39–44.  https://doi.org/10.1109/RTEICT46194.2019.9016703

  9. Arshad, M.A. Abido, M.A., An overview of sequential learning algorithms for single hidden layer networks: Current issues & future trends, IEEE Trans. Artif. Intell., 2020.  https://doi.org/10.36227/techrxiv.12253340

  10. Lecun, Y.A., Bottou, L., Orr, G.B., and Müller, K.-R., Efficient backprop, Neural Networks: Tricks of the Trade, Montavon, G., Orr, G.B., Müller, K.R., Eds., Lecture Notes in Computer Science, vol. 7700, Berlin: Springer, 2012, pp. 9–48.  https://doi.org/10.1007/978-3-642-35289-8_3

    Book  Google Scholar 

  11. Senior, A., Heigold, G., Ranzato, M., and Yang, K., An empirical study of learning rates in deep neural networks for speech recognition, IEEE Int. Conf. on Acoustics, Speech and Signal Processing, Vancouver, 2013, IEEE, 2013, pp. 6724–6728.  https://doi.org/10.1109/ICASSP.2013.6638963

  12. Narasimhan, S.V., Veena, S., and Lokesha, H., Variable step-size Griffiths’ algorithm for improved performance of feedforward/feedback active noise control, Signal, Image Video Process., 2010, vol. 4, no. 3, pp. 309–317.  https://doi.org/10.1007/s11760-009-0120-9

    Article  MATH  Google Scholar 

  13. Bharath, Y.K., Chitralekha, G., Veena, S., Lokesha, H., Nagalakshmi, K.V., and Dilna, U., Robust active noise control system for fighter aircraft pilot helmet application, Procedia Comput. Sci., 2016, vol. 89, pp. 690–699. https://doi.org/10.1016/j.procs.2016.06.037

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Malnad College of Engineering, Salagame Road, Hassan, Affiliated to Visvesvaraya Technological University, 573202, Belagavi Karnataka, India

    Y. K. Bharath

Authors
  1. Y. K. Bharath
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Y. K. Bharath.

Ethics declarations

The author declares that he has no conflicts of interest.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bharath, Y.K. Griffiths’ Variable Learning Rate Online Sequential Learning Algorithm for Feed-Forward Neural Networks. Aut. Control Comp. Sci. 56, 160–165 (2022). https://doi.org/10.3103/S0146411622020031

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.3103/S0146411622020031

Keywords:

Search

Navigation