Skip to main content
SpringerLink
Log in

Critical Echo State Networks that Anticipate Input Using Morphable Transfer Functions

  • Conference paper
  • First Online:
Advances in Neural Networks - ISNN 2017 (ISNN 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10261))

Included in the following conference series:

Abstract

The paper investigates a new type of truly critical echo state networks where individual transfer functions for every neuron can be modified to anticipate the expected next input. Deviations from expected input are only forgotten slowly in power law fashion. The paper outlines the theory, numerically analyzes a one neuron model network and finally discusses technical and also biological implications of this type of approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Lukoševičius, M., Jäger, H.: Reservoir computing approaches to recurrent neural network training. Comput. Sci. Rev. 3(3), 127–149 (2009)

    Article  MATH  Google Scholar 

  2. Schrauwen, B., Verstraeten, D., Van Campenhout, J.: An overview of reservoir computing: theory, applications and implementations. In: Proceedings of the 15th European Symposium on Artificial Neural Networks. Citeseer (2007)

    Google Scholar 

  3. Jäger, H.: The “echo state” approach to analysing and training recurrent neural networks - with an erratum note. In: GMD Report 148, GMD German National Research Insitute for Computer Science (2010). http://www.gmd.de/People/Herbert.Jaeger/Publications.html

  4. Jäger, H.: Adaptive nonlinear system identification with echo state networks. In: Proceedings of NIPS 2002, AA14 (2003)

    Google Scholar 

  5. Jäger, H., Maass, W., Principe, J.: Special issue on echo state networks and liquid state machines. Neural Netw. 20(3), 287–289 (2007)

    Article  Google Scholar 

  6. Natschläger, T., Bertschinger, N., Legenstein, R.: At the edge of chaos: real-time computations and self-organized criticality in recurrent neural networks. In: Advances in Neural Information Processing Systems, vol. 17 (2005)

    Google Scholar 

  7. Hajnal, M.A., Lörincz, A.: Critical echo state networks. In: Kollias, S.D., Stafylopatis, A., Duch, W., Oja, E. (eds.) ICANN 2006. LNCS, vol. 4131, pp. 658–667. Springer, Heidelberg (2006). doi:10.1007/11840817_69

    Chapter  Google Scholar 

  8. Boedecker, J., Obst, O., Lizier, J., Mayer, N., Asada, M.: Information processing in echo state networks at the edge of chaos. Theory Biosci. 131, 205–213 (2012)

    Article  Google Scholar 

  9. Beggs, J., Plenz, D.: Neuronal avalanches in neocortical curcuits. J. Neurosci. 24(22), 5216–5229 (2004)

    Article  Google Scholar 

  10. Mayer, N.M.: Adaptive critical reservoirs with power law forgetting of unexpected input events. Neural Comput. 27, 1102–1119 (2015)

    Article  Google Scholar 

  11. Mayer, N.M.: Critical echo state networks that anticipate input using adaptive transfer functions (2016). http://arxiv.org/abs/1606.03674

  12. Manjunath, G., Jaeger, H.: Echo state property linked to an input: exploring a fundamental characteristic of recurrent neural networks. Neural Comput. 25(3), 671–696 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  13. Wainrib, G., Galtier, M.N.: A local echo state property through the largest lyapunov exponent. Neural Netw. 76, 39–45 (2016)

    Article  Google Scholar 

  14. Mayer, N.M., Browne, M.: Self-prediction in echo state networks. In: Proceedings of the First International Workshop on Biological Inspired Approaches to Advanced Information Technology (BioAdIt2004), Lausanne (2004)

    Google Scholar 

  15. Mayer, N.M., Asada, M.: Is self-prediction a useful paradigm for echo state networks that are driven by robotic sensory input? In: 20th Neural Information Processing Systems Conference (NIPS 2006): Workshop on Echo State Networks and Liquid State Machines, H. Jaeger, W. Maass, Jose C. Principe (Organisers), December 2006

    Google Scholar 

  16. Wang, L., Chen, X., Li, S., Cai, X.: General adaptive transfer functions design for volume rendering by using neural networks. In: King, I., Wang, J., Chan, L.-W., Wang, D.L. (eds.) ICONIP 2006. LNCS, vol. 4233, pp. 661–670. Springer, Heidelberg (2006). doi:10.1007/11893257_74

    Chapter  Google Scholar 

Download references

Acknowledgements

This manuscript has been posted at arxiv.org. The authors thanks MOST of Taiwan for financial support and O. Obst for all his help.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering and AIM-HI, National Chung Cheng University, Min-Hsiung, Chia-Yi, Taiwan

    Norbert Michael Mayer

Authors
  1. Norbert Michael Mayer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Norbert Michael Mayer .

Editor information

Editors and Affiliations

  1. Dalian University of Technology, Dalian, China

    Fengyu Cong

  2. City University of Hong Kong, Kowloon Tong, Hong Kong

    Andrew Leung

  3. Chinese Academy of Sciences, Beijing, China

    Qinglai Wei

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Mayer, N.M. (2017). Critical Echo State Networks that Anticipate Input Using Morphable Transfer Functions. In: Cong, F., Leung, A., Wei, Q. (eds) Advances in Neural Networks - ISNN 2017. ISNN 2017. Lecture Notes in Computer Science(), vol 10261. Springer, Cham. https://doi.org/10.1007/978-3-319-59072-1_49

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-59072-1_49

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-59071-4

  • Online ISBN: 978-3-319-59072-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation