Skip to main content

Nonlinear-Dynamical Attention Allocation via Information Geometry

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNAI,volume 6830)

Abstract

Inspired by a broader perspective viewing intelligent system dynamics in terms of the geometry of “cognitive spaces,” we conduct a preliminary investigation of the application of information-geometry based learning to ECAN (Economic Attention Networks), the component of the integrative OpenCog AGI system concerned with attention allocation and credit assignment. We generalize Amari’s “natural gradient” algorithm for network learning to encompass ECAN and other recurrent networks, and apply it to small example cases of ECAN, demonstrating a dramatic improvement in the effectiveness of attention allocation compared to prior (Hebbian learning like) ECAN methods. Scaling up the method to deal with realistically-sized ECAN networks as used in OpenCog remains for the future, but should be achievable using sparse matrix methods on GPUs.

Keywords

  • information geometry
  • recurrent networks
  • economic attention allocation
  • ECAN
  • OpenCog

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-22887-2_7
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   64.99
Price excludes VAT (USA)
  • ISBN: 978-3-642-22887-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   84.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Amari, S.: Differential-geometrical methods in statistics. Lecture notes in statistics (1985)

    Google Scholar 

  2. Amari, S.: Natural gradient works efficiently in learning. Neural Computing 10, 251–276 (1998)

    CrossRef  Google Scholar 

  3. Amari, S.: i., Nagaoka, H.: Methods of information geometry. In: AMS (2000)

    Google Scholar 

  4. Arel, I., Rose, D., Coop, R.: Destin: A scalable deep learning architecture with application to high-dimensional robust pattern recognition. In: Proc. AAAI Workshop on Biologically Inspired Cognitive Architectures (2009)

    Google Scholar 

  5. Baskaran, M., Bordawekar, R.: Optimizing Sparse Matrix-Vector Multiplication on GPUs. IBM Research Report (2008)

    Google Scholar 

  6. Dabak, A.: A Geometry for Detection Theory. PhD Thesis, Rice U (1999)

    Google Scholar 

  7. Fauconnier, G., Turner, M.: The Way We Think: Conceptual Blending and the Minds Hidden Complexities. Basic (2002)

    Google Scholar 

  8. Frieden, R.: Physics from Fisher Information. Cambridge U. Press, New York (1998)

    CrossRef  Google Scholar 

  9. Garland, M.: Sparse matrix computations on manycore gpus. In: 45th Annual Design Automation Conference: 2008, pp. 2–6 (2008)

    Google Scholar 

  10. Goertzel, B., Iklé, M.: Steps toward a geometry of mind. In: Schmidhuber, J., Thórisson, K.R., Looks, M. (eds.) AGI 2011. LNCS(LNAI), pp. 334–339. Springer, Heidelberg (2011)

    Google Scholar 

  11. Goertzel, B., Ikl, M., Heljakka, I.G.: Probabilistic Logic Networks. Springer, Heidelberg (2008)

    MATH  Google Scholar 

  12. Goertzel, B.: The Hidden Pattern. Brown Walker (2006)

    Google Scholar 

  13. Goertzel, B., et al.: An integrative methodology for teaching embodied non-linguistic agents, applied to virtual animals in second life. In: Proc.of the First Conf. on AGI, IOS Press, Amsterdam (2008)

    Google Scholar 

  14. Goertzel, B., Pinto, H., Pennachin, C., Goertzel, I.F.: Using dependency parsing and probabilistic inference to extract relationships between genes, proteins and malignancies implicit among multiple biomedical research abstracts. In: Proc. of Bio-NLP 2006 (2006)

    Google Scholar 

  15. Goertzel, B., Pitt, J., Ikle, M., Pennachin, C., Liu, R.: Glocal memory: a design principle for artificial brains and minds. Neurocomputing (April 2010)

    Google Scholar 

  16. Goertzel, B., et al.: Opencogbot: An integrative architecture for embodied agi. In: Proc. of ICAI 2010, Beijing (2010)

    Google Scholar 

  17. Hutter, M.: Universal AI. Springer, Heidelberg (2005)

    Google Scholar 

  18. Hutter, M.: Feature dynamic bayesian networks. In: Proc. of the Second Conf. on AGI. Atlantis Press, London (2009)

    Google Scholar 

  19. Ikle, M., Pitt, J., Goertzel, B., Sellman, G.: Economic attention networks: Associative memory and resource allocation for general intelligence. In: Proceedings of AGI (2009)

    Google Scholar 

  20. Looks, M.: Competent Program Evolution. PhD Thesis. Computer Science Department, Washington University (2006)

    Google Scholar 

  21. Park, H., Amari, S., Fukumizu, K.: Adaptive natural gradient learning algorithms for various stochastic models. Neural Computing 13, 755–764 (2000)

    Google Scholar 

  22. Schaul, T., Schmidhuber, J.: Towards practical universal search. In: Proc. of the 3rd Conf. on AGI. Atlantis Press, London (2010)

    Google Scholar 

  23. Tulving, E., Craik, R.: The Oxford Handbook of Memory. Oxford U. Press (2005)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Ikle, M., Goertzel, B. (2011). Nonlinear-Dynamical Attention Allocation via Information Geometry. In: Schmidhuber, J., Thórisson, K.R., Looks, M. (eds) Artificial General Intelligence. AGI 2011. Lecture Notes in Computer Science(), vol 6830. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22887-2_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-22887-2_7

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-22886-5

  • Online ISBN: 978-3-642-22887-2

  • eBook Packages: Computer ScienceComputer Science (R0)