Abstract
Inspired by a broader perspective viewing intelligent system dynamics in terms of the geometry of “cognitive spaces,” we conduct a preliminary investigation of the application of information-geometry based learning to ECAN (Economic Attention Networks), the component of the integrative OpenCog AGI system concerned with attention allocation and credit assignment. We generalize Amari’s “natural gradient” algorithm for network learning to encompass ECAN and other recurrent networks, and apply it to small example cases of ECAN, demonstrating a dramatic improvement in the effectiveness of attention allocation compared to prior (Hebbian learning like) ECAN methods. Scaling up the method to deal with realistically-sized ECAN networks as used in OpenCog remains for the future, but should be achievable using sparse matrix methods on GPUs.
Keywords
- information geometry
- recurrent networks
- economic attention allocation
- ECAN
- OpenCog
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Amari, S.: Differential-geometrical methods in statistics. Lecture notes in statistics (1985)
Amari, S.: Natural gradient works efficiently in learning. Neural Computing 10, 251–276 (1998)
Amari, S.: i., Nagaoka, H.: Methods of information geometry. In: AMS (2000)
Arel, I., Rose, D., Coop, R.: Destin: A scalable deep learning architecture with application to high-dimensional robust pattern recognition. In: Proc. AAAI Workshop on Biologically Inspired Cognitive Architectures (2009)
Baskaran, M., Bordawekar, R.: Optimizing Sparse Matrix-Vector Multiplication on GPUs. IBM Research Report (2008)
Dabak, A.: A Geometry for Detection Theory. PhD Thesis, Rice U (1999)
Fauconnier, G., Turner, M.: The Way We Think: Conceptual Blending and the Minds Hidden Complexities. Basic (2002)
Frieden, R.: Physics from Fisher Information. Cambridge U. Press, New York (1998)
Garland, M.: Sparse matrix computations on manycore gpus. In: 45th Annual Design Automation Conference: 2008, pp. 2–6 (2008)
Goertzel, B., Iklé, M.: Steps toward a geometry of mind. In: Schmidhuber, J., Thórisson, K.R., Looks, M. (eds.) AGI 2011. LNCS(LNAI), pp. 334–339. Springer, Heidelberg (2011)
Goertzel, B., Ikl, M., Heljakka, I.G.: Probabilistic Logic Networks. Springer, Heidelberg (2008)
Goertzel, B.: The Hidden Pattern. Brown Walker (2006)
Goertzel, B., et al.: An integrative methodology for teaching embodied non-linguistic agents, applied to virtual animals in second life. In: Proc.of the First Conf. on AGI, IOS Press, Amsterdam (2008)
Goertzel, B., Pinto, H., Pennachin, C., Goertzel, I.F.: Using dependency parsing and probabilistic inference to extract relationships between genes, proteins and malignancies implicit among multiple biomedical research abstracts. In: Proc. of Bio-NLP 2006 (2006)
Goertzel, B., Pitt, J., Ikle, M., Pennachin, C., Liu, R.: Glocal memory: a design principle for artificial brains and minds. Neurocomputing (April 2010)
Goertzel, B., et al.: Opencogbot: An integrative architecture for embodied agi. In: Proc. of ICAI 2010, Beijing (2010)
Hutter, M.: Universal AI. Springer, Heidelberg (2005)
Hutter, M.: Feature dynamic bayesian networks. In: Proc. of the Second Conf. on AGI. Atlantis Press, London (2009)
Ikle, M., Pitt, J., Goertzel, B., Sellman, G.: Economic attention networks: Associative memory and resource allocation for general intelligence. In: Proceedings of AGI (2009)
Looks, M.: Competent Program Evolution. PhD Thesis. Computer Science Department, Washington University (2006)
Park, H., Amari, S., Fukumizu, K.: Adaptive natural gradient learning algorithms for various stochastic models. Neural Computing 13, 755–764 (2000)
Schaul, T., Schmidhuber, J.: Towards practical universal search. In: Proc. of the 3rd Conf. on AGI. Atlantis Press, London (2010)
Tulving, E., Craik, R.: The Oxford Handbook of Memory. Oxford U. Press (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ikle, M., Goertzel, B. (2011). Nonlinear-Dynamical Attention Allocation via Information Geometry. In: Schmidhuber, J., Thórisson, K.R., Looks, M. (eds) Artificial General Intelligence. AGI 2011. Lecture Notes in Computer Science(), vol 6830. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22887-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-22887-2_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-22886-5
Online ISBN: 978-3-642-22887-2
eBook Packages: Computer ScienceComputer Science (R0)