Skip to main content

Empowerment–An Introduction

  • Chapter

Part of the Emergence, Complexity and Computation book series (ECC,volume 9)

Abstract

Is it better for you to own a corkscrew or not? If asked, you as a human being would likely say “yes”, but more importantly, you are somehow able to make this decision. You are able to decide this, even if your current acute problems or task do not include opening a wine bottle. Similarly, it is also unlikely that you evaluated several possible trajectories your life could take and looked at them with and without a corkscrew, and then measured your survival or reproductive fitness in each. When you, as a human cognitive agent, made this decision, you were likely relying on a behavioural “proxy”, an internal motivation that abstracts the problem of evaluating a decision impact on your overall life, but evaluating it in regard to some simple fitness function. One example would be the idea of curiosity, urging you to act so that your experience new sensations and learn about the environment. On average, this should lead to better and richer models of the world, which give you a better chance of reaching your ultimate goals of survival and reproduction.

Keywords

  • Mutual Information
  • Intrinsic Motivation
  • Action Sequence
  • Channel Capacity
  • Power Constraint

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-53734-9_4
  • Chapter length: 48 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   119.00
Price excludes VAT (USA)
  • ISBN: 978-3-642-53734-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Hardcover Book
USD   219.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Almeida e Costa, F., Rocha, L.: Introduction to the special issue: Embodied and situated cognition. Artificial Life 11(1-2), 5–12 (2005)

    CrossRef  Google Scholar 

  • Anthony, T., Polani, D., Nehaniv, C.: On preferred states of agents: how global structure is reflected in local structure. In: Proc. Artificial Life XI, pp. 25–32. MIT Press (2008)

    Google Scholar 

  • Anthony, T., Polani, D., Nehaniv, C.L.: Impoverished empowerment: ‘Meaningful’ action sequence generation through bandwidth limitation. In: Kampis, G., Karsai, I., Szathmáry, E. (eds.) ECAL 2009, Part II. LNCS, vol. 5778, pp. 294–301. Springer, Heidelberg (2011)

    CrossRef  Google Scholar 

  • Arimoto, S.: An algorithm for computing the capacity of arbitrary discrete memoryless channels. IEEE Transactions on Information Theory 18(1), 14–20 (1972)

    MATH  MathSciNet  CrossRef  Google Scholar 

  • Atick, J.: Could information theory provide an ecological theory of sensory processing? Network: Computation in Neural Systems 3(2), 213–251 (1992)

    MATH  CrossRef  Google Scholar 

  • Attneave, F.: Some informational aspects of visual perception. Psychological Review 61(3), 183 (1954)

    CrossRef  Google Scholar 

  • Ay, N., Bertschinger, N., Der, R., Güttler, F., Olbrich, E.: Predictive information and explorative behavior of autonomous robots. The European Physical Journal B-Condensed Matter and Complex Systems 63(3), 329–339 (2008)

    MATH  MathSciNet  CrossRef  Google Scholar 

  • Ay, N., Polani, D.: Information flows in causal networks. Advances in Complex Systems 11(1), 17–41 (2008)

    MATH  MathSciNet  CrossRef  Google Scholar 

  • Barlow, H.: Sensory mechanisms, the reduction of redundancy, and intelligence. The Mechanisation of Thought Processes, 535–539 (1959)

    Google Scholar 

  • Blahut, R.: Computation of channel capacity and rate-distortion functions. IEEE Transactions on Information Theory 18(4), 460–473 (1972)

    MATH  MathSciNet  CrossRef  Google Scholar 

  • Capdepuy, P.: Informational Principles of Perception-Action Loops and Collective Behaviours. PhD thesis, University of Hertfordshire (2010)

    Google Scholar 

  • Capdepuy, P., Polani, D., Nehaniv, C.: Maximization of potential information flow as a universal utility for collective behaviour. In: IEEE Symposium on Artificial Life, ALIFE 2007, pp. 207–213. IEEE (2007)

    Google Scholar 

  • Capdepuy, P., Polani, D., Nehaniv, C.L.: Perception–action loops of multiple agents: informational aspects and the impact of coordination. Theory in Biosciences 131(3), 149–159 (2012)

    CrossRef  Google Scholar 

  • Cover, T.M., Thomas, J.A.: Elements of Information Theory, 99th edn. Wiley-Interscience (1991)

    Google Scholar 

  • Csikszentmihalyi, M.: Beyond boredom and anxiety. Jossey-Bass (2000)

    Google Scholar 

  • Der, R., Steinmetz, U., Pasemann, F.: Homeokinesis: A new principle to back up evolution with learning. Max-Planck-Inst. für Mathematik in den Naturwiss (1999)

    Google Scholar 

  • Dewar, R.: Information theory explanation of the fluctuation theorem, maximum entropy production and self-organized criticality in non-equilibrium stationary states. Journal of Physics A: Mathematical and General 36(3), 631 (2003)

    MATH  MathSciNet  CrossRef  Google Scholar 

  • Dewar, R.C.: Maximum entropy production and the fluctuation theorem. Journal of Physics A: Mathematical and General 38(21), L371 (2005)

    Google Scholar 

  • Gibson James, J.: The Ecological Approach to Visual Perception. Houghton Mifflin, Boston (1979)

    Google Scholar 

  • Gordon, G., Ahissar, E.: Hierarchical curiosity loops and active sensing. Neural Networks 32, 119–129 (2012)

    CrossRef  Google Scholar 

  • Grinstein, G., Linsker, R.: Comments on: a derivation and application of the’maximum entropy production’principle. Journal of physics. A, Mathematical and Theoretical 40(31), 9717–9720 (2007)

    MATH  MathSciNet  CrossRef  Google Scholar 

  • Jeffery, W.R.: Adaptive evolution of eye degeneration in the mexican blind cavefish. Journal of Heredity 96(3), 185–196 (2005)

    CrossRef  Google Scholar 

  • Jung, T., Polani, D., Stone, P.: Empowerment for continuous agent environment systems. Adaptive Behavior 19(1), 16 (2011)

    CrossRef  Google Scholar 

  • Kaplan, F., Oudeyer, P.-y.: Maximizing learning progress: An internal reward system for development. In: Iida, F., Pfeifer, R., Steels, L., Kuniyoshi, Y. (eds.) Embodied Artificial Intelligence. LNCS (LNAI), vol. 3139, pp. 259–270. Springer, Heidelberg (2004)

    CrossRef  Google Scholar 

  • Klyubin, A.S., Polani, D., Nehaniv, C.L.: All else being equal be empowered. In: Capcarrère, M.S., Freitas, A.A., Bentley, P.J., Johnson, C.G., Timmis, J. (eds.) ECAL 2005. LNCS (LNAI), vol. 3630, pp. 744–753. Springer, Heidelberg (2005a)

    CrossRef  Google Scholar 

  • Klyubin, A., Polani, D., Nehaniv, C.: Empowerment: A universal agent-centric measure of control. In: The 2005 IEEE Congress on Evolutionary Computation, vol. 1, pp. 128–135. IEEE (2005b)

    Google Scholar 

  • Klyubin, A., Polani, D., Nehaniv, C.: Keep your options open: an information-based driving principle for sensorimotor systems. PloS ONE 3(12), e4018 (2008)

    Google Scholar 

  • Kraskov, A., Stögbauer, H., Grassberger, P.: Estimating mutual information. Phys. Rev. E 69, 066138 (2004)

    Google Scholar 

  • López, C., Martínez, E.: Sub-finslerian metric associated to an optimal control system. SIAM J. Control Optim. 39(3), 798–811 (2000)

    MATH  MathSciNet  CrossRef  Google Scholar 

  • Lungarella, M., Pegors, T., Bulwinkle, D., Sporns, O.: Methods for quantifying the informational structure of sensory and motor data. Neuroinformatics 3(3), 243–262 (2005)

    CrossRef  Google Scholar 

  • Massey, J.: Causality, feedback and directed information. In: Proc. Int. Symp. Inf. Theory Applic., ISITA 1990, pp. 303–305 (1990)

    Google Scholar 

  • Olsson, L., Nehaniv, C., Polani, D.: Sensor adaptation and development in robots by entropy maximization of sensory data. In: Proceedings of the 2005 IEEE International Symposium on Computational Intelligence in Robotics and Automation, CIRA 2005, pp. 587–592. IEEE (2005)

    Google Scholar 

  • Pearl, J.: Causality: Models, Reasoning and Inference. Cambridge University Press (2000)

    Google Scholar 

  • Pfeifer, R., Bongard, J., Grand, S.: How the body shapes the way we think: a new view of intelligence. The MIT Press (2007)

    Google Scholar 

  • Polani, D.: Information: Currency of life? HFSP Journal 3(5), 307–316 (2009)

    CrossRef  Google Scholar 

  • Polani, D., Nehaniv, C., Martinetz, T., Kim, J.T.: Relevant information in optimized persistence vs. progeny strategies. In: Artificial Life X: Proceedings of the Tenth International Conference on the Simulation and Synthesis of Living Systems, pp. 337–343. Citeseer (2006)

    Google Scholar 

  • Salge, C., Glackin, C., Polani, D.: Approximation of empowerment in the continuous domain. Advances in Complex Systems 16(1&2), 1250079 (2012)

    Google Scholar 

  • Salge, C., Polani, D.: Digested information as an information theoretic motivation for social interaction. Journal of Artificial Societies and Social Simulation 14(1), 5 (2011)

    Google Scholar 

  • Schmidhuber, J.: Curious model-building control systems. In: IEEE International Joint Conference on Neural Networks, pp. 1458–1463. IEEE (1991)

    Google Scholar 

  • Schmidhuber, J.: Exploring the predictable. Advances in Evolutionary Computing 6, 579–612 (2002)

    Google Scholar 

  • Shannon, C.E.: A mathematical theory of communication. Bell Sys. Tech. Journal 27, 623–656 (1948)

    MathSciNet  CrossRef  Google Scholar 

  • Singh, S., Barto, A.G., Chentanez, N.: Intrinsically motivated reinforcement learning. In: Proceedings of the 18th Annual Conference on Neural Information Processing Systems (NIPS), Vancouver, B.C., Canada (2005)

    Google Scholar 

  • Singh, S., Lewis, R., Barto, A., Sorg, J.: Intrinsically motivated reinforcement learning: An evolutionary perspective. IEEE Transactions on Autonomous Mental Development 2(2), 70–82 (2010)

    CrossRef  Google Scholar 

  • Steels, L.: The autotelic principle. In: Iida, F., Pfeifer, R., Steels, L., Kuniyoshi, Y. (eds.) Embodied Artificial Intelligence. LNCS (LNAI), vol. 3139, pp. 231–242. Springer, Heidelberg (2004)

    CrossRef  Google Scholar 

  • Sutton, R.S., Barto, A.G.: Reinforcement Learning. MIT Press, Cambridge (1998)

    Google Scholar 

  • Telatar, E.: Capacity of multi-antenna gaussian channels. European Transactions on Telecommunications 10(6), 585–595 (1999)

    CrossRef  Google Scholar 

  • Tishby, N., Pereira, F., Bialik, W.: The information bottleneck method. In: Proceedings of the 37th Annual Allerton Conference on Communication, Control and Computing, pp. 368–377 (1999)

    Google Scholar 

  • Varela, F., Thompson, E., Rosch, E.: The Embodied Mind: Cognitive Science and Human Experience. The MIT Press (1992)

    Google Scholar 

  • von Uexküll, J.: Umwelt und Innenwelt der Tiere. Springer (1909)

    Google Scholar 

  • Wilkens, G.R.: Finsler geometry in low-dimensional control theory. In: Bao, D.D.-W., Shen Chern, S., Shen, Z. (eds.) Finsler Geometry: Joint Summer Research Conference on Finsler Geometry, Seattle, Washington, July 16-20. Contemporary Mathematics, vol. 196, pp. 245–260. AMS (1995)

    Google Scholar 

  • Wissner-Gross, A., Freer, C.: Causal entropic forces. Physical Review Letters 110(16), 168702 (2013)

    CrossRef  Google Scholar 

  • Yaeger, L.S.: How evolution guides complexity. HFSP Journal 3(5), 328–339 (2009)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Christoph Salge .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Salge, C., Glackin, C., Polani, D. (2014). Empowerment–An Introduction. In: Prokopenko, M. (eds) Guided Self-Organization: Inception. Emergence, Complexity and Computation, vol 9. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-53734-9_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-53734-9_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-53733-2

  • Online ISBN: 978-3-642-53734-9

  • eBook Packages: EngineeringEngineering (R0)