Abstract
Hierarchical structuring of behaviour is prevalent in natural and artificial agents and can be shown to be useful for learning and performing tasks. To progress systematic understanding of these benefits we study the effect of hierarchical architectures on the required information processing capability of an optimally acting agent. We show that an information-theoretical approach provides important insights into why factored and layered behaviour structures are beneficial.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Prete, F.R., Wells, H., Wells, P.H., Hurd, L.E.E.: The Praying Mantids. Johns Hopkins University Press, Baltimore (1999)
Yu, A.C., Margoliash, D.: Temporal Hierarchical Control of Singing in Birds. Science 273(5283), 1871–1875 (1996)
McGonigle, B.O., Chalmers, M., Dickinson, A.: Concurrent disjoint and reciprocal classification by Cebus apella in serial ordering tasks: evidence for hierarchical organization. Animal Cognition 6, 185–197 (2003)
Dawkins, R.: Hierarchical organisation: A candidate principle for ethology. In: Growing Points in Ethology, pp. 7–54. Cambridge University Press, Cambridge (1976)
Quinlan, J.R.: Induction of Decision Trees. Machine Learning 1(1), 81–106 (2007)
Barto, A.G., Mahadevan, S.: Recent Advances in Hierarchical Reinforcement Learning. Discrete Event Dynamic Systems 13(4), 341–379 (2003)
McGovern, A., Barto, A.G.: Automatic Discovery of Subgoals in Reinforcement Learning using Diverse Density. In: Proc. 18th International Conf. on Machine Learning, pp. 361–368. Morgan Kaufmann, San Francisco (2001)
Şimşek, O., Barto, A.G.: Using relative novelty to identify useful temporal abstractions in reinforcement learning. In: ICML 2004: Proceedings of the Twenty-First International Conference on Machine Learning, ACM, New York (2004)
Prescott, T.J., Redgrave, P., Gurney, K.: Layered Control Architectures in Robots and Vertebrates. Adaptive Behavior 7, 99–127 (1999)
Botvinick, M.M.: Hierarchical models of behavior and prefrontal function. Trends in Cognitive Science 12(5), 201–208 (2008)
Polani, D., Nehaniv, C., Martinetz, T., Kim, J.T.: Relevant Information in Optimized Persistence vs. Progeny Strategies. In: Proceedings of Artificial Life X. MIT Press, Cambridge (2006)
Capdepuy, P., Polani, D., Nehaniv, C.L.: Constructing the Basic Umwelt of Artificial Agents: An Information-Theoretic Approach. In: Almeida e Costa, F., Rocha, L.M., Costa, E., Harvey, I., Coutinho, A. (eds.) ECAL 2007. LNCS (LNAI), vol. 4648, pp. 375–383. Springer, Heidelberg (2007)
Klyubin, A.S., Polani, D., Nehaniv, C.L.: All Else Being Equal Be Empowered. In: Capcarrère, M.S., Freitas, A.A., Bentley, P.J., Johnson, C.G., Timmis, J. (eds.) ECAL 2005. LNCS (LNAI), vol. 3630, pp. 744–753. Springer, Heidelberg (2005)
Tishby, N., Pereira, F.C., Bialek, W.: The information bottleneck method. In: Proc. 37th Annual Allerton Conference on Communication, Control and Computing, Illinois, Urbana-Champaign (1999)
Sutton, R.S., Precup, D., Singh, S.: Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112(1-2), 181–211 (1999)
Blahut, R.E.: Computation of Channel Capacity and Rate-Distortion Functions. IEEE Transactions on Information Theory 18, 460–473 (1972)
Massey, J.L.: Causality, Feedback and Directed Information. In: Proceedings of the International Symposium on Information Theory and Its Applications (ISITA 1990), pp. 303–305 (1990)
Ay, N., Polani, D.: Information Flows in Causal Networks. Advances in Complex Systems 11(1), 17–41 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
van Dijk, S.G., Polani, D., Nehaniv, C.L. (2011). Hierarchical Behaviours: Getting the Most Bang for Your Bit. In: Kampis, G., Karsai, I., Szathmáry, E. (eds) Advances in Artificial Life. Darwin Meets von Neumann. ECAL 2009. Lecture Notes in Computer Science(), vol 5778. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21314-4_43
Download citation
DOI: https://doi.org/10.1007/978-3-642-21314-4_43
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21313-7
Online ISBN: 978-3-642-21314-4
eBook Packages: Computer ScienceComputer Science (R0)