Abstract
An evolutionary reinforcement-learning algorithm, the operation of which was not associated with an optimality condition, was instantiated in an artificial organism. The algorithm caused the organism’s behavior to evolve in response to selection pressure applied by reinforcement from the environment. The resulting behavior was consistent with the well-established quantitative law of effect, which asserts that the time rate of a behavior is a hyperbolic function of the time rate of reinforcement obtained for the behavior. The high-order, steady-state, hyperbolic relationship between behavior and reinforcement exhibited by the artificial organism did not depend on specific qualitative or quantitative features of the evolutionary algorithm, and it described the organism’s behavior significantly better than other, similar, function forms. This evolutionary algorithm is a good candidate for a dynamics of live behavior, and it might be a useful building block for more complex artificial organisms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Davison, M., McCarthy, D.: The matching law. Erlbaum, Hillsdale (1988)
Daw, N.D., Touretzky, D.S.: Operant behavior suggests attentional gating of dopamine system inputs. Neurocomputing 38-40, 1161–1167 (2001)
Herrnstein, R.J.: Relative and absolute strength of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior 4, 267–272 (1961)
Herrnstein, R.J.: On the law of effect. Journal of the Experimental Analysis of Behavior 13, 243–266 (1970)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)
McDowell, J.J.: A computational model of selection by consequences. Journal of the Experimental Analysis of Behavior 81, 297–317 (2004)
McDowell, J.J., Bass, R., Kessel, R.: A new understanding of the foundation of linear system theory and an extension to nonlinear cases. Psychological Review 100, 407–419 (1993)
Moriarty, D.E., Schultz, A.C., Grefenstette, J.J.: Evolutionary algorithms for reinforcement learning. Journal of Artifical Intelligence Research 11, 241–276 (1999)
Rachlin, H., Battalio, R., Kagel, J., Green, L.: Maximization theory in behavioral psychology. Behavioral and Brain Sciences 4, 371–417 (1981)
Seth, A.K.: Evolving behavioural choice: An investigation into Herrnstein’s matching law. In: Floreano, D., Nicoud, J.D., Mondana, F. (eds.) ECAL 1999. LNCS, vol. 1674, pp. 225–236. Springer, Heidelberg (1999)
Seth, A.K.: Modeling group foraging: Individual suboptimality, interference, and a kind of matching. Adaptive Behavior 9, 67–90 (2002)
Sutton, R.S., Barto, A.G.: Reinforcement learning: An introduction. MIT Press, Cambridge (1998)
Touretzky, D.S., Saksida, L.M.: Operant conditioning in Skinnerbots. Adaptive Behavior 5, 219–247 (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
McDowell, J.J., Ansari, Z. (2005). The Quantitative Law of Effect is a Robust Emergent Property of an Evolutionary Algorithm for Reinforcement Learning. In: Capcarrère, M.S., Freitas, A.A., Bentley, P.J., Johnson, C.G., Timmis, J. (eds) Advances in Artificial Life. ECAL 2005. Lecture Notes in Computer Science(), vol 3630. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11553090_42
Download citation
DOI: https://doi.org/10.1007/11553090_42
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28848-0
Online ISBN: 978-3-540-31816-3
eBook Packages: Computer ScienceComputer Science (R0)