Skip to main content
SpringerLink
Log in

Habit-Based and Goal-Directed Systems: Knowledge Transfer in Individual and Social Learning

  • Chapter
  • First Online:
The Logic of Social Practices

Part of the book series: Studies in Applied Philosophy, Epistemology and Rational Ethics ((SAPERE,volume 52))

Abstract

The present chapter discusses value-based habitual and goal-directed systems as studied in the animal and human learning literature. It focuses on the means by which these two systems might interact in knowledge transfer, particularly as it applies to social learning. Knowledge is conceived here in terms of types of logic computations as implemented by neural networks. A discussion of dual-process type structures in the brain is provided as well as neural-dynamic implementations thereof and considerations for how a perspective of the brain as carrying out logic computations might be useful for developing the general cognitive capacities of artificial agents.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Halford et al. [70] suggests that a transitive inference process that is not based on ‘omnidirectionality’, i.e. that ‘outputs’ of an otherwise feedforward process can be used to infer ‘inputs’ is only implicit and demonstrative of a lower order cognitive process.

  2. 2.

    The first and second phase of the transfer of control paradigm can, in fact, be presented in any order though more standardly the initial instrumental phase is used first.

References

  1. Seger, C.A.: How do the basal ganglia contribute to categorization? Their roles in generalization, response selection, and learning via feedback. Neurosci. Biobehav. Rev. 32, 265–278 (2008)

    Article  Google Scholar 

  2. Phillips, S., Wilson, W.H., Halford, G.S.: What do transitive inference and class inclusion have in common? Categorical (co) products and cognitive development. PLoS Comput. Biol. 5(12) (2009)

    Article  Google Scholar 

  3. Seger, C.A., Spiering, B.J.: A critical review of habit learning and the basal ganglia. Front. Syst. Neurosci. 5 (2011)

    Google Scholar 

  4. Davidson, R.J., Irwin, W.: The functional neuroanatomy of emotion and affective style. Trends Cognit. Sci. 3(1), 11–21 (1999)

    Article  Google Scholar 

  5. Dickinson, A.: Actions and habits: the development of behavioural autonomy. Philos. Trans. R. Soc. Lond. B Biol. Sci. 308, 67–78 (1985)

    Google Scholar 

  6. Hirsh, J.: The hippocampus and contextual retrieval of information from memory: a theory. Behav. Biol. 12, 421–444 (1974)

    Article  Google Scholar 

  7. Seger, C.A.: The involvement of corticostriatal loops in learning across tasks, species, and methodologies, In: Groenewegen, H.J., Voorn, P., Berendse, H.W., Mulder, A.B., Cools, A.R. (eds.) The Basal Ganglia IX, pp. 25–39. Springer, New York (2009)

    Google Scholar 

  8. Daw, N.D., O’Doherty, J.P.: Multiple systems for value learning. In: Neuroeconomics: Decision Making and the Brain, 2nd edn, pp. 393–410. Academic Press, London (2014)

    Chapter  Google Scholar 

  9. Miller, K.J., Shenhav, A., Ludvig, E.A.: Habits without values. Psychol. Rev. (2019)

    Google Scholar 

  10. Lowe, R., Ziemke, T.: Exploring the relationship of reward and punishment in reinforcement learning. In: 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pp. 140–147 (2013)

    Google Scholar 

  11. Lee, G., Lowe, R., Ziemke, T.: Modelling early infant walking: testing a generic CPG architecture on the NAO humanoid. In: 2011 IEEE International Conference on Development and Learning (ICDL), vol. 2, pp. 1–6 (2011)

    Google Scholar 

  12. Li, C., Lowe, R., Ziemke, T.: Crawling posture learning in humanoid robots using a natural-actor-critic CPG architecture. In: Artificial Life Conference Proceedings, 13, pp. 1182–1190. MIT Press, Cambridge, MA (2013)

    Google Scholar 

  13. Li, C., Lowe, R., Ziemke, T.: A novel approach to locomotion learning: Actor-Critic architecture using central pattern generators and dynamic motor primitives. Front. Neurorobotics 8, 23 (2014)

    Article  Google Scholar 

  14. Rolls, E.T.: On the brain and emotion. Behav. Brain Sci. 23(2), 219–228 (2000)

    Article  Google Scholar 

  15. Suri, R.E., Schultz, W.: Learning of sequential movements by neural network model with dopamine-like reinforcement signal. Exp. Brain Res. 121(3), 350–354 (1998)

    Article  Google Scholar 

  16. Suri, R.E., Schultz, W.: Temporal difference model reproduces anticipatory neural activity. Neural Comput. 13(4), 841–862 (2001)

    Article  Google Scholar 

  17. Peterson, G.B., Trapold, M.A.: Effects of altering outcome expectancies on pigeons’ delayed conditional discrimination performance. Learn. Motiv. 11, 267–288 (1980)

    Article  Google Scholar 

  18. Daw, N.D., Niv, Y., Dayan, P.: Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8(12), 1704 (2005)

    Article  Google Scholar 

  19. Sun, R.: Interpreting psychological notions: a dual-process computational theory. J. Appl. Res. Memory Cognit. 4(3), 191–196 (2015)

    Article  Google Scholar 

  20. de Wit, S., Dickinson, A.: Associative theories of goal-directed behaviour: a case for animal–human translational models. Psychol. Res. PRPF 73(4), 463–476 (2009)

    Article  Google Scholar 

  21. Mowrer, O.H.: On the dual nature of learning: A reinterpretation of ‘‘conditioning’’ and ‘‘problem-solving’’. Harv. Educ. Rev. 17, 102–148 (1947)

    Google Scholar 

  22. Trapold, M.A.: Are expectancies based upon different positive reinforcing events discriminably different? Learn. Motiv. 1, 129–140 (1970)

    Article  Google Scholar 

  23. Cardinal, R.N., Parkinson, J.A., Hall, J., Everitt, B.J.: Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci. Biobehav. Rev. 26(3), 321–352 (2002)

    Article  Google Scholar 

  24. Urcuioli, P.: Behavioral and associative effects of differential outcomes in discriminating learning. Learn. Behav. 33(1), 1–21 (2005)

    Article  Google Scholar 

  25. Lowe, R., Humphries, M., Ziemke, T.: The dual-route hypothesis: evaluating a neurocomputational model of fear conditioning in rats. Connect. Sci. 21(1), 15–37 (2009)

    Article  Google Scholar 

  26. Lowe, R., Billing, E.: Affective-associative two-process theory: a neural network investigation of adaptive behaviour in differential outcomes training. Adapt. Behav. 25(1), 5–23 (2017)

    Article  Google Scholar 

  27. Lowe, R., Almér, A., Billing, E., Sandamirskaya, Y., Balkenius, C.: Affective–associative two-process theory: a neurocomputational account of partial reinforcement extinction effects. Biol. Cybern. 111(5–6), 365–388 (2017)

    Article  Google Scholar 

  28. Braver, T.S., et al.: Mechanisms of motivation–cognition interaction: challenges and opportunities. Cognit. Affect. Behav. Neurosci. 14(2), 443–472 (2014)

    Article  Google Scholar 

  29. Urcuioli, P.: Some relationships between outcome expectancies and sample stimuli in pigeons’ delayed matching. Anim. Learn. Behav. 18(3), 302–314 (1990)

    Article  Google Scholar 

  30. Urcuioli, P.: Stimulus control and stimulus class formation. In: Madden, G.J., Dube, W.V., Hackenberg, T.D., Hanley, G.P., Lattal, K.A. (eds.) APA Handbook of Behavior Analysis, vol. 1, pp. 361–386. American Psychological Association, Washington, DC (2013)

    Google Scholar 

  31. Amsel, A.: The role of frustrative nonreward in noncontinuous reward situations. Psychol. Bull. 55, 102–119 (1958)

    Article  Google Scholar 

  32. Amsel, A.: Frustration Theory: An Analysis of Dispositional Learning and Memory. Cambridge University Press, Cambridge (1992)

    Book  Google Scholar 

  33. Overmier, J.B., Lawry, J.A.: Pavlovian conditioning and the mediation of behavior. Psychol. Learn. Motiv. 13, 1–55 (1979)

    Article  Google Scholar 

  34. McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5, 115–133 (1943)

    Article  Google Scholar 

  35. Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65, 386 (1958)

    Article  Google Scholar 

  36. Rumelhart, D.E., Hinton, G.E., McClelland, J.L.: A general framework for parallel distributed processing. In: Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1, pp. 45–76 (1986)

    Google Scholar 

  37. Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Netw. 2(5), 359–366 (1989)

    Article  Google Scholar 

  38. Navarro-Guerrero, N., Lowe, R.J., Wermter, S.: The effects on adaptive behaviour of negatively valenced signals in reinforcement learning. In: 2017 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), pp. 148–155 (2017)

    Google Scholar 

  39. Navarro-Guerrero, N., Lowe, R.J., Wermter, S.: Improving robot motor learning with negatively valenced reinforcement signals. Front. Neurorobotics 11, 10 (2017)

    Article  Google Scholar 

  40. Wierman, M.J.: An Introduction to the Mathematics of Uncertainty, 149–150. Creighton University (2010)

    Google Scholar 

  41. Zadeh, L.A., et al.: Fuzzy Sets, Fuzzy Logic. World Scientific Press, Fuzzy Systems (1996)

    Book  Google Scholar 

  42. Lange, S., Riedmiller, M., Voigtländer, A.: Autonomous reinforcement learning on raw visual input data in a real world application. In: The 2012 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2012)

    Google Scholar 

  43. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., … Petersen, S.: Human-level control through deep reinforcement learning. Nature 518(7540), 529 (2015)

    Article  Google Scholar 

  44. Yang, Z., Xie, Y., Wang, Z.: A theoretical analysis of deep Q-learning. arXiv preprint arXiv:1901.00137 (2019)

  45. Lowe, R., Almer, A., Lindblad, G., Gander, P., Michael, J., Vesper, C.: Minimalist social-affective value for use in joint action: a neural-computational hypothesis. Front. Comput. Neurosci. 10 (2016)

    Google Scholar 

  46. Lowe, R., Almér, A., Gander, P., Balkenius, C.: Vicarious value learning and inference in human-human and human-robot interaction. In: 2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), pp. 395–400. IEEE (2019)

    Google Scholar 

  47. Bratman, M.E.: Shared cooperative activity. Philos. Rev. 101, 327–341 (1992)

    Article  Google Scholar 

  48. Vesper, C., Butterfill, S., Knoblich, G., Sebanz, N.: A minimal architecture for joint action. Neural Netw. 23, 998–1003 (2010)

    Article  Google Scholar 

  49. Kahneman, D., Tversky, A.: Rational choice and the framing of decisions. J. Bus. 59(4), 251–278 (1986)

    Google Scholar 

  50. Halford, G.S., Wilson, W.H., Phillips, S.: A conceptual complexity metric based on representational rank. Psychology Computer Science & Engineering, University of Queensland University of New South Wales, Queensland 4072, Australia; Sydney, NSW 2052 Australia (1999)

    Google Scholar 

  51. Halford, G.S., Wilson, W.H., Andrews, G., Phillips, S.: Categorizing Cognition: Toward Conceptual Coherence in the Foundations of Psychology. MIT Press (2014)

    Google Scholar 

  52. Halford, G.S., Wilson, W.H., Phillips, S.: Processing capacity defined by relational complexity: implications for comparative, developmental, and cognitive psychology. Behav. Brain Sci. 21(6), 803–831 (1998)

    Article  Google Scholar 

  53. Halford, G.S., Wilson, W.H., Phillips, S.: Relational knowledge: the foundation of higher cognition. Trends Cognit. Sci. 14(11), 497–505 (2010)

    Article  Google Scholar 

  54. Hummel, J.E., Holyoak, K.J.: Distributed representations of structure: a theory of analogical access and mapping. Psychol. Rev. 104(3), 427 (1997)

    Article  Google Scholar 

  55. Hummel, J.E., Holyoak, K.J.: A process model of human transitive inference. In: Spatial Schemas in Abstract Thought, pp. 279–305 (2001)

    Google Scholar 

  56. Doumas, L.A., Morrison, R.G., Richland, L.E.: Individual differences in relational learning and analogical reasoning: a computational model of longitudinal change. Front. Psychol. 9 (2018)

    Google Scholar 

  57. Knowlton, B.J., Morrison, R.G., Hummel, J.E., Holyoak, K.J.: A neurocomputational system for relational reasoning. Trends Cognit. Sci. 16(7), 373–381 (2012)

    Article  Google Scholar 

  58. Hummel, J. E., Holyoak, K. J., Green, C., Doumas, L. A., Devnich, D., Kittur, A., Kalar, D. J.: A solution to the binding problem for compositional connectionism. In: Levy, S.D., Gayler, R. (eds.) Compositional Connectionism in Cognitive Science: Papers from the AAAI Fall Symposium, pp. 31–34 (2004)

    Google Scholar 

  59. Lowe, R. Almér, A., Balkenius, C.: Bridging connectionism and relational cognition through bi-directional affective-associative processing. Open Inf. Sci. 3, 235–260 (2019)

    Article  Google Scholar 

  60. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  Google Scholar 

  61. Frank, M.J., Rudy, J.W., Levy, W.B., O’Reilly, R.C.: When logic fails: implicit transitive inference in humans. Memory Cognit. 33(4), 742–750 (2005)

    Article  Google Scholar 

  62. Amari, S.I.: Dynamics of pattern formation in lateral-inhibition type neural fields. Biol. Cybern. 27(2), 77–87 (1977)

    Article  Google Scholar 

  63. Tran, S.N., Garcez, A.S.D.A.: Deep logic networks: inserting and extracting knowledge from deep belief networks. IEEE Trans. Neural Netw. Learn. Syst. (2016)

    Google Scholar 

  64. Rastegari, M., Ordonez, V., Redmon, J., Farhadi, A.: Xnor-net: Imagenet classification using binary convolutional neural networks. In: European Conference on Computer Vision, pp. 525–542. Springer International Publishing (2016)

    Google Scholar 

  65. Zhou, X., Li, S., Qin, K., Li, K., Tang, F., Hu, S., … & Lin, Z.: Deep adaptive network: an efficient deep neural network with sparse binary connections. IEEE Trans. Neural Netw. Learn. Syst. (forthcoming)

    Google Scholar 

  66. Goldental, A., Guberman, S., Vardi, R., Kanter, I.: A computational paradigm for dynamic logic-gates in neuronal activity. Front. Comput. Neurosci. 8 (2014)

    Google Scholar 

  67. Vardi, R., Guberman, S., Goldental, A., Kanter, I.: An experimental evidence-based computational paradigm for new logic-gates in neuronal activity. EPL 103, 66001 (2013)

    Article  Google Scholar 

  68. Kiryazov, K., Lowe, R., Becker-Asano, C., Randazzo, M.: The role of arousal in two-resource problem tasks for humanoid service robots. In: 2013 IEEE ROMAN, pp. 62–69 (2013)

    Google Scholar 

  69. Lowe, R., Philippe, P., Montebelli, A., Morse, A., Ziemke, T.: Affective modulation of embodied dynamics. In: The Role of Emotion in Adaptive Behaviour and Cognitive Robotics, Electronic Proceedings of SAB Workshop (2008)

    Google Scholar 

  70. Halford, G.S., Andrews, G., Wilson, W.H., Phillips, S.: Computational models of relational processes in cognitive development. Cogn. Dev. 27(4), 481–499 (2012)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Gothenburg, Forskningsgången 6, Gothenburg, 41756, Sweden

    Robert Lowe

Authors
  1. Robert Lowe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Robert Lowe .

Editor information

Editors and Affiliations

  1. Social Ontology and Social Epistemology, Faculty of Philosophy, Pontifical Lateran University, Rome, Italy

    Raffaela Giovagnoli

  2. Division of Cognition and Communication, Department of Applied IT, University of Gothenburg, Gothenburg, Sweden

    Robert Lowe

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Lowe, R. (2020). Habit-Based and Goal-Directed Systems: Knowledge Transfer in Individual and Social Learning. In: Giovagnoli, R., Lowe, R. (eds) The Logic of Social Practices. Studies in Applied Philosophy, Epistemology and Rational Ethics, vol 52. Springer, Cham. https://doi.org/10.1007/978-3-030-37305-4_10

Download citation

Publish with us

Policies and ethics

Search

Navigation