Encyclopedia of Machine Learning and Data Mining

2017 Edition
| Editors: Claude Sammut, Geoffrey I. Webb

Adaptive Resonance Theory

Reference work entry
DOI: https://doi.org/10.1007/978-1-4899-7687-1_6


Computational models based on cognitive and neural systems are now deeply embedded in the standard repertoire of machine learning and data mining methods, with intelligent learning systems enhancing performance in nearly every existing application area. Beyond data mining, this article shows how models based on adaptive resonance theory (ART) may provide entirely new questions and practical solutions for technological applications. ART models carry out hypothesis testing, search, and incremental fast or slow, self-stabilizing learning, recognition, and prediction in response to large nonstationary databases (big data). Three computational examples, each based on the distributed ART neural network, frame questions and illustrate how a learning system (each with no free parameters) may enhance the analysis of large-scale data. Performance of each task is simulated on a common mapping platform, a remote sensing dataset called the Boston Testbed, available online along with open-source system code. Key design elements of ART models and links to software for each system are included. The article further points to future applications for integrative ART-based systems that have already been computationally specified and simulated. New application directions include autonomous robotics, general-purpose machine vision, audition, speech recognition, language acquisition, eye movement control, visual search, figure-ground separation, invariant object recognition, social cognition, object and spatial attention, scene understanding, space-time integration, episodic memory, navigation, object tracking, system-level analysis of mental disorders, and machine consciousness.

This is a preview of subscription content, log in to check access.

Recommended Reading

  1. Amis GP, Carpenter GA (2010) Self-supervised ARTMAP. Neural Netw 23:265–282CrossRefGoogle Scholar
  2. Baloch AA, Grossberg S (1997) A neural model of high-level motion processing: line motion and formotion dynamics. Vis Res 37:3037–3059CrossRefGoogle Scholar
  3. Baloch AA, Grossberg S, Mingolla E, Nogueira CAM (1999) A neural model of first-order and second-order motion perception and magnocellular dynamics. J Opt Soc Am A 16:953–978CrossRefGoogle Scholar
  4. Berzhanskaya J, Grossberg S, Mingolla E (2007) Laminar cortical dynamics of visual form and motion interactions during coherent object motion perception. Spat Vis 20:337–395CrossRefGoogle Scholar
  5. Brown J, Bullock D, Grossberg S (1999) How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. J Neurosci 19:10502–10511Google Scholar
  6. Brown JW, Bullock D, Grossberg S (2004) How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades. Neural Netw 17:471–510zbMATHCrossRefGoogle Scholar
  7. Browning A, Grossberg S, Mingolla M (2009a) A neural model of how the brain computes heading from optic flow in realistic scenes. Cogn Psychol 59:320–356CrossRefGoogle Scholar
  8. Browning A, Grossberg S, Mingolla M (2009b) Cortical dynamics of navigation and steering in natural scenes: motion-based object segmentation, heading, and obstacle avoidance. Neural Netw 22:1383–1398CrossRefGoogle Scholar
  9. Bullock D, Grossberg S (1988) Neural dynamics of planned arm movements: emergent invariants and speed-accuracy properties during trajectory formation. Psychol Rev 95:49–90CrossRefGoogle Scholar
  10. Bullock D, Grossberg S (1991) Adaptive neural networks for control of movement trajectories invariant under speed and force rescaling. Hum Mov Sci 10:3–53CrossRefGoogle Scholar
  11. Bullock D, Cisek P, Grossberg S (1998) Cortical networks for control of voluntary arm movements under variable force conditions. Cereb Cortex 8:48–62CrossRefGoogle Scholar
  12. Cao Y, Grossberg S (2005) A laminar cortical model of stereopsis and 3D surface perception: closure and da Vinci stereopsis. Spat Vis 18:515–578CrossRefGoogle Scholar
  13. Cao Y, Grossberg S (2012) Stereopsis and 3D surface perception by spiking neurons in laminar cortical circuits: a method of converting neural rate models into spiking models. Neural Netw 26:75–98CrossRefGoogle Scholar
  14. Cao Y, Grossberg S, Markowitz J (2011) How does the brain rapidly learn and reorganize view- and positionally-invariant object representations in inferior temporal cortex? Neural Netw 24:1050–1061zbMATHCrossRefGoogle Scholar
  15. Carpenter GA (1994) A distributed outstar network for spatial pattern learning. Neural Netw 7:159–168CrossRefGoogle Scholar
  16. Carpenter GA (1997) Distributed learning, recognition, and prediction by ART and ARTMAP neural networks. Neural Netw 10:1473–1494CrossRefGoogle Scholar
  17. Carpenter GA (2001) Neural network models of learning and memory: leading questions and an emerging framework. Trends Cogn Sci 5:114–118CrossRefGoogle Scholar
  18. Carpenter GA, Gaddam SC (2010) Biased ART: a neural architecture that shifts attention toward previously disregarded features following an incorrect prediction. Neural Netw 23:435–451CrossRefGoogle Scholar
  19. Carpenter GA, Grossberg S (1987) A massively parallel architecture for a self-organizing neural pattern recognition machine. Comput Vis Graph Image Process 37:54–115zbMATHCrossRefGoogle Scholar
  20. Carpenter GA, Grossberg S (1990) ART 3: hierarchical search using chemical transmitters in self-organizing pattern recognition architectures. Neural Netw 4: 129–152CrossRefGoogle Scholar
  21. Carpenter G, Grossberg S (1993) Normal and amnesic learning, recognition, and memory by a neural model of cortico-hippocampal interactions. Trends Neurosci 16:131–137CrossRefGoogle Scholar
  22. Carpenter GA, Markuzon N (1998) ARTMAP-IC and medical diagnosis: instance counting and inconsistent cases. Neural Netw 11:323–336CrossRefGoogle Scholar
  23. Carpenter GA, Grossberg S, Reynolds JH (1991a) ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network. Neural Netw 4:565–588CrossRefGoogle Scholar
  24. Carpenter GA, Grossberg S, Rosen DB (1991b) Fuzzy ART: fast stable learning and categorization of analog patterns by an adaptive resonance system. Neural Netw 4:759–771CrossRefGoogle Scholar
  25. Carpenter GA, Grossberg S, Markuzon N, Reynolds JH, Rosen DB (1992) Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans Neural Netw 3:698–713CrossRefGoogle Scholar
  26. Carpenter GA, Martens S, Ogas OJ (2005) Self-organizing information fusion and hierarchical knowledge discovery: a new framework using ARTMAP neural networks. Neural Netw 18:287–295CrossRefGoogle Scholar
  27. Chandler B, Grossberg S (2012) Joining distributed pattern processing and homeostatic plasticity in recurrent on-center off-surround shunting networks: noise, saturation, short-term memory, synaptic scaling, and BDNF. Neural Netw 25: 21–29CrossRefGoogle Scholar
  28. Chang H-C, Grossberg S, Cao Y (2014) Where’s Waldo? How perceptual cognitive, and emotional brain processes cooperate during learning to categorize and find desired objects in a cluttered scene. Front Integr Neurosci doi:10.3389/fnint.2014.0043Google Scholar
  29. Chapelle O, Schölkopf B, Zien A (eds) (2006) Semi-supervised learning. MIT, CambridgeGoogle Scholar
  30. Contreras-Vidal JL, Grossberg S, Bullock D (1997) A neural model of cerebellar learning for arm movement control: cortico-spino-cerebellar dynamics. Learn Mem 3:475–502CrossRefGoogle Scholar
  31. Dranias M, Grossberg S, Bullock D (2008) Dopaminergic and non-dopaminergic value systems in conditioning and outcome-specific revaluation. Brain Res 1238:239–287CrossRefGoogle Scholar
  32. Elder D, Grossberg S, Mingolla M (2009) A neural model of visually guided steering, obstacle avoidance, and route selection. J Exp Psychol Hum Percept Perform 35:1501–1531CrossRefGoogle Scholar
  33. Fang L, Grossberg S (2009) From stereogram to surface: how the brain sees the world in depth. Spat Vis 22:45–82CrossRefGoogle Scholar
  34. Fazl A, Grossberg S, Mingolla E (2009) View-invariant object category learning, recognition, and search: how spatial and object attention are coordinated using surface-based attentional shrouds. Cogn Psychol 58:1–48CrossRefGoogle Scholar
  35. Foley NC, Grossberg S, Mingolla E (2012) Neural dynamics of object-based multifocal visual spatial attention and priming: object cueing, useful-field-of-view, and crowding. Cognitive Psychology 65:77–117CrossRefGoogle Scholar
  36. Gancarz G, Grossberg G (1999) A neural model of the saccadic eye movement control explains task-specific adaptation. Vis Res 39:3123–3143CrossRefGoogle Scholar
  37. Grossberg S (1984) Some psychophysiological and pharmacological correlates of a developmental, cognitive, and motivational theory. In: Karrer R, Cohen J, Tueting P (eds) Brain and information: event related potential. New York Academy of Sciences, New York, pp 58–142.Google Scholar
  38. Grossberg S (2000a) The complementary brain: unifying brain dynamics and modularity. Trends Cogn Sci 4:233–246CrossRefGoogle Scholar
  39. Grossberg S (2000b) The imbalanced brain: from normal behavior to schizophrenia. Biol Psychiatry 48:81–98CrossRefGoogle Scholar
  40. Grossberg S (2013) Adaptive resonance theory: how a brain learns to consciously attend, learn, and recognize a changing World. Neural Netw 37:1–47CrossRefGoogle Scholar
  41. Grossberg, S. (2016). Towards solving the hard problem of consciousness: the varieties of brain resonances and the conscious experiences that they support. Submitted for publicationGoogle Scholar
  42. Grossberg S, Huang T-R (2009) ARTSCENE: a neural system for natural scene classification. J Vis 9(6):1–19CrossRefGoogle Scholar
  43. Grossberg S, Merrill JWL (1992) A neural network model of adaptively timed reinforcement learning and hippocampal dynamics. Cogn Brain Res 1:3–38CrossRefGoogle Scholar
  44. Grossberg S, Merrill JWL (1996) The hippocampus and cerebellum in adaptively timed learning, recognition, and movement. J Cogn Neurosci 8:257–277CrossRefGoogle Scholar
  45. Grossberg S, Pearson L (2008) Laminar cortical dynamics of cognitive and motor working memory, sequence learning and performance: toward a unified theory of how the cerebral cortex works. Psychol Rev 115:677–732CrossRefGoogle Scholar
  46. Grossberg S, Pilly PK (2012) How entorhinal grid cells may learn multiple spatial scales from a dorsoventral gradient of cell response rates in a self-organizing map. PLoS Comput Biol 8(10):31002648. doi: 10.1371/journal.pcbi.1002648MathSciNetCrossRefGoogle Scholar
  47. Grossberg S, Pilly PK (2014) Coordinated learning of grid cell and place cell spatial and temporal properties: multiple scales, attention, and oscillations. Philos Trans R Soc B 369:20120524CrossRefGoogle Scholar
  48. Grossberg S, Rudd ME (1992) Cortical dynamics of visual motion perception: short-range and long-range apparent motion (with Rudd ME). Psychol Rev 99:78–121CrossRefGoogle Scholar
  49. Grossberg S, Schmajuk NA (1989) Neural dynamics of adaptive timing and temporal discrimination during associative learning. Neural Netw 2:79–102CrossRefGoogle Scholar
  50. Grossberg S, Seidman D (2006) Neural dynamics of autistic behaviors: cognitive, emotional, and timing substrates. Psychol Rev 113:483–525CrossRefGoogle Scholar
  51. Grossberg S, Swaminathan G (2004) A laminar cortical model for 3D perception of slanted and curved surfaces and of 2D images: development, attention and bistability. Vis Res 44:1147–1187CrossRefGoogle Scholar
  52. Grossberg S, Vladusich T (2010) How do children learn to follow gaze, share joint attention, imitate their teachers, and use tools during social interactions? Neural Netw 23:940–965CrossRefGoogle Scholar
  53. Grossberg S, Leveille J, Versace M (2011) How do object reference frames and motion vector decomposition emerge in laminar cortical circuits? Atten Percept Psychophys 73:1147–1170CrossRefGoogle Scholar
  54. Grossberg S, Kuhlmann L, Mingolla E (2007) A neural model of 3D shape-from-texture: multiple-scale filtering, boundary grouping, and surface filling-in. Vis Res 47:634–672CrossRefGoogle Scholar
  55. Grossberg S, Mingolla E, Viswanathan L (2001) Neural dynamics of motion integration and segmentation within and across apertures. Vis Res 41:2521–2553CrossRefGoogle Scholar
  56. Grossberg S, Srihasam K, Bullock D (2012) Neural dynamics of saccadic and smooth pursuit eye movement coordination during visual tracking of unpredictably moving targets. Neural Netw 27:1–20CrossRefGoogle Scholar
  57. Huang T-R, Grossberg S (2010) Cortical dynamics of contextually cued attentive visual learning and search: spatial and object evidence accumulation. Psychol Rev 117:1080–1112CrossRefGoogle Scholar
  58. Hurvich LM, Jameson D (1957) An opponent-process theory of color vision. Psychol Rev 64: 384–390CrossRefGoogle Scholar
  59. Kelly FJ, Grossberg S (2000) Neural dynamics of 3-D surface perception: figure-ground separation and lightness perception. Percept Psychophys 62:1596–1619CrossRefGoogle Scholar
  60. Mhatre H, Gorchetchnikov A, Grossberg S (2012) Grid cell hexagonal patterns formed by fast self-organized learning within entorhinal cortex. Hippocampus 22:320–334CrossRefGoogle Scholar
  61. Pack C, Grossberg S, Mingolla E (2001) A neural model of smooth pursuit control and motion perception by cortical area MST. J Cogn Neurosci 13:102–120CrossRefGoogle Scholar
  62. Pilly PK, Grossberg S (2012) How do spatial learning and memory occur in the brain? Coordinated learning of entorhinal grid cells and hippocampal place cells. J Cogn Neurosci 24:1031–1054CrossRefGoogle Scholar
  63. Pilly PK, Grossberg S (2014) How does the modular organization of entorhinal grid cells develop? Front Hum Neurosci. doi: 10.3389/fnhum.2014.0037Google Scholar
  64. Schiller PH (1982) Central connections of the retinal ON and OFF pathways. Nature 297:580–583CrossRefGoogle Scholar
  65. Simone G, Farina A, Morabito FC, Serpico SB, Bruzzone L (2002) Image fusion techniques for remote sensing applications. Inf Fusion 3:3–15CrossRefGoogle Scholar
  66. Srihasam K, Bullock D, Grossberg S (2009) Target selection by frontal cortex during coordinated saccadic and smooth pursuit eye movements. J Cogn Neurosci 21:1611–1627CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Department of Mathematics & Center for Adaptive SystemsBoston UniversityBostonUSA
  2. 2.Center for Adaptive SystemsGraduate Program in Cognitive and Neural Systems, Department of Mathematics, Boston UniversityBostonUSA