Brain Topography

, Volume 27, Issue 4, pp 565–577 | Cite as

Predictive Regularity Representations in Violation Detection and Auditory Stream Segregation: From Conceptual to Computational Models

  • Erich Schröger
  • Alexandra Bendixen
  • Susan L. Denham
  • Robert W. Mill
  • Tamás M. Bőhm
  • István Winkler


Predictive accounts of perception have received increasing attention in the past 20 years. Detecting violations of auditory regularities, as reflected by the Mismatch Negativity (MMN) auditory event-related potential, is amongst the phenomena seamlessly fitting this approach. Largely based on the MMN literature, we propose a psychological conceptual framework called the Auditory Event Representation System (AERS), which is based on the assumption that auditory regularity violation detection and the formation of auditory perceptual objects are based on the same predictive regularity representations. Based on this notion, a computational model of auditory stream segregation, called CHAINS, has been developed. In CHAINS, the auditory sensory event representation of each incoming sound is considered for being the continuation of likely combinations of the preceding sounds in the sequence, thus providing alternative interpretations of the auditory input. Detecting repeating patterns allows predicting upcoming sound events, thus providing a test and potential support for the corresponding interpretation. Alternative interpretations continuously compete for perceptual dominance. In this paper, we briefly describe AERS and deduce some general constraints from this conceptual model. We then go on to illustrate how these constraints are computationally specified in CHAINS.


Auditory object Auditory scene analysis Deviance detection Predictive modelling Mismatch negativity (MMN) 



This research was supported by the Hungarian Academy of Sciences (Lendület project, LP2012-36/2012 to IW), by the Reinhart Koselleck grant of the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG, SCH 375/20-1 to ES), by the DFG Cluster of Excellence 1077 “Hearing4all”, by the German Academic Exchange Service (Deutscher Akademischer Austauschdient, DAAD, Project 56265741), and by the Hungarian Scholarship Board (Magyar Ösztöndíj Bizottság, MÖB, Project 39589).


  1. Althen H, Grimm S, Escera C (2013) Simple and complex acoustic regularities are encoded at different levels of the auditory hierarchy. Eur J Neurosci 38(10):3448–3455Google Scholar
  2. Baldeweg T (2007) ERP repetition effects and mismatch negativity generation—A predictive coding perspective. J Psychophysiol 21(3–4):204–213CrossRefGoogle Scholar
  3. Beauvois MW, Meddis R (1991) A computer-model of auditory stream segregation. Q J Exp Psychol Sect A 43(3):517–541CrossRefGoogle Scholar
  4. Bendixen A, Roeber U, Schröger E (2007) Regularity extraction and application in dynamic auditory stimulus sequences. J Cogn Neurosci 19(10):1664–1677PubMedCrossRefGoogle Scholar
  5. Bendixen A, Prinz WG, Horváth J, Trujillo-Barreto NJ, Schröger E (2008) Rapid extraction of auditory feature contingencies. Neuroimage 41(3):1111–1119PubMedCrossRefGoogle Scholar
  6. Bendixen A, Schröger E, Winkler I (2009) I heard that coming: event-related potential evidence for stimulus-driven prediction in the auditory system. J Neurosci 29(26):8447–8451PubMedCrossRefGoogle Scholar
  7. Bendixen A, Denham SL, Gyimesi K, Winkler I (2010) Regular patterns stabilize auditory streams. J Acoust Soc Am 128(6):3658–3666PubMedCrossRefGoogle Scholar
  8. Bendixen A, SanMiguel I, Schröger E (2012a) Early electrophysiological indicators for predictive processing in audition: a review. Int J Psychophysiol 83(2):120–131PubMedCrossRefGoogle Scholar
  9. Bendixen A, Schröger E, Ritter W, Winkler I (2012b) Regularity extraction from non-adjacent sounds. Frontiers Psychol 3:143CrossRefGoogle Scholar
  10. Bendixen A, Bőhm TM, Szalárdy O, Mill R, Denham SL, Winkler I (2013) Different roles of similarity and predictability in auditory stream segregation. Learn Percept 5:37–54CrossRefGoogle Scholar
  11. Boh B, Herholz SC, Lappe C, Pantev C (2011) Processing of complex auditory patterns in musicians and nonmusicians. PLoS One 6(7):e21458PubMedCentralPubMedCrossRefGoogle Scholar
  12. Bregman AS (1990) Auditory scene analysis. The perceptual organization of sound. MIT PressMA, CambridgeGoogle Scholar
  13. Costa-Faidella J, Grimm S, Slabu L, Diaz-Santaella F, Escera C (2011) Multiple time scales of adaptation in the auditory system as revealed by human evoked potentials. Psychophysiology 48(6):774–783PubMedCrossRefGoogle Scholar
  14. Cowan N (1984) On short and long auditory stores. Psychol Bull 96(2):341–370PubMedCrossRefGoogle Scholar
  15. Cowan N, Winkler I, Teder W, Näätänen R (1993) Memory prerequisites of mismatch negativity in the auditory event-related potential (ERP). J Exp Psychol-Learn Mem Cogn 19(4):909–921PubMedCrossRefGoogle Scholar
  16. Deike S, Heil P, Böckmann-Barthel M, Brechmann A (2012) The build-up of auditory stream segregation: a different perspective. Frontiers Psychol 3:461CrossRefGoogle Scholar
  17. Denham SL, Gyimesi K, Stefanics G, Winkler I (2013) Perceptual bistability in auditory streaming: how much do stimulus features matter? Learn Percept 5(2):73–100CrossRefGoogle Scholar
  18. Elhilali M, Shamma SA (2008) A cocktail party with a cortical twist: how cortical mechanisms contribute to sound segregation. J Acoust Soc Am 124(6):3751–3771PubMedCentralPubMedCrossRefGoogle Scholar
  19. Escera C, Leung S, Grimm S (2013) Deviance detection based on regularity encoding along the auditory hierarchy: electrophysiological evidence in humans. Brain Topogr. doi: 10.1007/s10548-013-0328-4
  20. Fowler CA, Rosenblum LD (1990) Duplex perception—a comparison of monosyllables and slamming doors. J Exp Psychol-Hum Percep Perform 16(4):742–754CrossRefGoogle Scholar
  21. Friston K, Kiebel S (2009a) Cortical circuits for perceptual inference. Neural Netw 22(8):1093–1104PubMedCentralPubMedCrossRefGoogle Scholar
  22. Friston K, Kiebel S (2009b) Predictive coding under the free-energy principle. Philos Trans R Soc Lond B 364(1521):1211–1221CrossRefGoogle Scholar
  23. Garrido MI, Kilner JM, Stephan KE, Friston KJ (2009) The mismatch negativity: a review of underlying mechanisms. Clin Neurophysiol 120(3):453–463PubMedCentralPubMedCrossRefGoogle Scholar
  24. Griffiths TD, Warren JD (2004) Opinion: what is an auditory object? Nat Review Neurosci 5:887–892CrossRefGoogle Scholar
  25. Grimm S, Escera C (2012) Auditory deviance detection revisited: evidence for a hierarchical novelty system. Int J Psychophysiol 85(1):88–92PubMedCrossRefGoogle Scholar
  26. Grossberg S, Govindarajan KK, Wyse LL, Cohen MA (2004) ARTSTREAM: a neural network model of auditory scene analysis and source segregation. Neural Netw 17(4):511–536PubMedCrossRefGoogle Scholar
  27. Haenschel C, Vernon DJ, Dwivedi P, Gruzelier JH, Baldeweg T (2005) Event-related brain potential correlates of human auditory sensory memory-trace formation. J Neurosci 25(45):10494–10501PubMedCrossRefGoogle Scholar
  28. Helmholtz Hv (1867) Handbuch der physiologischen Optik. Allgemeine Encyklopädie der Physik, vol Bd 9. Voss, LeipzigGoogle Scholar
  29. Herrmann CS, Munk MHJ, Engel AK (2004) Cognitive functions of gamma-band activity: memory match and utilization. Trends Cogn Sci 8(8):347–355PubMedCrossRefGoogle Scholar
  30. Horváth J, Czigler I, Sussman E, Winkler I (2001) Simultaneously active pre-attentive representations of local and global rules for sound sequences in the human brain. Cogn Brain Res 12(1):131–144CrossRefGoogle Scholar
  31. Horváth J, Winkler I, Bendixen A (2008) Do N1/MMN, P3a, and RON form a strongly coupled chain reflecting the three stages of auditory distraction? Biol Psychol 79(2):139–147PubMedCrossRefGoogle Scholar
  32. Jacobs AM, Grainger J (1994) Models of visual word recognition—sampling the state-of-the-art. J Exp Psychol-Hum Percep Perform 20(6):1311–1334CrossRefGoogle Scholar
  33. Jones MR (1976) Time, our lost dimension: toward a new theory of perception, attention, and memory. Psychol Rev 83(5):323–355PubMedCrossRefGoogle Scholar
  34. Kiebel SJ, von Kriegstein K, Daunizeau J, Friston KJ (2009) Recognizing sequences of sequences. PLoS Comput Biol 5(8):e1000464PubMedCentralPubMedCrossRefGoogle Scholar
  35. Köhler W (1947) Gestalt psychology: an introduction to new concepts in modern psychology. Liveright Publishing, New YorkGoogle Scholar
  36. Kubovy M, Van Valkenburg D (2001) Auditory and visual objects. Cognition 80:97–126PubMedCrossRefGoogle Scholar
  37. Lieder F, Daunizeau J, Garrido MI, Friston KJ, Stephan KE (2013) Modelling trial-by-trial changes in the mismatch negativity. PLoS Comput Biol 9(2):e1002911PubMedCentralPubMedCrossRefGoogle Scholar
  38. Massaro DW (1972) Preperceptual images, processing time, and perceptual units in auditory perception. Psychol Rev 79:124–145PubMedCrossRefGoogle Scholar
  39. McCabe SL, Denham MJ (1997) A model of auditory streaming. J Acoust Soc Am 101(3):1611–1621CrossRefGoogle Scholar
  40. Mill R, Bőhm T, Bendixen A, Winkler I, Denham SL CHAINS—Competition and cooperation between fragmentary event predictors in a model of auditory scene analysis. In: Information Sciences and Systems (CISS), 2011, IEEE 45th Annual Conference on, Baltimore, pp 1–6, 2011Google Scholar
  41. Mill RW, Bőhm TM, Bendixen A, Winkler I, Denham SL (2013) Modelling the emergence and dynamics of perceptual organisation in auditory streaming. PLoS Comput Biol 9(3):e1002925PubMedCentralPubMedCrossRefGoogle Scholar
  42. Moore BCJ, Gockel H (2002) Factors influencing sequential stream segregation. Acta Acust United Acust 88(3):320–333Google Scholar
  43. Moore BCJ, Gockel HE (2012) Properties of auditory stream formation. Philos Trans Royal Soc B 367(1591):919–931CrossRefGoogle Scholar
  44. Müller D, Widmann A, Schröger E (2005) Deviance-repetition effects as a function of stimulus feature, feature value variation, and timing: a mismatch negativity study. Biol Psychol 68(1):1–14PubMedCrossRefGoogle Scholar
  45. Mumford D (1992) On the computational architecture of the neocortex II. The role of cortico-cortical loops. Biol Cybern 66(3):241–251PubMedCrossRefGoogle Scholar
  46. Näätänen R (1990) The Role of attention in auditory information-processing as revealed by event-related potentials and other brain measures of cognitive function. Behav Brain Sci 13(2):201–232CrossRefGoogle Scholar
  47. Näätänen R, Winkler I (1999) The concept of auditory stimulus representation in cognitive neuroscience. Psychol Bull 125(6):826–859PubMedCrossRefGoogle Scholar
  48. Näätänen R, Gaillard A, Mäntysalo S (1978) Early selective-attention effect on evoked potential reinterpreted. Acta Psychol 42:313–329CrossRefGoogle Scholar
  49. Näätänen R, Kujala T, Winkler I (2011) Auditory processing that leads to conscious perception: a unique window to central auditory processing opened by the mismatch negativity and related responses. Psychophysiology 48(1):4–22PubMedCrossRefGoogle Scholar
  50. Neisser U (1967) Cognitive psychology. Appleton-Century-Crofts, New YorkGoogle Scholar
  51. Paavilainen P, Arajärvi P, Takegata R (2007) Preattentive detection of nonsalient contingencies between auditory features. NeuroReport 18(2):159–163PubMedCrossRefGoogle Scholar
  52. Rao RPN, Ballard DH (1999) Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat Neurosci 2(1):79–87PubMedCrossRefGoogle Scholar
  53. Rensink RA (2000) Seeing, sensing, and scrutinizing. Vision Res 40(10–12):1469–1487PubMedCrossRefGoogle Scholar
  54. Rinne T, Särkkä A, Degerman A, Schröger E, Alho K (2006) Two separate mechanisms underlie auditory change detection and involuntary control of attention. Brain Res 1077:135–143PubMedCrossRefGoogle Scholar
  55. Ritter W, Sussman E, Molholm S (2000) Evidence that the mismatch negativity system works on the basis of objects. NeuroReport 11(1):61–63PubMedCrossRefGoogle Scholar
  56. Ritter W, De Sanctis P, Molholm S, Javitt DC, Foxe JJ (2006) Preattentively grouped tones do not elicit MMN with respect to each other. Psychophysiology 43(5):423–430PubMedCentralPubMedCrossRefGoogle Scholar
  57. Schröger E (2007) Mismatch negativity - A microphone into auditory memory. J Psychophysiol 21(3–4):138–146CrossRefGoogle Scholar
  58. Schwartz JL, Grimault N, Hupé J-M, Moore BC, Pressnitzer D (2012) Multistability in perception: binding sensory modalities, an overview. Philos Trans R Soc Lond B 367(1591):896–905CrossRefGoogle Scholar
  59. Sokolov EN (1963) Higher nervous functions: the orienting reflex. Annu Rev Physiol 25:545–580PubMedCrossRefGoogle Scholar
  60. Sussman ES (2005) Integration and segregation in auditory scene analysis. J Acoust Soc Am 117(3 Pt 1):1285–1298PubMedCrossRefGoogle Scholar
  61. Sussman ES, Gumenyuk V (2005) Organization of sequential sounds in auditory memory. NeuroReport 16(13):1519–1523PubMedCrossRefGoogle Scholar
  62. Sussman ES, Horváth J, Winkler I, Orr M (2007) The role of attention in the formation of auditory streams. Perc Psychophys 69(1):136–152CrossRefGoogle Scholar
  63. Szalárdy O, Winkler I, Schröger E, Widmann A, Bendixen A (2013) Foreground-background discrimination indicated by event-related brain potentials in a new auditory multistability paradigm. Psychophysiology 12:1239–1250Google Scholar
  64. van Noorden LPAS (1975) Temporal coherence in the perception of tone sequences. Technical University, EindhovenGoogle Scholar
  65. Wacongne C, Labyt E, van Wassenhove V, Bekinschtein T, Naccache L, Dehaene S (2011) Evidence for a hierarchy of predictions and prediction errors in human cortex. Proc Natl Acad Sci USA 108(51):20754–20759PubMedCentralPubMedCrossRefGoogle Scholar
  66. Wacongne C, Changeux JP, Dehaene S (2012) A neuronal model of predictive coding accounting for the mismatch negativity. J Neurosci 32(11):3665–3678PubMedCrossRefGoogle Scholar
  67. Wang DL, Chang P (2008) An oscillatory correlation model of auditory streaming. Cogn Neurodyn 2(1):7–19PubMedCentralPubMedCrossRefGoogle Scholar
  68. Winkler I (2007) Interpreting the mismatch negativity. J Psychophysiol 21(3–4):147–163CrossRefGoogle Scholar
  69. Winkler I, Czigler I (1998) Mismatch negativity: deviance detection or the maintenance of the ‘standard’. NeuroReport 9(17):3809–3813PubMedCrossRefGoogle Scholar
  70. Winkler I, Czigler I (2012) Evidence from auditory and visual event-related potential (ERP) studies of deviance detection (MMN and vMMN) linking predictive coding theories and perceptual object representations. Int J Psychophysiol 83(2):132–143PubMedCrossRefGoogle Scholar
  71. Winkler I, Karmos G, Näätänen R (1996) Adaptive modeling of the unattended acoustic environment reflected in the mismatch negativity event-related potential. Brain Res 742(1–2):239–252PubMedCrossRefGoogle Scholar
  72. Winkler I, Korzyukov O, Gumenyuk V, Cowan N, Linkenkaer-Hansen K, Ilmoniemi RJ, Alho K, Näätänen R (2002) Temporary and longer term retention of acoustic information. Psychophysiology 39(4):530–534PubMedCrossRefGoogle Scholar
  73. Winkler I, Takegata R, Sussman E (2005) Event-related brain potentials reveal multiple stages in the perceptual organization of sound. Cogn Brain Res 25(1):291–299CrossRefGoogle Scholar
  74. Winkler I, van Zuijen TL, Sussman E, Horváth J, Näätänen R (2006) Object representation in the human auditory system. Eur J Neurosci 24(2):625–634PubMedCentralPubMedCrossRefGoogle Scholar
  75. Winkler I, Denham SL, Nelken I (2009) Modeling the auditory scene: predictive regularity representations and perceptual objects. Trends Cogn Sci 13(12):532–540PubMedCrossRefGoogle Scholar
  76. Winkler I, Denham S, Mill R, Bőhm TM, Bendixen A (2012) Multistability in auditory stream segregation: a predictive coding view. Philos Trans Royal Soc B 367(1591):1001–1012CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Erich Schröger
    • 1
  • Alexandra Bendixen
    • 1
    • 2
  • Susan L. Denham
    • 3
  • Robert W. Mill
    • 4
  • Tamás M. Bőhm
    • 5
  • István Winkler
    • 5
    • 6
  1. 1.Institute of PsychologyUniversity of LeipzigLeipzigGermany
  2. 2.Auditory Psychophysiology Lab, Department of Psychology, Cluster of Excellence “Hearing4all”, European Medical SchoolCarl von Ossietzky UniversityOldenburgGermany
  3. 3.Cognition Institute and School of PsychologyUniversity of PlymouthPlymouthUK
  4. 4.MRC Institute of Hearing ResearchNottinghamUK
  5. 5.Institute of Cognitive Neuroscience and Psychology, Research Centre for Natural SciencesHungarian Academy of SciencesBudapestHungary
  6. 6.Institute of PsychologyUniversity of SzegedSzegedHungary

Personalised recommendations