Sentence Understanding and Learning of New Words with Large-Scale Neural Networks

  • Heiner Markert
  • Zöhre Kara Kayikci
  • Günther Palm
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5064)


We have implemented a speech command system which can understand simple command sentences like “Bot lift ball” or “Bot go table” using hidden Markov models (HMMs) and associative memories with sparse distributed representations. The system is composed of three modules: (1) A set of HMMs is used on phoneme level to get a phonetic transcription of the spoken sentence, (2) a network of associative memories is used to determine the word belonging to the phonetic transcription and (3) a neural network is used on the sentence level to determine the meaning of the sentence. The system is also able to learn new object words during performance.


Associative Memories Hidden Markov Models Hebbian Learning Speech Recognition Language Understanding 


  1. 1.
    Rabiner, L., Juang, B.H.: Fundamentals of Speech Recognition. Prentice-Hall, Inc, Englewood Cliffs (1993)Google Scholar
  2. 2.
    Young, S., et al.: The HTK Book (for HTK Version 3.2.1). Cambridge University Engineering Department, Cambridge (2002)Google Scholar
  3. 3.
    Hebb, D.O.: The Organization of Behaviour. John Wiley, New York (1949)Google Scholar
  4. 4.
    Schwenker, F., Sommer, F., Palm, G.: Iterative Retrieval of Sparsely Coded Associative Memory Patterns. Neural Networks 9, 445–455 (1996)CrossRefGoogle Scholar
  5. 5.
    Willshaw, D., Buneman, O., Longuet-Higgins, H.: Non-holographic Associative Memory. Nature 222, 960–962 (1969)CrossRefGoogle Scholar
  6. 6.
    Palm, G.: On Associative Memory. Biological Cybernetics 36, 19–31 (1980)zbMATHCrossRefGoogle Scholar
  7. 7.
    Palm, G.: Memory Capacities of Local Rules for Synaptic Modification. A Comparative Review, Concepts in Neuroscience 2, 97–128 (1991)Google Scholar
  8. 8.
    Knoblauch, A., Palm, G.: Pattern Separation and Synchronization in Spiking Associative Memories and Visual Areas. Neural Networks 14, 763–780 (2001)CrossRefGoogle Scholar
  9. 9.
    TIMIT Acoustic-Phonetic Continuous Speech Corpus. National Institute of Standards and Technology, Speech Disc 1-1.1, NTIS Order No. PB91-505065 (1990)Google Scholar
  10. 10.
    Markert, H., Knoblauch, A., Palm, G.: Modelling of syntactical processing in the cortex. Biosystems 89, 300–315 (2007)CrossRefGoogle Scholar
  11. 11.
    Fay, R., Kaufmann, U., Knoblauch, A., Markert, H., Palm, G.: Combining Visual Attention, Object Recognition and Associative Information Processing in a Neurobotic System. In: Wermter, S., Palm, G., Elshaw, M. (eds.) Biomimetic Neural Learning for Intelligent Robots. LNCS (LNAI), vol. 3575, pp. 118–143. Springer, Heidelberg (2005)Google Scholar
  12. 12.
    Arbib, M.A., Billard, A., Iacoboni, M., Oztop, E.: Synthetic brain imaging: grasping, mirror neurons and imitation. Neural Networks 13(8/9), 931–997 (2000)Google Scholar
  13. 13.
    Roy, D.: Learning visually grounded words and syntax for a scene description task. Comput. Speech Lang. 16(3), 353–385 (2002)CrossRefGoogle Scholar
  14. 14.
    Kirchmar, J.L., Edelman, G.: Machine psychology: autonomous behavior, perceptual categorization and conditioning in a brain-based device. Cereb. Cortex 12(8), 818–830 (2002)CrossRefGoogle Scholar
  15. 15.
    Billard, A., Hayes, G.: DRAMA, a connectionist architecture for control and learning in autonomous robots. Adapt. Behav. J. 7(1), 35–64 (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Heiner Markert
    • 1
  • Zöhre Kara Kayikci
    • 1
  • Günther Palm
    • 1
  1. 1.Institute of Neural Information ProcessingUniversity of UlmUlmGermany

Personalised recommendations