Unsupervised coding with lococode

  • Sepp Hochreiter
  • Jürgen Schmidhuber
Part IV: Signal Processing: Blind Source Separation, Vector Quantization, and Self Organization

DOI: 10.1007/BFb0020229

Volume 1327 of the book series Lecture Notes in Computer Science (LNCS)
Cite this paper as:
Hochreiter S., Schmidhuber J. (1997) Unsupervised coding with lococode. In: Gerstner W., Germond A., Hasler M., Nicoud JD. (eds) Artificial Neural Networks — ICANN'97. ICANN 1997. Lecture Notes in Computer Science, vol 1327. Springer, Berlin, Heidelberg

Abstract

Traditional approaches to sensory coding use code component-oriented objective functions (COCOFs) to evaluate code quality. Previous COCOFs do not take into account the information-theoretic complexity of the code-generating mapping itself. We do: “Low-complexity coding and decoding” (LOCOCODE) generates so-called lococodes that (1) convey information about the input data, (2) can be computed from the data by a low-complexity mapping (LCM), and (3) can be decoded by a LCM. We implement LococoDE by training autoassociators with Flat Minimum Search (FMS), a general method for finding lowcomplexity neural nets. LococoDE extracts optimal codes for difficult versions of the “bars” benchmark problem. As a preprocessor for a vowel recognition benchmark problem it sets the stage for excellent classification performance.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag 1997

Authors and Affiliations

  • Sepp Hochreiter
    • 1
    • 2
  • Jürgen Schmidhuber
    • 1
    • 2
  1. 1.Technische Universität MünchenMünchenGermany
  2. 2.IDSIALuganoSwitzerland