Neural Processing Letters

, Volume 13, Issue 1, pp 17–30 | Cite as

Efficient Vector Quantization Using the WTA-Rule with Activity Equalization

  • Gunther Heidemann
  • Helge Ritter


We propose a new algorithm for vector quantization, the Activity Equalization Vector quantization (AEV). It is based on the winner takes all rule with an additional supervision of the average node activities over a training interval and a subsequent re-positioning of those nodes with low average activities. The re-positioning is aimed to both an exploration of the data space and a better approximation of already discovered data clusters by an equalization of the node activities. We introduce a learning scheme for AEV which requires as previous knowledge about the data only their bounding box. Using an example of Martinetz et al. [1], AEV is compared with the Neural Gas, Frequency Sensitive Competitive Learning (FSCL) and other standard algorithms. It turns out to converge much faster and requires less computational effort.

clustering codebook generation competitive learning neural gas unsupervised learning vector quantization winner takes all 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Martinetz, T., Berkovich, S. G. and Schulten, K.: `Neural-gas' network for vector quantization and its application to time-series prediction, IEEE Transactions on Neural Networks 4(4) (1993), 558–569.Google Scholar
  2. 2.
    Buhmann, J. and Khnel, H.: Vector quantization with complexity costs, IEEE Trans-actions on Information Theory 39(4) (1993), 1133–1145.Google Scholar
  3. 3.
    Kohonen, T.: Self-organized formation of topologically correct feature maps, Biol. Cybernetics 43 (1982), 59–69.Google Scholar
  4. 4.
    Kohonen, T.: Self-Organizing Maps, Springer-Verlag, Berlin, Heidelberg, New York, 1995.Google Scholar
  5. 5.
    Kohonen, T.: Learning vector quantization, In: M. A. Arbib (ed.), The Handbook of Brain Theory and Neural Networks, MIT Press (1995), pp. 537–540.Google Scholar
  6. 6.
    Grossberg, S.: Competitive learning: From interactive activation to adaptive resonance, Cognitive Sci. 11 (1987), 23–63.Google Scholar
  7. 7.
    Choy, C. S.-T. and Siu, W.-C.: Fast Sequential Implementation of `Neural-Gas' Network for Vector Quantization, IEEE Transactions on Communications 46(3) (1998), 301–304.Google Scholar
  8. 8.
    MacQueen, J.: Some methods for classification and analysis of multivariate obser-vations, In: Proc. 5th Berkeley Symp. Math. Stat. Probab., Vol. 1 (1965), pp. 281–297.Google Scholar
  9. 9.
    Rose, K., Gurewitz, F. and Fox, G.: Statistical mechanics and phase transitions in clustering, Phys. Rev. Lett. 65(8) (1990), 945–948Google Scholar
  10. 10.
    Heidemann, G.: Ein £exibel einsetzbares Objekterkennung r neuronaler Netze. PhD thesis, Universität Bielefeld, Technische Fakultät, Infix, DISKI 190 (1998).Google Scholar
  11. 11.
    Heidemann, G. and Ritter, H.: Combining multiple neural nets for visual feature selection and classification, In: Proceedings of ICANN 99 (1999), pp. 365–370.Google Scholar
  12. 12.
    Ahalt, S. C., Krisnamurthy, A. K., Chen, P. and Melton, D. E.: Competitive learning algorithms for vector quantization, Neural Networks 3 (1990), 277–290.Google Scholar
  13. 13.
    Galanopoulos, A. S. and Ahalt, S. C.: Codeword Distribution for Frequency Sensitive Competitive Learning with One-Dimensional Input Data, IEEE Trans. on Neural Networks 7(3) (1996), 752–756.Google Scholar
  14. 14.
    Galanopoulos, A. S. Moses, R. L. and Ahalt, S. C.: Diffusion approximation of Fre-quency Sensitive Competitive Learning, IEEE Trans. on Neural Networks 8(5) (1997), 1026–1030.Google Scholar
  15. 15.
    Scheunders, P. and De Backer, S.: High-dimensional clustering using frequency sensitive competitive learning, Pattern Recognition 32 (1999), 193–202.Google Scholar
  16. 16.
    Wang, J.-H. and Hsiao, C.-P.: Representation-burden Conservation Network Applied to Learning VQ, Neural Processing Letters 5(3) (1997), 209–217.Google Scholar
  17. 17.
    Wang, J.-H. and Sun, W.-D.: Improved Representation-burden Conservation Network for Learning Non-stationary VQ, Neural Processing Letters 8(1) (1998), 41–53.Google Scholar

Copyright information

© Kluwer Academic Publishers 2001

Authors and Affiliations

  • Gunther Heidemann
    • 1
  • Helge Ritter
    • 1
  1. 1.AG NeuroinformatikUniversität BielefeldGermany

Personalised recommendations