Advertisement

An algorithm for intrinsic dimensionality estimation

  • J. Bruske
  • G. Sommer
Pattern Analysis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1296)

Abstract

In this paper a new method for analyzing the intrinsic dimensionality (ID) of low dimensional manifolds in high dimensional feature spaces is presented. The basic idea is to first extract a low-dimensional representation that captures the intrinsic topological structure of the input data and then to analyze this representation, i.e. to estimate the intrinsic dimensionality. Compared to previous approaches based on 1ocal PCA the method has a number of important advantages: First, it can be shown to have only linear time complexity w.r.t. the dimensionality of the input space (in contrast to the cubic complexity of the conventional approach) and hence becomes applicable even for very high dimensional input spaces. Second, it is less sensitive to noise than former approaches, and, finally, the extracted representation can be directly used for further data processing tasks including auto-association and classification.

The presented method for ID estimation is illustrated on a synthetic data set. It has also been successfully applied to ID estimation of full scale image sequences, see [BS97].

Keywords

Quantization Error High Dimensional Feature Space Linear Time Complexity Topology Preservation Local Subspace 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [Ben69]
    R. S. Bennett. The intrinsic dimensionality of signal collections. IEEE Transactions on Information Theory, 15:517–525, 1969.CrossRefGoogle Scholar
  2. [BS97]
    J. Bruske and G. Sommer. Intrinsic dimensionality estimation with optimally topology preserving maps. Technical Report 9703, Inst. f. Inf. u. Prakt. Math. Christian-Albrechts-Universitaet zu Kiel, 1997. (submitted to IEEE PAMI).Google Scholar
  3. [FO71]
    K. Fukunaga and D. R. Olsen. An algorithm for finding intrinsic dimensionality of data. IEEE Transactions on Computers, 20(2):176–183, 1971.Google Scholar
  4. [JD88]
    A. K. Jain and R. C. Dubes. Algorithms for Clustering Data. Prentince Hall, 1988.Google Scholar
  5. [KL94]
    N. Kambhatla and T.K. Leen. Fast non-linear dimension reduction. In Advances in Neural Information Processing Systems, NIPS 6, pages 152–159, 1994.Google Scholar
  6. [Kru64]
    J. B. Kruskal. Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika, 29:1–27, 1964.Google Scholar
  7. [LBG80]
    Y. Linde, A. Buzo and R.m. Gray. An algorithm for vector quantizer design. IEEE Transaction on Communications, 28(1):84–95, 1980.CrossRefGoogle Scholar
  8. [MS94]
    T. Martinetz and K. Schulten. Topology representing networks. In Neural Networks, volume 7. pages 505–522, 1994.CrossRefGoogle Scholar
  9. [PBJD79]
    K. Pettis, T. Bailey. T. Jain, and R. Dubes. An intrinsic dimensionality estimator from near-neighbor information. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI, 1:25–37, 1979.Google Scholar
  10. [PTVF88]
    W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery. Numerical Recipes in C-The Art of Scientific Computing. Cambridge University Press, 1988.Google Scholar
  11. [Tru76]
    G. V. Trunk. Statistical estimation of the intrinsic dimensionality of a noisy signal collection. IEEE Transactions on Computers, 25:165–171, 1976.Google Scholar
  12. [VD95]
    P. J. Verveer and R. P.W. Duin. An evaluation of intrinsic dimensionality estimators. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI, 17(1):81–86, 1995.CrossRefGoogle Scholar
  13. [VDNI94]
    T. Villmann, R. Der. and T. vlartinetz. A novel aproach to measure the topology preservation of feature maps. ICANN, pages 289–301, 1994.Google Scholar
  14. [Zad82]
    P. L. Zador. Asymptotic quantization error of continuous signals and the quantization dimension. IEEE Transactions on Information Theory, 28(2):139–149, 1982.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1997

Authors and Affiliations

  • J. Bruske
    • 1
  • G. Sommer
    • 1
  1. 1.Computer Science InstituteChristian-Albrechts University zu KielGermany

Personalised recommendations