Sampling Representative Examples for Dimensionality Reduction and Recognition – Bootstrap Bumping LDA

  • Hui Gao
  • James W. Davis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3953)


We present a novel method for dimensionality reduction and recognition based on Linear Discriminant Analysis (LDA), which specifically deals with the Small Sample Size (SSS) problem in Computer Vision applications. Unlike the traditional methods, which impose specific assumptions to address the SSS problem, our approach introduces a variant of bootstrap bumping technique, which is a general framework in statistics for model search and inference. An intermediate linear representation is first hypothesized from each bootstrap sample. Then LDA is performed in the reduced subspace. Lastly, the final model is selected among all hypotheses for the best classification. Experiments on synthetic and real datasets demonstrate the advantages of our Bootstrap Bumping LDA (BB-LDA) approach over the traditional LDA based methods.


Bootstrap Sample Sampling Ratio Quadratic Discriminant Analysis Gait Recognition Computer Vision Application 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Jain, A., Duin, R., Mao, J.: Statistical pattern recognition: a review. IEEE Trans. Patt. Analy. and Mach. Intell. 22(1), 4–37 (2000)CrossRefGoogle Scholar
  2. 2.
    Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Trans. Patt. Analy. and Mach. Intell. 19(7), 711–720 (1997)CrossRefGoogle Scholar
  3. 3.
    Liu, C., Wechsler, H.: Enhanced Fisher linear discriminant models for face recognition. In: Proc. Int. Conf. Pat. Rec., pp. 1368–1372. IEEE, Los Alamitos (1998)Google Scholar
  4. 4.
    Cui, Y., Swets, D., Weng, J.: Learning-based hand sign recognition using SHOSLIF-M. In: Proc. Int. Conf. Comp. Vis., pp. 631–636. IEEE, Los Alamitos (1995)Google Scholar
  5. 5.
    Huang, P., Harris, C., Nixon, M.: Human gait recognition in canonical space using temporal templates. In: Proc. Vision Image Signal Process, vol. 146, pp. 93–100. IEE (1999)Google Scholar
  6. 6.
    Krzanowski, W., Jonathan, P., McCarthy, W., Thomas, M.: Discriminant analysis with singular covariance matrices: methods and applications to spectroscopic data. Applied Statistics 44, 101–115 (1995)MATHCrossRefGoogle Scholar
  7. 7.
    Yu, H., Yang, J.: A direct LDA algorithm for high-dimensional data - with application to face recognition. Pattern Recognition 34, 2067–2070 (2001)MATHCrossRefGoogle Scholar
  8. 8.
    Friedman, J.: Regularized discriminant analysis. J. Am. Statistical Assoc. 84(405), 165–175 (1989)CrossRefGoogle Scholar
  9. 9.
    Tibshirani, R., Knight, K.: Model search by bootstrap “bumping”. J. of Computational and Graphical Statistics 8(4), 671–686 (1999)CrossRefGoogle Scholar
  10. 10.
    Fisher, R.: The use of multiple measurements in taxonomic problems. Annals of Eugenics 7(Part II), 179–188 (1936)Google Scholar
  11. 11.
    Rao, C.: The utilization of multiple measurements in problems of biological classification. J. Royal Statistical Soc., B 10, 159–203 (1948)MATHGoogle Scholar
  12. 12.
    Campbell, N.: Canonical variate analysis - a general model formulation. Australian J. Statistics 26, 86–96 (1984)MATHCrossRefGoogle Scholar
  13. 13.
    Duda, R., Hart, P., Stork, D.: Pattern Classification. John Wiley & Sons, New York (2001)MATHGoogle Scholar
  14. 14.
    Gao, H., Davis, J.: Why Direct LDA is not equivalent to LDA. In Pattern Recognition (2006) (to appear)Google Scholar
  15. 15.
    Baudat, G., Anouar, F.: Generalized discriminant analysis using a kernel approach. Neural Computation 12(10), 2385–2404 (2000)CrossRefGoogle Scholar
  16. 16.
    Kim, T., Kittler, J.: Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image. IEEE Trans. Patt. Analy. and Mach. Intell. 27(3), 318–327 (2005)CrossRefGoogle Scholar
  17. 17.
    Torre, F., Kanade, T.: Oriented discriminant analysis (ODA). In: Brit. Mach. Vis. Conf., pp. 132–141 (2004)Google Scholar
  18. 18.
    Liu, X., Srivastava, A., Gallivan, K.: Optimal linear representations of images for object recognition. IEEE Trans. Patt. Analy. and Mach. Intell. 26(5), 662–666 (2004)CrossRefGoogle Scholar
  19. 19.
    Efron, B.: Bootstrap methods: another look at the jackknife. Annals of Statistics 7, 1–26 (1979)MATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Breiman, L.: Bagging predictors. Machine Learning Journal 24(2), 123–140 (1996)MATHMathSciNetGoogle Scholar
  21. 21.
    Schapire, R.: The strength of weak learnability. Machine Learning 5(2), 197–227 (1990)Google Scholar
  22. 22.
    Freund, Y.: Boosting a weak learning algorithm by majority. Information and Computation 121(2), 256–285 (1995)MATHCrossRefMathSciNetGoogle Scholar
  23. 23.
    Freund, Y., Schapire, R.: Experiments with a new boosting algorithm. In: Machine Learning: Proc. of the 13th Int. Conf., pp. 148–156 (1996)Google Scholar
  24. 24.
    Viola, P., Jones, M., Snow, D.: Detecting pedestrians using patterns of motion and appearance. In: Proc. Int. Conf. Comp. Vis., pp. 734–741 (2003)Google Scholar
  25. 25.
    Skurichina, M., Duin, R.: Bagging, boosting and the random subspace method for linear classifiers. Pattern Analysis & Applications 5, 121–135 (2002)MATHCrossRefMathSciNetGoogle Scholar
  26. 26.
    Lu, X., Jain, A.K.: Resampling for face recognition. In: Int. Conf. on Audio and Video Based Biometric Person Auth., pp. 869–877 (2003)Google Scholar
  27. 27.
    Efron, B., Tibshirani, R.: An Introduction to the Bootstrap. Chapman and Hall, New York (1993)MATHGoogle Scholar
  28. 28.
    Samaria, F., Harter, A.: Parameterisation of a stochastic model for human face identification. In: 2nd IEEE Workshop on Applications of Computer Vision (1994)Google Scholar
  29. 29.
    Gross, R., Shi, J.: The CMU motion of body (MoBo) database. Technical Report CMU-RITR-01-18, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA (2001)Google Scholar
  30. 30.
    Davis, J., Bobick, A.: The representation and recognition of action using temporal templates. In: Comp. Vis. and Pattern Rec., pp. 928–93421. IEEE, Los Alamitos (1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Hui Gao
    • 1
  • James W. Davis
    • 1
  1. 1.Dept. of Computer Science and EngineeringThe Ohio State UniversityColumbusUSA

Personalised recommendations