Advertisement

Relative Stability of Random Projection-Based Image Classification

  • Ewa Skubalska-RafajłowiczEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10841)

Abstract

Our aim is to show that randomly generated transformation of high-dimensional data vectors, for example, images, could provide low dimensional features which are stable and suitable for classification tasks. We examine two types of projections: (a) global random projections, i.e., projections of the whole images, and (b) concatenated local projections of spatially-organized parts of an image (for example rectangular image blocks). In both cases, the transformed images provide good features for correct classification. The computational complexity of designing the transformation is linear with respect to the size of images and in case (b) it does not depend on the form of image partition. We have analyzed the stability of classification results with respect to random projection and to different randomly generated training sets. Experiments on the images of ten persons taken from the Extended Yale Database B demonstrate that the methods of classification based on Gaussian random projection are effective and positively comparable with PCA-based methods, both from the point of view of stability and classification accuracy.

Notes

Acknowledgments

This research was supported by grant 041/0145/17 at the Faculty of Electronics, Wrocław University of Science and Technology.

References

  1. 1.
    Achlioptas, D.: Database-friendly random projections: Johnson-Lindenstrauss with binary coins. J. Comput. Syst. Sci. 66, 671–687 (2003)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Ailon, N., Chazelle, B.: The fast Johnson-Lindenstrauss transform and approximate nearest neighbors. SIAM J. Comput. 39(1), 302–322 (2009)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Amador, J.J.: Random projection and orthonormality for lossy image compression. Image Vis. Comput. 25, 754–766 (2007)CrossRefGoogle Scholar
  4. 4.
    Baraniuk, R., Davenport, M., DeVore, R., Wakin, M.: A simple proof of the restricted isometry property for random matrices. Constr. Approx. 28(3), 253–263 (2008)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Breiman, L.: Arcing classifiers. Ann. Stat. 26(3), 801–849 (1998)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefGoogle Scholar
  7. 7.
    Briand, B., Ducharme, G.R., Parache, V., Mercat-Rommens, C.: A similarity measure to assess the stability of classification trees. Comput. Stat. Data Anal. 53(4), 1208–1217 (2009)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Brigham, E., Mannila, H.: Random projection in dimensionality reduction: applications to image and text data. In: Proceedings of the Conference on Knowledge Discovery and Data Mining, vol. 16, pp. 245–250 (2001)Google Scholar
  9. 9.
    Fodor, I.K.: A survey of dimension reduction techniques. Technical report, Lawrence Livermore National Lab., CA (US) (2002)Google Scholar
  10. 10.
    Du, Q., Fowler, J.E.: Low-complexity principal component analysis for hyperspectral image compression. Int. J. High Perform. Comput. Appl. 22, 438–448 (2008)CrossRefGoogle Scholar
  11. 11.
    Frankl, P., Maehara, H.: Some geometric applications of the beta distribution. Ann. Inst. Stat. Math. 42(3), 463–474 (1990)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Fowler, J.E., Du, Q.: Anomaly detection and reconstruction from random projections. IEEE Trans. Image Process. 21(1), 184–195 (2012)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Gottmukkal, R., Asari, V.K.: An improved face recognition technique based on modular PCA approach. Pattern Recogn. Lett. 24(4), 429–436 (2004)CrossRefGoogle Scholar
  14. 14.
    Georghiades, A.S., Belhumeur, P.N., Kriegman, D.J.: From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell. 21(6), 643–660 (2001)CrossRefGoogle Scholar
  15. 15.
    James, G., Witten, D., Hastie, T., Tibshirani, R.: An Introduction to Statistical Learning. Springer, New York (2013).  https://doi.org/10.1007/978-1-4614-7138-7CrossRefzbMATHGoogle Scholar
  16. 16.
    Jeong, K., Principe, J.C.: Enhancing the correntropy MACE filter with random projections. Neurocomputing 72(1–2), 102–111 (2008)CrossRefGoogle Scholar
  17. 17.
    Jolliffe, I.: Principal Component Analysis, 2nd edn. Springer, NewYork (2002).  https://doi.org/10.1007/b98835CrossRefzbMATHGoogle Scholar
  18. 18.
    Johnson, W.B., Lindenstrauss, J.: Extensions of Lipshitz mapping into Hilbert space. Contemp. Math. 26, 189–206 (1984)CrossRefGoogle Scholar
  19. 19.
    Lee, K.-C., Ho, J., Driegman, D.: Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 684–698 (2005)CrossRefGoogle Scholar
  20. 20.
    Matouŝek, J.: On variants of the Johnson-Lindenstrauss lemma. Random Struct. Algorithms 33(2), 142–156 (2008)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Marzetta, T.L., Tucci, G.H., Simon, S.H.: A random matrix-theoretic approach to handling singular covariance estimates. IEEE Trans. Inf. Theory 57, 6256–6271 (2011)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Ng, A.Y., Jordan, M.I.: On discriminative vs. generative classifiers: a comparison of logistic regression and Naive Bayes. In: Advances in Neural Information Processing Systems, vol. 14, pp. 841–848 (2002)Google Scholar
  23. 23.
    Skubalska-Rafajłowicz, E.: Random projections and Hotelling’s T 2 statistics for change detection in high-dimensional data streams. Int. J. Appl. Math. Comput. Sci. 23(2), 447–461 (2013)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Skubalska-Rafajłowicz, E.: Neural networks with sigmoidal activation functions – dimension reduction using normal random projection. Nonlinear Anal.: Theory Methods Appl. 71(12), e1255–e1263 (2009)CrossRefGoogle Scholar
  25. 25.
    Skubalska-Rafajłowicz, E.: Spatially-organized random projections of images for dimensionality reduction and privacy-preserving classification. In: Proceedings of 10th International Workshop on Multidimensional (nD) Systems (nDS), pp. 1–5 (2017)Google Scholar
  26. 26.
    Steinwart, I., Christmann, A.: Support Vector Machines. Springer, New York (2008).  https://doi.org/10.1007/978-0-387-77242-4CrossRefzbMATHGoogle Scholar
  27. 27.
    Tsagkatakis, G., Savakis, A.: A random projections model for object tracking under variable pose and multi-camera views. In: Proceedings of the Third ACM/IEEE International Conference on Distributed Smart Cameras, ICDSC, pp. 1–7 (2009)Google Scholar
  28. 28.
    Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–86 (1991)CrossRefGoogle Scholar
  29. 29.
    Vempala, S.: The Random Projection Method. American Mathematical Society, Providence (2004)zbMATHGoogle Scholar
  30. 30.
    Yang, J., Zhang, D., Frangi, A.F., Yang, J.: Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 131–137 (2004)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer Engineering, Faculty of ElectronicsWrocław University of Science and TechnologyWrocławPoland

Personalised recommendations