Skip to main content
SpringerLink
Log in

Feature Representation and Learning

  • Chapter
  • First Online:
Sparse Representation, Modeling and Learning in Visual Recognition

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

  • 2246 Accesses

Abstract

In this chapter, first, some basics concepts about feature extraction and how to use sparse coding for feature representation and dimension reduction are detailed. Then it gives the concepts of dictionary learning methods including K-SVD, discriminative dictionary learning, online dictionary learning, supervised dictionary learning, and joint dictionary leaning and some applications about dictionary learning. Lastly, it includes some works about feature learning which are dimension reduction, sparse support vector machine, recursive feature elimination, MSE criterions, elastic net criterions, sparse linear discriminant analysis, saliency feature learning, and restricted Boltzmann machine.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)

    Article  Google Scholar 

  2. Aharon, M., Elad, M.: Sparse and redundant modeling of image content using an image-signature-dictionary. SIAM J. Imaging Sci. 1(3), 228–247 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  3. Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011)

    Article  Google Scholar 

  4. Bach, F., Moulines, E., et al.: Non-asymptotic analysis of stochastic approximation algorithms for machine learning. In: NIPS (2011)

    Google Scholar 

  5. Bach, F.: Stochastic gradient methods for machine learning (2014)

    Google Scholar 

  6. Bo, L., Ren, X., Fox, D.: Hierarchical matching pursuit for image classification: architecture and fast algorithms. In: NIPS, vol. 1, pp. 2115–2123 (2011)

    Google Scholar 

  7. Bottou, L., Bousquet, O.: The tradeoffs of large scale learning. In: NIPS, vol. 4 (2007)

    Google Scholar 

  8. Boureau, Y.L., Ponce, J., LeCun, Y.: A theoretical analysis of feature pooling in visual recognition. In: ICML (2010)

    Google Scholar 

  9. Chan, A.B., Vasconcelos, N., Lanckriet, G.R.: Direct convex relaxations of sparse SVM. In: ACM International Conference on Machine Learning (2007)

    Google Scholar 

  10. Cheng, H., Liu, Z., Zheng, N., Yang, L.: A deformable local image descriptor. In: IEEE CVPR (2008)

    Google Scholar 

  11. Chiang, C.K., Duan, C.H., Lai, S.H., Chang, S.F.: Learning component-level sparse representation using histogram information for image classification. In: IEEE ICCV (2011)

    Google Scholar 

  12. Cong, Y., Yuan, J., Liu, J.: Sparse reconstruction cost for abnormal event detection. In: IEEE CVPR (2011)

    Google Scholar 

  13. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE CVPR (2005)

    Google Scholar 

  14. Dong, W., Li, X., Zhang, D., Shi, G.: Sparsity-based image denoising via dictionary learning and structural clustering. In: IEEE CVPR (2011)

    Google Scholar 

  15. Elad, M., Aharon, M.: Image denoising via learned dictionaries and sparse representation. In: IEEE CVPR (2006)

    Google Scholar 

  16. Guyon, I., Weston, J., Barnhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machines. Mach. Learn. 46(1–3), 389–422 (2002)

    Article  MATH  Google Scholar 

  17. Han, J., He, S., Qian, X., Wang, D., Guo, L., Liu, T.: An object-oriented visual saliency detection framework based on sparse coding representations. IEEE Trans. Circuits Syst. Video Technol. 23(12), 2009–2021 (2013)

    Article  Google Scholar 

  18. Hou, X., Zhang, L.: Dynamic visual attention: searching for coding length increments. In: Advances in Neural Information Processing Systems (2009)

    Google Scholar 

  19. Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: IEEE CVPR (2007)

    Google Scholar 

  20. Jiang, Z., Lin, Z., Davis, L.S.: Learning a discriminative dictionary for sparse coding via label consistent K-SVD. In: IEEE CVPR (2011)

    Google Scholar 

  21. Jiang, Z., Zhang, G., Davis, L.S.: Submodular dictionary learning for sparse coding. In: IEEE CVPR (2012)

    Google Scholar 

  22. Jiang, Y.G., Ngo, C.W.: Visual word proximity and linguistics for semantic video indexing and near-duplicate retrieval. Comput. Vis. Image Underst. 113(3), 405–414 (2009)

    Article  Google Scholar 

  23. Jurie, F., Triggs, B.: Creating efficient codebooks for visual recognition. In: IEEE ICCV (2005)

    Google Scholar 

  24. Kavukcuoglu, K., Ranzato, M., LeCun, Y.: Fast inference in sparse coding algorithms with applications to object recognition (2010) arXiv preprint

    Google Scholar 

  25. Kavukcuoglu, K., Sermanet, P., Boureau, Y.L., Gregor, K., Mathieu, M., LeCun, Y.: Learning convolutional feature hierarchies for visual recognition. In: NIPS, vol. 1 (2010)

    Google Scholar 

  26. Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)

    Article  Google Scholar 

  27. Kreutz-Delgado, K., Murray, J.F., Rao, B.D., Engan, K., Lee, T.W., Sejnowski, T.J.: Dictionary learning algorithms for sparse representation. Neural Comput. 15(2), 349–396 (2003)

    Article  MATH  Google Scholar 

  28. Krishnan, D., Tay, T., Fergus, R.: Blind deconvolution using a normalized sparsity measure. In: IEEE CVPR (2011)

    Google Scholar 

  29. Kulkarni, N., Li, B.: Discriminative affine sparse codes for image classification. In: IEEE CVPR (2011)

    Google Scholar 

  30. Lang, C., Liu, G., Yu, J., Yan, S.: Saliency detection by multitask sparsity pursuit. IEEE Trans. Image Process. 21(3), 1327–1338 (2012)

    Article  MathSciNet  Google Scholar 

  31. Lee, H., Battle, A., Raina, R., Ng, A.Y.: Efficient sparse coding algorithms. Adv. Neural Inf. Process. Syst. 19, 801 (2007)

    Google Scholar 

  32. Li, Y., Fermuller, C., Aloimonos, Y., Ji, H.: Learning shift-invariant sparse representation of actions. In: IEEE CVPR (2010)

    Google Scholar 

  33. Li, X., Jia, T., Zhang, H.: Expression-insensitive 3d face recognition using sparse representation. In: IEEE CVPR (2009)

    Google Scholar 

  34. Liang, Y., Wang, L., Xiang, Y., Zou, B.: Feature selection via sparse approximation for face recognition (2011) arXiv preprint

    Google Scholar 

  35. Liu, G., Lin, Z., Yu, Y.: Robust subspace segmentation by low-rank representation. In: ICML (2010)

    Google Scholar 

  36. Liu, H., Yu, L.: Toward integrating feature selection algorithms for classification and clustering. IEEE Trans. Knowl. Data Eng. 17(4), 491–502 (2005)

    Article  Google Scholar 

  37. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  38. Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A., et al.: Supervised dictionary learning (2008)

    Google Scholar 

  39. Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A.: Discriminative learned dictionaries for local image analysis. In: IEEE CVPR (2008)

    Google Scholar 

  40. Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online dictionary learning for sparse coding. In: Annual International Conference on Machine Learning. ACM (2009)

    Google Scholar 

  41. Mairal, J., Sapiro, G., Elad, M.: Learning multiscale sparse representations for image and video restoration. Technical report, DTIC Document (2007)

    Google Scholar 

  42. Mairal, J., Elad, M., Sapiro, G.: Sparse representation for color image restoration. IEEE Trans. Image Process. 17(1), 53–69 (2008)

    Article  MathSciNet  Google Scholar 

  43. Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online learning for matrix factorization and sparse coding. J. Mach. Learn. Res. 11, 19–60 (2010)

    MATH  MathSciNet  Google Scholar 

  44. Maire, M., Arbelaez, P., Fowlkes, C., Malik. J.: Using contours to detect and localize junctions in natural images. In: IEEE CVPR (2008)

    Google Scholar 

  45. Martin, D.R., Fowlkes, C.C., Malik, J.: Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans. Pattern Anal. Mach. Intell. 26(5), 530–549 (2004)

    Article  Google Scholar 

  46. Masaeli, M., Dy, J.G., Fung, G.M.: From transformation-based dimensionality reduction to feature selection. In: ICML (2010)

    Google Scholar 

  47. Moghaddam, B., Weiss, Y., Avidan, S.: Fast pixel/part selection with sparse eigenvectors. In: IEEE ICCV (2007)

    Google Scholar 

  48. Moghaddam, B., Weiss, Y., Avidan, S.: Generalized spectral bounds for sparse LDA. In: International conference on Machine Learning. ACM (2006)

    Google Scholar 

  49. Morel, J.M., Yu, G.: ASIFT: a new framework for fully affine invariant image comparison. SIAM J. Imaging Sci. 2(2), 438–469 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  50. Nagesh, P., Li, B.: A compressive sensing approach for expression-invariant face recognition. In: IEEE CVPR (2009)

    Google Scholar 

  51. Naikal, N., Yang, A.Y., Sastry, S.S.: Informative feature selection for object recognition via sparse PCA. In: IEEE ICCV (2011)

    Google Scholar 

  52. Oliva, A., Torralba, A.: Building the gist of a scene: the role of global image features in recognition. Prog. Brain Res. 155, 23–36 (2006)

    Google Scholar 

  53. Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: a strategy employed by v1? Vis. Res. 37(23), 3311–3325 (1997)

    Article  Google Scholar 

  54. Pham, D.S., Venkatesh, S.: Joint learning and dictionary construction for pattern recognition. In: IEEE CVPR (2008)

    Google Scholar 

  55. Ramirez, I., Sprechmann, P., Sapiro, G.: Classification and clustering via dictionary learning with structured incoherence and shared features. In: IEEE CVPR (2010)

    Google Scholar 

  56. Ren, X., Bo, L.: Discriminatively trained sparse code gradients for contour detection. In: NIPS, vol. 2, pp. 584–592 (2012)

    Google Scholar 

  57. Ren, X., Ramanan, D.: Histograms of sparse codes for object detection. In: IEEE CVPR (2013)

    Google Scholar 

  58. Riesenhuber, M., Poggio, T.: Hierarchical models of object recognition in cortex. Nat. Neurosci. 2(11), 1019–1025 (1999)

    Article  Google Scholar 

  59. Shi, J., Ren, X., Dai, G., Wang, J., Zhang, Z.: A non-convex relaxation approach to sparse dictionary learning. In: IEEE CVPR (2011)

    Google Scholar 

  60. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)

    Article  Google Scholar 

  61. Sivic, J., Zisserman, A.: Video Google: a text retrieval approach to object matching in videos. In: IEEE ICCV (2003)

    Google Scholar 

  62. Sohn, K., Jung, D.Y., Lee, H., Hero, A.O.: Efficient learning of sparse, distributed, convolutional feature representations for object recognition. In: IEEE ICCV (2011)

    Google Scholar 

  63. Sohn, K., Zhou, G., Lee, H.: Jointly learning and selecting features via conditional point-wise mixture RBMs (2012)

    Google Scholar 

  64. Srebro, N., Tewari, A.: Stochastic optimization for machine learning. In: ICML (2010)

    Google Scholar 

  65. Tan, M., Wang, L., Tsang, I.W.: Learning sparse SVM for feature selection on very high dimensional datasets. In: International Conference on Machine Learning (2010)

    Google Scholar 

  66. Tapia, E., Bulacio, P., Angelone, L.: Sparse and stable gene selection with consensus SVM-RFE. Pattern Recognit. Lett. 33(2), 164–172 (2012)

    Article  Google Scholar 

  67. Wang, L., Cheng, H.: Robust sparse PCA via weighted elastic net. In: Pattern Recognition (2012)

    Google Scholar 

  68. Wang, L., Cheng, H., Liu, Z., Zhu, C.: A robust elastic net approach for feature learning. J. Vis. Commun. Image Represent. 25(2), 313–321 (2014)

    Article  Google Scholar 

  69. Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 210–227 (2009)

    Article  Google Scholar 

  70. Yang, J., Yu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: IEEE CVPR (2009)

    Google Scholar 

  71. Yang, J., Yu, K., Huang, T.: Supervised translation-invariant sparse coding. In: IEEE CVPR (2010)

    Google Scholar 

  72. Yang, M., Zhang, D., Feng, X.: Fisher discrimination dictionary learning for sparse representation. In: IEEE ICCV (2011)

    Google Scholar 

  73. Yu, K., Lin, Y., Lafferty, J.: Learning image representations from the pixel level via hierarchical sparse coding. In: IEEE CVPR (2011)

    Google Scholar 

  74. Zhang, T., Ghanem, B., Liu, S., Ahuja, N.: Robust visual tracking via multi-task sparse learning. In: IEEE CVPR (2012)

    Google Scholar 

  75. Zhang, Q., Li, B.: Discriminative K-SVD for dictionary learning in face recognition. In: IEEE CVPR (2010)

    Google Scholar 

  76. Zhu, J., Rosset, S., Hastie, T., Tibshirani, R.: 1-norm support vector machines. In: NIPS (2003)

    Google Scholar 

  77. Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soc. 67(2), 301–320 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  78. Zou, H., Hastie, T., Tibshirani, R.: Sparse principal component analysis. J. Comput. Graph. Stat. 15(2), 265–286 (2006)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Electronic Science and Technology of China, Chengdu, Sichuan, China

    Hong Cheng

Authors
  1. Hong Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hong Cheng .

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer-Verlag London

About this chapter

Cite this chapter

Cheng, H. (2015). Feature Representation and Learning. In: Sparse Representation, Modeling and Learning in Visual Recognition. Advances in Computer Vision and Pattern Recognition. Springer, London. https://doi.org/10.1007/978-1-4471-6714-3_6

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-6714-3_6

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-6713-6

  • Online ISBN: 978-1-4471-6714-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation