Skip to main content
SpringerLink
Log in
Book cover

Algorithmic Advances in Riemannian Geometry and Applications pp 145–172Cite as

Dictionary Learning on Grassmann Manifolds

  • Chapter
  • First Online:

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

Sparse representations have recently led to notable results in various visual recognition tasks. In a separate line of research, Riemannian manifolds have been shown useful for dealing with features and models that do not lie in Euclidean spaces. With the aim of building a bridge between the two realms, we address the problem of sparse coding and dictionary learning in Grassmann manifolds, i.e, the space of linear subspaces. To this end, we introduce algorithms for sparse coding and dictionary learning by embedding Grassmann manifolds into the space of symmetric matrices. Furthermore, to handle nonlinearity in data, we propose positive definite kernels on Grassmann manifolds and make use of them to perform coding and dictionary learning.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    The function that maps each vector \({{y}} \in T_\mathscr {P}\mathscr {M}\) to a point \(\mathscr {X}\) of the manifold that is reached after a unit time by the geodesic starting at \(\mathscr {P}\) with this tangent vector is called the exponential map. For complete manifolds, this map is defined in the whole tangent space \(T_\mathscr {P}\mathscr {M}\). The logarithm map is the inverse of the exponential map, i.e, \({{y}} = \log _{\mathscr {P}}(\mathscr {X})\) is the smallest vector \({{y}}\) such that \(\mathscr {X} = \exp _\mathscr {P}({{y}})\).

  2. 2.

    On an abstract Riemannian manifold \({\mathscr {M}}\), the gradient of a smooth real function f at a point \(x \in {\mathscr {M}}\), denoted by \(\mathrm {grad} f(x)\), is the element of \(T_x({\mathscr {M}})\) satisfying \(\langle \mathrm {grad}f(x), \zeta \rangle _x = Df_x[\zeta ]\) for all \(\zeta \in T_x({\mathscr {M}})\). Here, \(Df_x[\zeta ]\) denotes the directional derivative of f at x in the direction of \(\zeta \). The interested reader is referred to [1] for more details on how the gradient of a function on Grassmann manifolds can be computed.

  3. 3.

    Another situation where this applies in Computer Vision is the study of the essential manifold, which may be envisaged as the coset space of \(SO(3) \times SO(3)\) modulo a subgroup isomorphic to SO(2). For details see [25].

  4. 4.

    O(d) has dimension \(d(d-1)/2\), since its Lie algebra is the set of \(n\times n\) skew-symmetric matrices.

  5. 5.

    In our experiments, we observed that the projection kernel almost always outperforms the Binet–Cauchy kernel.

References

  1. P.A. Absil, R. Mahony, R. Sepulchre, Optimization Algorithms on Matrix Manifolds (Princeton University Press, Princeton, 2008)

    Book  MATH  Google Scholar 

  2. M. Aharon, M. Elad, A. Bruckstein, K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)

    Article  Google Scholar 

  3. V. Arsigny, P. Fillard, X. Pennec, N. Ayache, Log-Euclidean metrics for fast and simple calculus on diffusion tensors. Magn. Reson. Med. 56(2), 411–421 (2006)

    Article  Google Scholar 

  4. R. Basri, D.W. Jacobs, Lambertian reflectance and linear subspaces. IEEE Trans. Pattern Anal. Mach. Intell. 25(2), 218–233 (2003)

    Article  Google Scholar 

  5. E. Begelfor, M. Werman, Affine invariance revisited, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2006), pp. 2087–2094

    Google Scholar 

  6. C. Berg, J.P.R. Christensen, P. Ressel, Harmonic Analysis on Semigroups (Springer, New York, 1984)

    Book  MATH  Google Scholar 

  7. E.J. Candès, J. Romberg, T. Tao, Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  8. H.E. Cetingul, M.J. Wright, P.M. Thompson, R. Vidal, Segmentation of high angular resolution diffusion MRI using sparse Riemannian manifold clustering. IEEE Trans. Med. Imaging 33(2), 301–317 (2014)

    Article  Google Scholar 

  9. S. Chen, C. Sanderson, M. Harandi, B.C. Lovell, Improved image set classification via joint sparse approximated nearest subspaces, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013), pp. 452–459

    Google Scholar 

  10. N. Dalal, B. Triggs, Histograms of oriented gradients for human detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2005), pp. 886–893

    Google Scholar 

  11. D.L. Donoho, Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  12. M. Elad, Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing (Springer, New York, 2010)

    Book  MATH  Google Scholar 

  13. E. Elhamifar, R. Vidal, Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2765–2781 (2013)

    Article  Google Scholar 

  14. M. Faraki, M. Harandi, F. Porikli, More about VLAD: a leap from Euclidean to Riemannian manifolds, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), pp. 4951–4960

    Google Scholar 

  15. K.A. Gallivan, A. Srivastava, X. Liu, P. Van Dooren, Efficient algorithms for inferences on Grassmann manifolds, in IEEE Workshop on Statistical Signal Processing (2003), pp. 315–318

    Google Scholar 

  16. B. Gong, Y. Shi, F. Sha, K. Grauman, Geodesic flow kernel for unsupervised domain adaptation, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012), pp. 2066–2073

    Google Scholar 

  17. R. Gopalan, R. Li, R. Chellappa, Unsupervised adaptation across domain shifts by generating intermediate data representations. IEEE Trans. Pattern Anal. Mach. Intell. 36(11), 2288–2302 (2014). doi:10.1109/TPAMI.2013.249

    Google Scholar 

  18. J. Hamm, D.D. Lee, Grassmann discriminant analysis: a unifying view on subspace-based learning, in Proceedings of the International Conference on Machine Learning (ICML) (2008), pp. 376–383

    Google Scholar 

  19. M. Harandi, M. Salzmann, Riemannian coding and dictionary learning: kernels to the rescue, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), pp. 3926–3935

    Google Scholar 

  20. M. Harandi, R. Hartley, C. Shen, B. Lovell, C. Sanderson, Extrinsic methods for coding and dictionary learning on Grassmann manifolds. Int. J. Comput. Vis. 114(2–3), 113–136 (2015)

    Article  MathSciNet  Google Scholar 

  21. M.T. Harandi, M. Salzmann, S. Jayasumana, R. Hartley, H. Li, Expanding the family of Grassmannian kernels: an embedding perspective, in Proceedings of the European Conference on Computer Vision (ECCV), vol. 8695, Lecture Notes in Computer Science, ed. by D. Fleet, T. Pajdla, B. Schiele, T. Tuytelaars (Springer International Publishing, Cham, 2014), pp. 408–423. doi:10.1007/978-3-319-10584-0_27

    Google Scholar 

  22. M.T. Harandi, R. Hartley, B.C. Lovell, C. Sanderson, Sparse coding on symmetric positive definite manifolds using Bregman divergences. IEEE Trans. Neural Netw. Learn. Syst. (TNNLS) PP(99), 1–1 (2015)

    Google Scholar 

  23. R. Hartley, J. Trumpf, Y. Dai, H. Li, Rotation averaging. Int. J. Comput. Vis. 103(3), 267–305 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  24. U. Helmke, K. Hper, J. Trumpf, Newton’s method on Gramann manifolds (2007)

    Google Scholar 

  25. U. Helmke, K. Hüper, P.Y. Lee, J.B. Moore, Essential matrix estimation using Gauss-Newton iterations on a manifold. Int. J. Comput. Vis. 74(2), 117–136 (2007). doi:10.1007/s11263-006-0005-0

    Google Scholar 

  26. J. Ho, Y. Xie, B. Vemuri, On a nonlinear generalization of sparse coding and dictionary learning, in Proceedings of the International Conference on Machine Learning (ICML) (2013), pp. 1480–1488

    Google Scholar 

  27. S. Jayasumana, R. Hartley, M. Salzmann, H. Li, M. Harandi, Kernel methods on Riemannian manifolds with Gaussian RBF kernels. IEEE Trans. Pattern Anal. Mach. Intell. 37(12), 2464–2477 (2015). doi:10.1109/TPAMI.2015.2414422

    Google Scholar 

  28. H. Karcher, Riemannian center of mass and mollifier smoothing. Commun. Pure Appl. Math. 30(5), 509–541 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  29. J.M. Lee, Introduction to Smooth Manifolds, vol. 218 (Springer, New York, 2012)

    Book  Google Scholar 

  30. J. Mairal, F. Bach, J. Ponce, G. Sapiro, A. Zisserman, Discriminative learned dictionaries for local image analysis, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2008), pp. 1–8

    Google Scholar 

  31. J. Mairal, M. Elad, G. Sapiro, Sparse representation for color image restoration. IEEE Trans. Image Process. (TIP) 17(1), 53–69 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  32. J.H. Manton, A globally convergent numerical algorithm for computing the centre of mass on compact lie groups. Int. Conf. Control Autom. Robot. Vis. 3, 2211–2216 (2004)

    Google Scholar 

  33. B.A. Olshausen, D.J. Field, Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381(6583), 607–609 (1996)

    Article  Google Scholar 

  34. R. Ramamoorthi, Analytic PCA construction for theoretical analysis of lighting variability in images of a Lambertian object. IEEE Trans. Pattern Anal. Mach. Intell. 24(10), 1322–1333 (2002)

    Article  Google Scholar 

  35. B. Schölkopf, R. Herbrich, A.J. Smola, A generalized representer theorem, Computational Learning Theory (Springer, New York, 2001), pp. 416–426

    Chapter  Google Scholar 

  36. J. Shawe-Taylor, N. Cristianini, Kernel Methods for Pattern Analysis (Cambridge University Press, Cambridge, 2004)

    Book  MATH  Google Scholar 

  37. S. Shirazi, M. Harandi, B. Lovell, C. Sanderson, Object tracking via non-Euclidean geometry: a Grassmann approach, in IEEE Winter Conference on Applications of Computer Vision (WACV) (2014), pp. 901–908. doi:10.1109/WACV.2014.6836008

  38. I. Steinwart, A. Christmann, Support Vector Machines (Springer, Berlin, 2008)

    Google Scholar 

  39. R. Subbarao, P. Meer, Nonlinear mean shift over Riemannian manifolds. Int. J. Comput. Vis. 84(1), 1–20 (2009)

    Article  Google Scholar 

  40. R. Tibshirani, Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Methodol.) 58, 267–288 (1996)

    Google Scholar 

  41. P. Turaga, A. Veeraraghavan, A. Srivastava, R. Chellappa, Statistical computations on Grassmann and Stiefel manifolds for image and video-based recognition. IEEE Trans. Pattern Anal. Mach. Intell. 33(11), 2273–2286 (2011)

    Google Scholar 

  42. R. Vemulapalli, J.K. Pillai, R. Chellappa, Kernel learning for extrinsic classification of manifold features, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013), pp. 1782–1789

    Google Scholar 

  43. J. Wang, J. Yang, K. Yu, F. Lv, T. Huang, Y. Gong, Locality-constrained linear coding for image classification, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2010), pp. 3360–3367

    Google Scholar 

  44. Y. Wang, G. Mori, Human action recognition by semilatent topic models. IEEE Trans. Pattern Anal. Mach. Intell. 31(10), 1762–1774 (2009)

    Google Scholar 

  45. L. Wolf, A. Shashua, Learning over sets using kernel principal angles. J. Mach. Learn. Res. 4, 913–931 (2003)

    MathSciNet  MATH  Google Scholar 

  46. J. Wright, A.Y. Yang, A. Ganesh, S.S. Sastry, Y. Ma, Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 210–227 (2009)

    Google Scholar 

  47. J. Wright, Y. Ma, J. Mairal, G. Sapiro, T.S. Huang, S. Yan, Sparse representation for computer vision and pattern recognition. Proc. IEEE 98(6), 1031–1044 (2010)

    Article  Google Scholar 

  48. J. Yang, K. Yu, Y. Gong, T. Huang, Linear spatial pyramid matching using sparse coding for image classification, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009), pp. 1794–1801

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Engineering and Computer Science, Australian National University, Canberra, ACT, 2601, Australia

    Mehrtash Harandi, Richard Hartley & Jochen Trumpf

  2. CVLab, EPFL, Lausanne, Switzerland

    Mathieu Salzmann

Authors
  1. Mehrtash Harandi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Richard Hartley
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mathieu Salzmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jochen Trumpf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mehrtash Harandi .

Editor information

Editors and Affiliations

  1. Istituto Italiano di Tecnologia, Pattern Analysis and Computer Vision, Genoa, Italy

    Hà Quang Minh

  2. Istituto Italiano di Tecnologia, Pattern Analysis and Computer Vision , Genoa, Italy

    Vittorio Murino

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Harandi, M., Hartley, R., Salzmann, M., Trumpf, J. (2016). Dictionary Learning on Grassmann Manifolds. In: Minh, H., Murino, V. (eds) Algorithmic Advances in Riemannian Geometry and Applications. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-319-45026-1_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-45026-1_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-45025-4

  • Online ISBN: 978-3-319-45026-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation