Skip to main content

From Bayesian Inference to MCMC and Convex Optimisation in Hadamard Manifolds

  • Conference paper
  • First Online:
Geometric Science of Information (GSI 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12829))

Included in the following conference series:

  • 2470 Accesses

Abstract

The present work is motivated by the problem of Bayesian inference for Gaussian distributions in symmetric Hadamard spaces (that is, Hadamard manifolds which are also symmetric spaces). To investigate this problem, it introduces new tools for Markov Chain Monte Carlo, and convex optimisation: (1) it provides easy-to-verify sufficient conditions for the geometric ergodicity of an isotropic Metropolis-Hastings Markov chain, in a symmetric Hadamard space, (2) it shows how the Riemannian gradient descent method can achieve an exponential rate of convergence, when applied to a strongly convex function, on a Hadamard manifold. Using these tools, two Bayesian estimators, maximum-a-posteriori and minimum-mean-squares, are compared. When the underlying Hadamard manifold is a space of constant negative curvature, they are found to be surprisingly close to each other. This leads to an open problem: are these two estimators, in fact, equal (assuming constant negative curvature)?

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Bhattacharya, R., Patrangenaru, V.: Large sample theory of instrinsic and extrinsic sample means on manifolds I. Ann. Stat. 31(1), 1–29 (2003)

    Article  Google Scholar 

  2. Jeuris, B., Vandebril, R.: The Kähler mean of block-Toeplitz matrices with Toeplitz structured blocks. SIAM J. Matrix Anal. Appl. (2016). https://lirias.kuleuven.be/handle/123456789/497758

  3. Meyn, S., Tweedie, R.L.: Markov Chains and Stochastic Stability. Cambridge University Press, Cambridge (2008)

    MATH  Google Scholar 

  4. Nielsen, F., Nock, R.: Hyperbolic Voronoi diagrams made easy (2009). https://arxiv.org/abs/0903.3287

  5. Petersen, P.: Riemannian Geometry, 2nd edn. Springer, New York (2006). https://doi.org/10.1007/978-0-387-29403-2

    Book  MATH  Google Scholar 

  6. Roberts, R.O., Rosenthal, J.S.: General state-space Markov chains and MCMC algorithms. Probab. Surv. 1, 20–71 (2004)

    Article  MathSciNet  Google Scholar 

  7. Said, S.: Statistical models and probabilistic methods on Riemannian manifolds. Technical report, Université de Bordeaux (2021). http://arxiv.org/abs/2101.10855

  8. Said, S., Bombrun, L., Berthoumieu, Y., Manton, J.H.: Riemannian Gaussian distributions on the space of symmetric positive definite matrices. IEEE Trans. Inf. Theory (2016). http://arxiv.org/abs/1507.01760

  9. Said, S., Hajri, H., Bombrun, L., Vemuri, B.C.: Gaussian distributions on Riemannian symmetric spaces: statistical learning with structured covariance matrices. IEEE Trans. Inf. Theory 64(2), 752–772 (2018)

    Article  MathSciNet  Google Scholar 

  10. Sturm, K.T.: Probability measures on metric spaces of nonpositive curvature. Contemp. Math. 338, 1–34 (2003)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Salem Said .

Editor information

Editors and Affiliations

A Symmetric Hadamard Spaces

A Symmetric Hadamard Spaces

The present work focuses on symmetric Hadamard spaces. These are Hadamard manifolds which are also symmetric spaces. This class of spaces includes: spaces of constant negative curvature (precisely, hyperbolic spaces), cones of covariance matrices (real, complex, or quaternion), as well as Siegel domaines (recently used in describing spaces of block-Toeplitz covariance matrices). These spaces play a prominent role in several applications [2, 4, 8].

1.1 A.1 Hadamard Manifolds

Definition

A Hadamard manifold is a Riemannian manifold which is complete, simply connected, and has negative sectional curvatures [5]. Completeness means that geodesics extend indefinitely in both directions, and is equivalent to the property that closed and bounded sets are compact (by the Hopf-Rinow theorem [5]). The simple connectedness and negative curvature properties combine to ensure that the Riemannian exponential map is a diffeomorphism. In particular, any two points xy in a Hadamard manifold M are connected by a unique length-minimising geodesic \(c:[0,1]\rightarrow M\), with \(c(0) = x\), and \(dc/dt|_{t=0} = \mathrm {Exp}^{-1}_x(y)\).

Convexity

Now, this geodesic \(c:[0,1]\rightarrow M\) is called the segment between x and y. For \(t \in [0,1]\), the point \(x\, \#_{t}\, y = c(t)\) is said to be a geodesic convex combination of x and y, with respective weights \((1-t)\) and t. In a Hadamard manifold [5], any open geodesic ball \(B(x^*,R)\) (or any closed geodesic ball \(\bar{B}(x^*,R)\)) is convex: if x and y belong to this ball, so does \(x\, \#_{t}\, y\) for all \(t \in (0,1)\).

A function \(f:M \rightarrow \mathbb {R}\) is called \((\alpha /2)\)-strongly convex, if \((f\circ c)(t)\) is an \((\alpha /2)\)-strongly convex function of t, for any geodesic \(c:[0,1]\rightarrow M\). That is, if there exists \(\alpha > 0\), such that for any x and y in M, and \(t \in [0,1]\),

$$\begin{aligned} f(x\, \#_{t}\, y) \le (1-t)f(x) + tf(y) - (\alpha /2)t(1-t)d^{ 2}(x,y) \end{aligned}$$
(19)

When f is \(C^2\)-smooth, this is equivalent to (14).

Squared Distance Function

For \(y \in M\), consider the function \(f_y:M\rightarrow \mathbb {R}\), where \(f_y(x) = d^{ 2}(x,y)/2\). If M is a Hadamard manifold, then this function has several remarkable properties.

Specifically, it is both smooth and (1/2)-strongly convex (irrespective of y). It’s gradient is given by \(\mathrm {grad}\,f_y(x) = -\mathrm {Exp}^{-1}_x(y)\), in terms of the Riemannian exponential map. On the other hand, on any geodesic ball B(yR), the operator norm of its Hessian is bounded by \(cR\coth (cR)\), where \(c > 0\) is such that the sectional curvatures of M lie in the interval \([-c^{2},0]\) (see [5], Theorem 27).

A result of the strong convexity of these functions \(f_y\) is the uniqueness of Riemannian barycentres. If \(\pi \) is a probability distribution on M, its Riemannian barycentre \(\hat{x}_{\pi }\) is the unique global minimiser of the so-called variance function \(\mathcal {E}_{\pi }(x) \,=\, \int _M\,f_y(x)\pi (dy) \). If M is a Hadamard manifold, each function \(f_y\) is (1/2)-strongly convex, and therefore \(\mathcal {E}_{\pi }\) is (1/2)-strongly convex. Thus, \(\mathcal {E}_{\pi }\) has a unique global minimiser.

1.2 A.2 Symmetric Spaces

Definition

A Riemannian symmetric space is a Riemannian manifold M, such that for each \(x \in M\) there exists an isometry \(s_{x}:M \rightarrow M\), which fixes x and reverses geodesics passing through x: if c is a geodesic curve with \(c(0) = x\), then \(s_x(c(t)) = c(-t)\). As a consequence of this definition, a Riemannian symmetric space is always complete and homogeneous. Completeness was discussed in Appendix A.1. On the other hand, homogeneity means that for each x and y in M, there exists an isometry \(g:M\rightarrow M\), with \(g(x) = y\). A symmetric space M may always be expressed as a quotient manifold \(M = G/K\), where G is a certain Lie group of isometries of M, and K is the subgroup of elements of G which fix some point \(x \in M\).

A Familiar Example

A familiar example of a symmetric Hadamard manifold is the cone \(\mathcal {P}_d\) of \(d \times d\) symmetric positive-definite matrices, which was discussed in some length in [8, 9]. A geodesic curve through \(x \in \mathcal {P}_d\) is of the form \(c(t) = x\exp (tx^{-1}u)\) where \(\exp \) is the matrix exponential, and u a symmetric matrix, \(dc/dt|_{t=0} = u\). The isometry \(s_x\) is given by \(s_x(y) = xy^{-1}x\) for \(y \in \mathcal {P}_d\), and one easily checks \(s_x(c(t)) = c(-t)\).

Let \( G = \mathrm {GL}(d,\mathbb {R})\) denote the group of invertible \(d \times d\) real matrices. Then, \(g \in G\) defines an isometry \(\tau _g:\mathcal {P}_d \rightarrow \mathcal {P}_d\), given by \(\tau _g(x) = gxg^\dagger \) (\(\dagger \) the transpose). Typically, one writes \(\tau _g(x) = g(x)\). Note that \(x = \mathrm {I}_d\) (the identity matrix) belongs to \(\mathcal {P}_d\), and then \(g(x) = x\) if and only if \(g \in K\), where \(K = O(d)\) is the orthogonal group. In fact, one has the quotient manifold structure \(\mathcal {P}_d = \mathrm {GL}(d,\mathbb {R})/O(d)\).

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Said, S., Le Bihan, N., Manton, J.H. (2021). From Bayesian Inference to MCMC and Convex Optimisation in Hadamard Manifolds. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2021. Lecture Notes in Computer Science(), vol 12829. Springer, Cham. https://doi.org/10.1007/978-3-030-80209-7_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-80209-7_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-80208-0

  • Online ISBN: 978-3-030-80209-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics