Skip to main content

Projections with Logarithmic Divergences

Part of the Lecture Notes in Computer Science book series (LNIP,volume 12829)

Abstract

In information geometry, generalized exponential families and statistical manifolds with curvature are under active investigation in recent years. In this paper we consider the statistical manifold induced by a logarithmic \(L^{(\alpha )}\)-divergence which generalizes the Bregman divergence. It is known that such a manifold is dually projectively flat with constant negative sectional curvature, and is closely related to the \(\mathcal {F}^{(\alpha )}\)-family, a generalized exponential family introduced by the second author [16]. Our main result constructs a dual foliation of the statistical manifold, i.e., an orthogonal decomposition consisting of primal and dual autoparallel submanifolds. This decomposition, which can be naturally interpreted in terms of primal and dual projections with respect to the logarithmic divergence, extends the dual foliation of a dually flat manifold studied by Amari [1]. As an application, we formulate a new \(L^{(\alpha )}\)-PCA problem which generalizes the exponential family PCA [5].

Keywords

  • Logarithmic divergence
  • Generalized exponential family
  • Dual foliation
  • Projection
  • Principal component analysis

This research is supported by NSERC Discovery Grant RGPIN-2019-04419 and a Connaught New Researcher Award.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-80209-7_52
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   119.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-80209-7
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   159.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.

References

  1. Amari, S.I.: Information geometry on hierarchy of probability distributions. IEEE Trans. Inf. Theor. 47(5), 1701–1711 (2001)

    MathSciNet  CrossRef  Google Scholar 

  2. Amari, S.: Information Geometry and Its Applications. AMS, vol. 194. Springer, Tokyo (2016). https://doi.org/10.1007/978-4-431-55978-8

    CrossRef  MATH  Google Scholar 

  3. Ay, N., Amari, S.I.: A novel approach to canonical divergences within information geometry. Entropy 17(12), 8111–8129 (2015)

    Google Scholar 

  4. Ay, N., Jost, J., Lê, H.V., Schwachhöfer, L.: Information Geometry. EMGFASMSM, vol. 64. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-56478-4

    CrossRef  MATH  Google Scholar 

  5. Collins, M., Dasgupta, S., Schapire, R.E.: A generalization of principal components analysis to the exponential family. In: Advances in Neural Information Processing Systems, pp. 617–624 (2002)

    Google Scholar 

  6. Egozcue, J.J., Pawlowsky-Glahn, V., Mateu-Figueras, G., Barcelo-Vidal, C.: Isometric logratio transformations for compositional data analysis. Math. Geol. 35(3), 279–300 (2003)

    MathSciNet  CrossRef  Google Scholar 

  7. Eguchi, S.: Second order efficiency of minimum contrast estimators in a curved exponential family. Ann. Statist. 11 793–803 (1983)

    Google Scholar 

  8. Jolliffe, I.T., Cadima, J.: Principal component analysis: a review and recent developments. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 374(2065), 20150202 (2016)

    MathSciNet  CrossRef  Google Scholar 

  9. Murata, N., Takenouchi, T., Kanamori, T., Eguchi, S.: Information geometry of u-boost and bregman divergence. Neural Comput. 16(7), 1437–1481 (2004)

    CrossRef  Google Scholar 

  10. Nagaoka, H., Amari, S.I.: Differential geometry of smooth families of probability distributions. Tech. Rep. pp. 82–87. METR, University of Tokyo (1982)

    Google Scholar 

  11. Naudts, J.: Generalised Thermostatistics. Springer, London (2011). https://doi.org/10.1007/978-0-85729-355-8

  12. Pal, S., Wong, T.-K.L.: The geometry of relative arbitrage. Math. Fin. Econ. 10(3), 263–293 (2015). https://doi.org/10.1007/s11579-015-0159-z

    MathSciNet  CrossRef  MATH  Google Scholar 

  13. Pal, S., Wong, T.K.L.: Exponentially concave functions and a new information geometry. Ann. Probabl. 46(2), 1070–1113 (2018)

    MathSciNet  MATH  Google Scholar 

  14. Pal, S., Wong, T.K.L.: Multiplicative Schröodinger problem and the Dirichlet transport. Probabl. Theor. Relat.Fields 178(1), 613–654 (2020)

    CrossRef  Google Scholar 

  15. Pawlowsky-Glahn, V., Buccianti, A.: Compositional Data Analysis: Theory and Applications. John Wiley & Sons, Chichester (2011)

    Google Scholar 

  16. Wong, T.-K.L.: Logarithmic divergences from optimal transport and Rényi geometry. Inf. Geom. 1(1), 39–78 (2018). https://doi.org/10.1007/s41884-018-0012-6

    MathSciNet  CrossRef  MATH  Google Scholar 

  17. Wong, T.-K.L.: Information geometry in portfolio theory. In: Nielsen, F. (ed.) Geometric Structures of Information. SCT, pp. 105–136. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-02520-5_6

    CrossRef  Google Scholar 

  18. Wong, T.-K.L., Yang, J.: Logarithmic divergences: geometry and interpretation of curvature. In: Nielsen, F., Barbaresco, F. (eds.) GSI 2019. LNCS, vol. 11712, pp. 413–422. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-26980-7_43

    CrossRef  Google Scholar 

  19. Wong, T.K.L., Yang, J.: Pseudo-Riemannian geometry embeds information geometry in optimal transport. arXiv preprint. arXiv:1906.00030 (2019)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ting-Kam Leonard Wong .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Tao, Z., Wong, TK.L. (2021). Projections with Logarithmic Divergences. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2021. Lecture Notes in Computer Science(), vol 12829. Springer, Cham. https://doi.org/10.1007/978-3-030-80209-7_52

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-80209-7_52

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-80208-0

  • Online ISBN: 978-3-030-80209-7

  • eBook Packages: Computer ScienceComputer Science (R0)