Abstract
The traditional Minkowski distances are induced by the corresponding Minkowski norms in real-valued vector spaces. In this work, we propose statistical symmetric distances based on the Minkowski’s inequality for probability densities belonging to Lebesgue spaces. These statistical Minkowski distances admit closed-form formula for Gaussian mixture models when parameterized by integer exponents.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Like any distance based on the log ratio of triangle inequality gap induced by a homogeneous norm.
- 2.
Because \(\sum _i \alpha _i B_F(\theta _i:\bar{\theta })=J_F({\theta _1},\ldots , {\theta _k};\alpha _1,\ldots ,\alpha _k)\) for the barycenter \(\bar{\theta }=\sum _i\alpha _i \theta _i\), where \(B_F(\theta :\theta ')=F(\theta )-F(\theta ')-(\theta -\theta ')^\top \nabla F(\theta ')\) is a Bregman divergence.
- 3.
To apply the multinomial expansion, we need elements to commute wrt. the product. Thus it does not apply to the matrix cases.
References
Alabiso, C., Weiss, I.: A Primer on Hilbert Space Theory: Linear Spaces, Topological Spaces, Metric Spaces, Normed Spaces, and Topological Groups. UNITEXT for Physics. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-03713-4
Balakrishnan, V.K.: Introductory Discrete Mathematics. Courier Corporation, North Chelmsford (2012)
Banerjee, A., Merugu, S., Dhillon, I.S., Ghosh, J.: Clustering with Bregman divergences. J. Mach. Learn. Res. 6, 1705–1749 (2005)
Bogachev, V.I.: Measure Theory, vol. 1. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-34514-5
Bolton, D.W.: The multinomial theorem. Math. Gaz. 52(382), 336–342 (1968)
Bronstein, M.: Symbolic Integration I: Transcendental Functions, vol. 1. Springer, New York (2013)
Brown, L.D.: Fundamentals of Statistical Exponential Families: With Applications in Statistical Decision Theory. IMS (1986)
Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, Hoboken (2012)
Deza, M.M., Deza, E.: Encyclopedia of distances. In: Deza, E., Deza, M.M. (eds.) Encyclopedia of Distances, pp. 1–583. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-00234-2_1
Durrieu, J.L., Thiran, J.P., Kelly, F.: Lower and upper bounds for approximation of the Kullback-Leibler divergence between Gaussian mixture models. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4833–4836. IEEE (2012)
Fujisawa, H., Eguchi, S.: Robust parameter estimation with a small bias against heavy contamination. J. Multivar. Anal. 99(9), 2053–2081 (2008)
Goldberger, J., Aronowitz, H.: A distance measure between GMMs based on the unscented transform and its application to speaker recognition. In: European Conference on Speech Communication and Technology (INTERSPEECH), pp. 1985–1988 (2005)
Goldberger, J., Gordon, S., Greenspan, H.: An efficient image similarity measure based on approximations of KL-divergence between two Gaussian mixtures, p. 487 (2003)
Jenssen, R., Principe, J.C., Erdogmus, D., Eltoft, T.: The Cauchy-Schwarz divergence and Parzen windowing: connections to graph theory and Mercer kernels. J. Frankl. Inst. 343(6), 614–629 (2006)
Jian, B., Vemuri, B.C.: Robust point set registration using Gaussian mixture models. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 33(8), 1633–1645 (2011)
Minkowski, H.: Geometrie der Zahlen, vol. 40 (1910)
Mousley, S., Schley, N., Shoemaker, A.: Planar rook algebra with colors and Pascal’s simplex. arXiv preprint arXiv:1211.0663 (2012)
Nielsen, F.: A family of statistical symmetric divergences based on Jensen’s inequality. arXiv preprint arXiv:1009.4004 (2010)
Nielsen, F.: Closed-form information-theoretic divergences for statistical mixtures. In: 21st International Conference on Pattern Recognition (ICPR), pp. 1723–1726. IEEE (2012)
Nielsen, F., Boltz, S.: The Burbea-Rao and Bhattacharyya centroids. IEEE Trans. Inf. Theory 57(8), 5455–5466 (2011)
Nielsen, F., Garcia, V.: Statistical exponential families: a digest with flash cards. preprint arXiv:0911.4863 (2009)
Nielsen, F., Nock, R.: Patch matching with polynomial exponential families and projective divergences. In: Amsaleg, L., Houle, M.E., Schubert, E. (eds.) SISAP 2016. LNCS, vol. 9939, pp. 109–116. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46759-7_8
Nielsen, F., Sun, K.: Guaranteed bounds on information-theoretic measures of univariate mixtures using piecewise log-sum-exp inequalities. Entropy 18(12), 442 (2016)
Nielsen, F., Sun, K.: On the chain rule optimal transport distance. arXiv preprint arXiv:1812.08113 (2018)
Nielsen, F., Sun, K., Marchand-Maillet, S.: On Hölder projective divergences. Entropy 19(3), 122 (2017)
Risch, R.H.: The solution of the problem of integration in finite terms. Bull. Am. Math. Soc. 76(3), 605–608 (1970)
Tolsted, E.: An elementary derivation of the Cauchy, Hölder, and Minkowski inequalities from Young’s inequality. Math. Mag. 37(1), 2–12 (1964)
Wang, F., Syeda-Mahmood, T., Vemuri, B.C., Beymer, D., Rangarajan, A.: Closed-form jensen-renyi divergence for mixture of gaussians and applications to group-wise shape registration. In: Yang, G.-Z., Hawkes, D., Rueckert, D., Noble, A., Taylor, C. (eds.) MICCAI 2009. LNCS, vol. 5761, pp. 648–655. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04268-3_80
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Nielsen, F. (2019). The Statistical Minkowski Distances: Closed-Form Formula for Gaussian Mixture Models. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2019. Lecture Notes in Computer Science(), vol 11712. Springer, Cham. https://doi.org/10.1007/978-3-030-26980-7_37
Download citation
DOI: https://doi.org/10.1007/978-3-030-26980-7_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-26979-1
Online ISBN: 978-3-030-26980-7
eBook Packages: Computer ScienceComputer Science (R0)