, Volume 82, Issue 1, pp 1–16 | Cite as

Four simple axioms of dependence measures

  • Tamás F. MóriEmail author
  • Gábor J. Székely


Recently new methods for measuring and testing dependence have appeared in the literature. One way to evaluate and compare these measures with each other and with classical ones is to consider what are reasonable and natural axioms that should hold for any measure of dependence. We propose four natural axioms for dependence measures and establish which axioms hold or fail to hold for several widely applied methods. All of the proposed axioms are satisfied by distance correlation. We prove that if a dependence measure is defined for all bounded nonconstant real valued random variables and is invariant with respect to all one-to-one measurable transformations of the real line, then the dependence measure cannot be weakly continuous. This implies that the classical maximal correlation cannot be continuous and thus its application is problematic. The recently introduced maximal information coefficient has the same disadvantage. The lack of weak continuity means that as the sample size increases the empirical values of a dependence measure do not necessarily converge to the population value.


Correlation Distance correlation Maximal correlation Maximal information coefficient Invariance 


  1. Dedecker J, Prieur C (2005) New dependence coefficients. Examples and applications to statistics. Probab Theory Relat Fields 132:203–236. MathSciNetCrossRefzbMATHGoogle Scholar
  2. Dueck J, Edelmann D, Gneiting T, Richards D (2014) The affinely invariant distance correlation. Bernoulli 20:2305–2330. MathSciNetCrossRefzbMATHGoogle Scholar
  3. Eaton ML (1989) Group invariance. Applications in statistics, NSF-CBMS regional conference series in probability and statistics 1. IMS, HaywardGoogle Scholar
  4. Escoufier Y (1973) Le Traitement des Variables Vectorielles. Biometrics 29:751–760. MathSciNetCrossRefGoogle Scholar
  5. Gebelein H (1941) Das statistische Problem der Korrelation als Variations- und Eigenwert-problem und sein Zusammenhang mit der Ausgleichungsrechnung. Z Angew Math Mech 21:364–379. MathSciNetCrossRefzbMATHGoogle Scholar
  6. Gouvêa FQ (2011) Was cantor surprised? Am Math Mon 118:198–209. MathSciNetCrossRefzbMATHGoogle Scholar
  7. Hirschfeld HO (1935) A connection between correlation and contingency. Math Proc Camb Philos Soc 31:520–524. CrossRefzbMATHGoogle Scholar
  8. Hoeffding W (1940) Masstabinvariante Korrelationstherie. Schr Math Inst und Inst Angew Math Univ Berlin 5:181–233Google Scholar
  9. Hoeffding W (1948) A non-parametric test of independence. Ann Math Stat 19:546–557. MathSciNetCrossRefzbMATHGoogle Scholar
  10. Huang Q, Zhu Y (2016) Model-free sure screening via maximum correlation. J Multivar Anal 148:89–106. MathSciNetCrossRefzbMATHGoogle Scholar
  11. Jakobsen ME (2017) Distance covariance in metric spaces: non-parametric independence testing in metric spaces. arXiv:1706.03490. Accessed 9 Jan 2018
  12. Josse J, Holmes S (2014) Tests of independence and beyond. arXiv:1307.7383v3. Accessed 9 Jan 2018
  13. Kendall MG (1938) A new measure of rank correlation. Biometrika 30:81–93. CrossRefzbMATHGoogle Scholar
  14. Kimeldorf G, Sampson AR (1978) Monotone dependence. Ann Stat 6:895–903. MathSciNetCrossRefzbMATHGoogle Scholar
  15. Lehmann EL (1966) Some concepts of dependence. Ann Math Stat 37:1137–1153. MathSciNetCrossRefzbMATHGoogle Scholar
  16. Lehmann EL, Romano JP (2005) Testing statistical hypotheses, 3rd edn. Springer, New York. zbMATHGoogle Scholar
  17. Linfoot EH (1957) An informational measure of correlation. Inf Control 1:85–89. MathSciNetCrossRefzbMATHGoogle Scholar
  18. López Blázquez F, Salamanca Miño B (2014) Maximal correlation in a non-diagonal case. J Multivar Anal 131:265–278. MathSciNetCrossRefzbMATHGoogle Scholar
  19. Lyons R (2013) Distance covariance in metric spaces. Ann Probab 41:3284–3305. MathSciNetCrossRefzbMATHGoogle Scholar
  20. Papadatos N (2014) Some counterexamples concerning maximal correlation and linear regression. J Multivar Anal 126:114–117. MathSciNetCrossRefzbMATHGoogle Scholar
  21. Papadatos N, Xifara T (2013) A simple method for obtaining the maximal correlation coefficient and related characterizations. J Multivar Anal 118:102–114. MathSciNetCrossRefzbMATHGoogle Scholar
  22. Pearson K (1920) Notes on the history of correlation. Biometrika 13:25–45. CrossRefGoogle Scholar
  23. Reimherr M, Nicolae DL (2013) On quantifying dependence: a framework for developing interpretable measures. Stat Sci 28:116–130. MathSciNetCrossRefzbMATHGoogle Scholar
  24. Rényi A (1959) On measures of dependence. Acta Mat Acad Sci Hung 10:441–451. MathSciNetCrossRefzbMATHGoogle Scholar
  25. Reshef DN, Reshef YA, Finucane HK, Grossman SR, McVean G, Turnbaugh PJ, Lander ES, Mitzenmacher M, Sabeti PC (2011) Detecting novel associations in large data sets. Science 334(6062):1518–1524. CrossRefzbMATHGoogle Scholar
  26. Reshef YA, Reshef DN, Finucane HK, Sabeti PC, Mitzenmacher M (2016) Measuring dependence powerfully and equitably. J Mach Learn Res 17(212):1–63MathSciNetzbMATHGoogle Scholar
  27. Richards DStP (2017) Distance correlation: a new tool for detecting association and measuring correlation between data sets. Plenary talk at the Joint Mathematics Meeting, Atlanta, 2017. Not Am Math Soc 64:16–18. Google Scholar
  28. Sampson AR (1984) A multivariate correlation ratio. Stat Probab Lett 2:77–81. MathSciNetCrossRefzbMATHGoogle Scholar
  29. Schweizer B, Wolff EF (1981) On nonparametric measures of dependence for random variables. Ann Stat 9:879–885. MathSciNetCrossRefzbMATHGoogle Scholar
  30. Sejdinovic D, Sriperumbudur B, Gretton A, Fukumiyu K (2013) Equivalence of distance-based and RKHS-based statistics in hypothesis testing. Ann Stat 41:2263–2291. MathSciNetCrossRefzbMATHGoogle Scholar
  31. Simon N, Tibshirani R (2011) Comment on “Detecting novel associations in large data set” by Reshef et al. Science Dec 16, 2011. arXiv:1401.7645v1. Accessed 9 Jan 2018
  32. Spearman C (1904) A proof and measurement of association between two things. Am J Psychol 15:72–101. CrossRefGoogle Scholar
  33. Speed T (2011) A correlation for the 21st century. Science 334(6062):1502–1503. CrossRefGoogle Scholar
  34. Stigler S (1989) Francis Galton’s account of the invention of correlation. Stat Sci 4:73–79. MathSciNetCrossRefzbMATHGoogle Scholar
  35. Székely GJ, Rizzo ML, Bakirov NK (2007) Measuring and testing independence by correlation of distances. Ann Stat 35:2769–2794. CrossRefzbMATHGoogle Scholar
  36. Székely GJ, Rizzo ML (2009) Brownian distance covariance. Ann Appl Stat 3:1236–1265. MathSciNetCrossRefzbMATHGoogle Scholar
  37. Székely GJ, Rizzo ML (2014) Partial distance correlation with methods for dissimilarities. Ann Stat 42:2382–2412. MathSciNetCrossRefzbMATHGoogle Scholar
  38. Volokh E (2015) Zero correlation between State Homicide and State Gun Laws. The Washington Post, October 6, 2015Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Probability Theory and StatisticsELTE Eötvös Loránd UniversityBudapestHungary
  2. 2.National Science FoundationAlexandriaUSA
  3. 3.Rényi Institute of MathematicsHungarian Academy of SciencesBudapestHungary

Personalised recommendations