Abstract
We consider asymptotic inference for the concentration of directional data. More precisely, we propose tests for concentration (1) in the low-dimensional case where the sample size n goes to infinity and the dimension p remains fixed, and (2) in the high-dimensional case where both n and p become arbitrarily large. To the best of our knowledge, the tests we provide are the first procedures for concentration that are valid in the (n, p)-asymptotic framework. Throughout, we consider parametric FvML tests, that are guaranteed to meet asymptotically the nominal level constraint under FvML distributions only, as well as “pseudo-FvML” versions of such tests, that meet asymptotically the nominal level constraint within the whole class of rotationally symmetric distributions. We conduct a Monte-Carlo study to check our asymptotic results and to investigate the finite-sample behavior of the proposed tests.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Amos, D.E.: Computation of modified Bessel functions and their ratios. Math. Comput. 28 (125), 239–251 (1974)
Banerjee, A., Ghosh, J.: Frequency sensitive competitive learning for scalable balanced clustering on high-dimensional hyperspheres. IEEE Trans. Neural Netw. 15, 702–719 (2004)
Banerjee, A., Dhillon, I.S., Ghosh, J., Sra, S.: Clustering on the unit hypersphere using von Mises-Fisher distributions. J. Mach. Learn. Res. 6, 1345–1382 (2005)
Briggs, M.S.: Dipole and quadrupole tests of the isotropy of gamma-ray burst locations. Astrophys. J. 407, 126–134 (1993)
Cai, T., Jiang, T.: Phase transition in limiting distributions of coherence of high-dimensional random matrices. J. Multivar. Anal. 107, 24–39 (2012)
Cai, T., Fan, J., Jiang, T.: Distributions of angles in random packing on spheres. J. Mach. Learn. Res. 14, 1837–1864 (2013)
Cutting, C., Paindaveine, D., Verdebout, T.: testing uniformity on high-dimensional spheres against monotone rotationally symmetric alternatives. Ann. Stat. (to appear)
Dryden, I.L.: Statistical analysis on high-dimensional spheres and shape spaces. Ann. Statist. 33, 1643–1665 (2005)
Fisher, R.A.: Dispersion on a sphere. Proc. R. Soc. Lond. Ser. A 217, 295–305 (1953)
Fisher, N.: Problems with the current definitions of the standard deviation of wind direction. J. Clim. Appl. Meteorol. 26 (11), 1522–1529 (1987)
Ko, D.: Robust estimation of the concentration parameter of the von Mises-Fisher distribution. Ann. Statist. 20 (2), 917–928 (1992)
Ko, D., Guttorp, P.: Robustness of estimators for directional data. Ann. Statist. 16 (2), 609–618 (1988)
Larsen, P., Blæsild, P., Sørensen, M.: Improved likelihood ratio tests on the von Mises–Fisher distribution. Biometrika 89 (4), 947–951 (2002)
Ley, C., Verdebout, T.: Local powers of optimal one-and multi-sample tests for the concentration of Fisher-von Mises-Langevin distributions. Int. Stat. Rev. 82, 440–456 (2014)
Ley, C., Paindaveine, D., Verdebout, T.: High-dimensional tests for spherical location and spiked covariance. J. Multivar. Anal. 139, 79–91 (2015)
Mardia, K.V., Jupp, P.E.: Directional Statistics, vol. 494. Wiley, New York (2009)
Paindaveine, D., Verdebout, T.: On high-dimensional sign tests. Bernoulli 22, 1745–1769 (2016)
Paindaveine, D., Verdebout, T.: Optimal rank-based tests for the location parameter of a rotationally symmetric distribution on the hypersphere. In: Hallin, M., Mason, D., Pfeifer, D., Steinebach, J. (eds.) Mathematical Statistics and Limit Theorems: Festschrift in Honor of Paul Deheuvels, pp. 249-270. Springer (2015)
Silverman, B.W.: Density Estimation for Statistics and Data Analysis, vol. 26. CRC Press, London (1986)
Stephens, M.: Multi-sample tests for the fisher distribution for directions. Biometrika 56 (1), 169–181 (1969)
Verdebout, T.: On some validity-robust tests for the homogeneity of concentrations on spheres. J. Nonparametr. Stat. 27, 372–383 (2015)
Watamori, Y., Jupp, P.E.: Improved likelihood ratio and score tests on concentration parameters of von Mises–Fisher distributions. Stat. Probabil. Lett. 72 (2), 93–102 (2005)
Watson, G.S.: Statistics on Spheres. Wiley, New York (1983)
Acknowledgements
D. Paindaveine’s research supported by an A.R.C. contract from the Communauté Française de Belgique and by the IAP research network grant P7/06 of the Belgian government (Belgian Science Policy).
T. Verdebout’s research is supported by a grant from the “Banque Nationale de Belgique”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix
Appendix
Proof of Theorem 1
(i) All expectations and variances when proving Part (i) of the theorem are taken under \(\mathcal{R}_{p}^{(n)}(\boldsymbol{\theta },F)\) and all stochastic convergences are taken as n → ∞ under \(\mathcal{R}_{p}^{(n)}(\boldsymbol{\theta },F)\). Since
the delta method (applied to the mapping x ↦ x∕ ∥ x ∥ ) yields
where we wrote \(\mathbf{Y}_{n}:=\bar{ \mathbf{X}}_{n}/\|\bar{\mathbf{X}}_{n}\|\). This, and the fact that
where I p denotes the p-dimensional identity matrix, readily implies that
Now, write
say. It directly follows from (5) to (7) that S 1n = o P(1) as n → ∞. As for S 2n , the central limit theorem and Slutsky’s lemma yield that S 2n is asymptotically standard normal. This readily implies that
(ii) In view of the derivations above, the continuous mapping theorem implies that, for any \(\boldsymbol{\theta }\in \mathcal{S}^{p-1}\) and \(F \in \mathcal{F}_{0}\),
as n → ∞ under \(\mathcal{R}_{p}^{(n)}(\boldsymbol{\theta },F)\). The result then follows from the fact that, under \(\mathcal{R}_{p}^{(n)}(\boldsymbol{\theta },F_{p,\kappa _{0}})\), with κ 0 = h p −1(e 10), \(\mathrm{Var}[\mathbf{X}_{1}^{{\prime}}\boldsymbol{\theta }] = 1 -\frac{p-1} {\kappa _{0}} e_{10} - e_{10}^{2};\) see, e.g., Lemma S.2.1 from [7]. □
Proof of Proposition 1
From Lemma S.2.1 in [7], we have that, under \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{p_{n},\kappa _{n}})\),
The result then readily follows from
for any ν, z > 0; see (9) in [1]. □
Proof of Theorem 2
Writing \(e_{n2}:= \mathrm{E}[(\mathbf{X}_{n1}^{{\prime}}\boldsymbol{\theta }_{n})^{2}]\), Theorem 5.1 in [7] entails that, under \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{p_{n},\kappa _{n}})\), where (κ n ) is an arbitrary sequence in (0, ∞),
converges weakly to the standard normal distribution as n → ∞. The result then follows from the fact that, under \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{p_{n},\kappa _{n}})\), where the sequence (κ n ) is such that, for any n, e n1 = e 10 under \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{p_{n},\kappa _{n}})\), one has
see Proposition 1(ii). □
The proof of Theorem 3 requires the three following preliminary results:
Lemma 1
Let Z be a random variable such that P [|Z|≤ 1] = 1. Then Var [Z 2 ] ≤ 4 Var [Z].
Lemma 2
Let the assumptions of Theorem 3 hold. Write \(\hat{e}_{n1} =\|\bar{ \mathbf{X}}_{n}\|\) and \(\hat{e}_{n2}:=\bar{ \mathbf{X}}_{n}^{{\prime}}\mathbf{S}_{n}\bar{\mathbf{X}}_{n}/\|\bar{\mathbf{X}}_{n}\|^{2}\) . Then, as n →∞ under \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{p_{n},\kappa _{n}})\) , (i) \((\hat{e}_{n1}^{2} - e_{10}^{2})/(e_{n2} - e_{10}^{2}) = o_{\mathrm{P}}(1)\) and (ii) \((\hat{e}_{2n} - e_{n2})/(e_{n2} - e_{10}^{2}) = o_{\mathrm{P}}(1)\).
Lemma 3
Let the assumptions of Theorem 3 hold. Write σ n 2 := p n (e n2 − e 10 2 ) 2 + 2np n e 10 2 (e n2 − e 10 2 ) + (1 − e n2 ) 2 and \(\hat{\sigma }_{n}^{2}:= p_{n}(\hat{e}_{n2} -\hat{ e}_{n1}^{2})^{2} + 2np_{n}e_{10}^{2}(\hat{e}_{n2} -\hat{ e}_{n1}^{2}) + (1 -\hat{ e}_{n2})^{2}\) . Then \((\hat{\sigma }_{n}^{2} -\sigma _{n}^{2})/\sigma _{n}^{2} = o_{\mathrm{P}}(1)\) as n →∞ under \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{p_{n},\kappa _{n}})\).
Proof of Lemma 1
Let Z a and Z b be mutually independent and identically distributed with the same distribution as Z. Since | x 2 − y 2 | ≤ 2 | x − y | for any x, y ∈ [−1, 1], we have that
which proves the result. □
Proof of Lemma 2
All expectations and variances in this proof are taken under the sequence of hypotheses \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{n})\) considered in the statement of Theorem 3, and all stochastic convergences are taken as n → ∞ under the same sequence of hypotheses. (i) Proposition 5.1 from [7] then yields
and
as n → ∞. In view of Condition (i) in Theorem 3, this readily implies
as n → ∞, which establishes Part (i) of the result.
(ii) Write
Part (i) of the result shows that \((\hat{e}_{n1}^{2} - e_{10}^{2})/\tilde{e}_{n2}\) is o P(1) as n → ∞. Since (10) and (11) yield that \(\hat{e}_{n1}\) converges in probability to e 10( ≠ 0), this implies that \((\hat{e}_{n1}^{-2} - e_{10}^{-2})/\tilde{e}_{n2}\) is o P(1) as n → ∞. This, and the fact that \(\bar{\mathbf{X}}_{n}^{{\prime}}\mathbf{S}_{n}\bar{\mathbf{X}}_{n} = O_{\mathrm{P}}(1)\) as n → ∞, readily yields
as n → ∞. Since
the result follows if we can prove that
all are o P(1) as n → ∞.
Starting with A n , (10) yields
as n → ∞. Since convergence in L 1 is stronger than convergence in probability, this implies that A n = o P(1) as n → ∞. Turning to B n , the Cauchy–Schwarz inequality and (13) provide
as n → ∞, so that B n is indeed o P(1) as n → ∞. Finally, it follows from Lemma 1 that
as n → ∞, so that C n is also o P(1) as n → ∞. This establishes the result. □
Proof of Lemma 3
As in the proof of Lemma 2, all expectations and variances in this proof are taken under the sequence of hypotheses \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{n})\) considered in the statement of Theorem 3, and all stochastic convergences are taken as n → ∞ under the same sequence of hypotheses.
Let then \(\tilde{\sigma }_{n}^{2}:= 2np_{n}e_{10}^{2}(e_{n2} - e_{10}^{2})\). Since Condition (i) in Theorem 3 directly entails that \(\sigma _{n}^{2}/\tilde{\sigma }_{n}^{2} \rightarrow 1\) as n → ∞, it is sufficient to show that \((\hat{\sigma }_{n}^{2} -\sigma _{n}^{2})/\tilde{\sigma }_{n}^{2}\) is o P(1) as n → ∞. To do so, write
where
and
Since
almost surely, Condition (i) in Theorem 3 implies that \(A_{n}/\tilde{\sigma }_{n}^{2}\) and \(C_{n}/\tilde{\sigma }_{n}^{2}\) are o P(1) as n → ∞. The result then follows from the fact that, in view of Lemma 2,
is also o P(1) as n → ∞. □
Proof of Theorem 3
Decompose Q CPVm (n) into
say. Theorem 5.1 in [7] entails that, under the sequence of hypotheses \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{n})\) considered in the statement of the theorem, V n is asymptotically standard normal as n → ∞. The result therefore follows from Lemma 3 and the Slutsky’s lemma. □
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Cutting, C., Paindaveine, D., Verdebout, T. (2017). Tests of Concentration for Low-Dimensional and High-Dimensional Directional Data. In: Ahmed, S. (eds) Big and Complex Data Analysis. Contributions to Statistics. Springer, Cham. https://doi.org/10.1007/978-3-319-41573-4_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-41573-4_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-41572-7
Online ISBN: 978-3-319-41573-4
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)