Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Multiple Monte Carlo testing, with applications in spatial point processes


The rank envelope test (Myllymäki et al. in J R Stat Soc B, doi:10.1111/rssb.12172, 2016) is proposed as a solution to the multiple testing problem for Monte Carlo tests. Three different situations are recognized: (1) a few univariate Monte Carlo tests, (2) a Monte Carlo test with a function as the test statistic, (3) several Monte Carlo tests with functions as test statistics. The rank test has correct (global) type I error in each case and it is accompanied with a p-value and with a graphical interpretation which determines subtests and distances of the used test function(s) which lead to the rejection at the prescribed significance level of the test. Examples of null hypotheses from point process and random set statistics are used to demonstrate the strength of the rank envelope test. The examples include goodness-of-fit test with several test functions, goodness-of-fit test for a group of point patterns, test of dependence of components in a multi-type point pattern, and test of the Boolean assumption for random closed sets. A power comparison to the classical multiple testing procedures is given.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12


  1. Baddeley, A., Diggle, P.J., Hardegen, A., Lawrence, T., Milne, R.K., Nair, G.: On tests of spatial pattern based on simulation envelopes. Ecol. Monogr. 84, 477–489 (2014)

  2. Baddeley, A., Rubak, E., Turner, R.: Spatial Point Patterns: Methodology and Applications with R. Chapman and Hall/CRC Press, London (2015)

  3. Baddeley, A.J., Moyeed, R.A., Boyde, A.: Analysis of a three-dimensional point pattern with replication. Appl. Stat. 42(4), 641–666 (1993)

  4. Barnard, G.A.: Discussion of professor Bartlett’s paper. J. R. Stat. Soc. B 25, 294 (1963)

  5. Benjamini, Y., Hochberg, Y.: Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. B 57(1), 289–300 (1995)

  6. Besag, J., Diggle, P.J.: Simple Monte Carlo tests for spatial pattern. J. R. Stat. Soc. C 26(3), 327–333 (1977)

  7. Chiu, S.N., Stoyan, D., Kendall, W.S., Mecke, J.: Stochastic Geometry and Its Applications, 3rd edn. Wiley, Chichester (2013)

  8. Condit, R.: Tropical Forest Census Plots. Springer and R. G. Landes Company, Berlin and Georgetown (1998)

  9. Dao, N.A., Genton, M.G.: A Monte Carlo adjusted goodness-of-fit test for parametric models describing spatial point patterns. J. Comput. Graph. Stat. 23, 497–517 (2014)

  10. Diggle, P.J.: Statistical Analysis of Spatial and Spatio-temporal Point Patterns, 3rd edn. CRC Press, Boca Raton (2013)

  11. Grabarnik, P., Myllymäki, M., Stoyan, D.: Correct testing of mark independence for marked point patterns. Ecol. Model. 222(23–24), 3888–3894 (2011)

  12. Hochberg, Y.: A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75, 800–803 (1988)

  13. Hommel, G.: A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika 75, 383–386 (1988)

  14. Hubbell, S.P., Condit, R., Foster, R.B.: Barro Colorado forest census plot data. https://ctfs.arnarb.harvard.edu/webatlas/datasets/bci (2005)

  15. Hubbell, S.P., Foster, R.B., O’Brien, S.T., Harms, K.E., Condit, R., Wechsler, B., Wright, S.J., Loo de Lao, S.: Light gap disturbances, recruitment limitation, and tree diversity in a neotropical forest. Science 283, 554–557 (1999)

  16. Illian, J., Penttinen, A., Stoyan, H., Stoyan, D.: Statistical Analysis and Modelling of Spatial Point Patterns. Wiley, Chichester (2008)

  17. Kennedy, W.R., Wendelschafer-Crabb, G., Johnson, T.: Quantitation of epidermal nerves in diabetic neuropathy. Neurology 47, 1042–1448 (1996)

  18. Loosmore, N.B., Ford, E.D.: Statistical inference using the G or K point pattern spatial statistics. Ecology 87(8), 1925–1931 (2006)

  19. López-Pintado, S., Romo, J.: On the concept of depth for functional data. J. Am. Stat. Assoc. 104(486), 718–734 (2009)

  20. López-Pintado, S., Romo, J.: A half-region depth for functional data. Comput. Stat. Data Anal. 55(4), 1679–1695 (2011)

  21. Molchanov, I.: Statistics of the Boolean Model for Practitionars and Mathematicians. Wiley, New York (1995)

  22. Mrkvička, T.: On testing of general random closed set model hypothesis. Kybernetika 45(2), 293–308 (2009)

  23. Mrkvička, T., Mattfeldt, T.: Testing histological images of mammary tissues on compatibility with the boolean model of random sets. Image Anal. Stereol. 30(1), 101–108 (2011)

  24. Myllymäki, M., Grabarnik, P., Seijo, H., Stoyan, D.: Deviation test construction and power comparison for marked spatial point patterns. Spat. Stat. 11, 19–34 (2015)

  25. Myllymäki, M., Mrkvička, T., Grabarnik, P., Seijo, H., Hahn, U.: Global envelope tests for spatial processes. J. R. Stat. Soc. B (2016). doi:10.1111/rssb.12172

  26. Myllymäki, M., Särkkä, A., Vehtari, A.: Hierarchical second-order analysis of replicated spatial point patterns with non-spatial covariates. Spat. Stat. 8, 104–121 (2014)

  27. Olsbo, V., Myllymäki, M., Waller, L.A., Särkkä, A.: Development and evaluation of spatial point process models for epidermal nerve fibers. Math. Biosci. 243(2), 178–189 (2013)

  28. Rom, D.: A sequentially rejective test procedure based on a modified bonferroni inequality. Biometrika 77, 663–665 (1990)

  29. Schladitz, K., Särkkä, A., Pavenstädt, I., Haferkamp, O., Mattfeldt, T.: Statistical analysis of intramembranous particles using freeze fracture specimens. J. Microsc. 211, 137–153 (2003)

  30. Schneider, R.: Convex Bodies. The Brunn-Minkowski Theory. Cambridge University Press, Cambridge (1993)

  31. Simes, R.J.: An improved bonferroni procedure for multiple tests of significance. Biometrika 73, 751–754 (1986)

  32. Waller, L.A., Särkkä, A., Olsbo, V., Myllymäki, M., Panoutsopoulou, I.G., Kennedy, W.R., Wendelschafer-Crabb, G.: Second-order spatial analysis of epidermal nerve fibers. Stat. Med. 30, 2827–2841 (2011)

Download references


The authors would like to express great appreciation for the helpful comments they got from the anonymous reviewers. Mrkvička has been financially supported by the Grant Agency of Czech Republic (Project No. 16-03708S) and Mari Myllymäki by the Academy of Finland (Project Numbers 250860, 294162). Hahn’s research has been supported by the Centre for Stochastic Geometry and Advanced Bioimaging, funded by the Villum Foundation. The authors thank William R. Kennedy, Gwen Wendelschafer-Crabb and Ioanna G. Panoutsopoulou for providing the ENF data and Torsten Mattfeldt for providing the intramembranous particle data. The rainforest data set origins from the Forest Dynamics Plot of Barro Colorado Island, which is made possible through the generous support of the U.S. National Science Foundation, the John D. and Catherine T. MacArthur Foundation, the Smithsonian Tropical Research Institute and through the hard work of over 100 people from 10 countries over the past two decades. The BCI Forest Dynamics Plot is part of the Center for Tropical Forest Science, a global network of large-scale demographic tree plots.

Author information

Correspondence to Tomáš Mrkvička.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Mrkvička, T., Myllymäki, M. & Hahn, U. Multiple Monte Carlo testing, with applications in spatial point processes. Stat Comput 27, 1239–1255 (2017). https://doi.org/10.1007/s11222-016-9683-9

Download citation


  • Boolean model test
  • Envelope test
  • Extreme rank ordering
  • Goodness-of-fit test
  • Multi-type point process
  • Rank envelope test
  • Superposition hypothesis

Mathematics Subject Classification

  • 62G10
  • 62H15