Statistics and Computing

, Volume 24, Issue 6, pp 941–952 | Cite as

Permutation tests for between-unit fixed effects in multivariate generalized linear mixed models



A permutation testing approach in multivariate mixed models is presented. The solutions proposed allow for testing between-unit effect; they are exact under some assumptions, while approximated in the more general case. The classes of models comprised by this approach include generalized linear models, vector generalized additive models and other nonparametric models based on smoothing. Moreover it does not assume observations of different units to have the same distribution. The extensions to a multivariate framework are presented and discussed. The proposed multivariate tests exploit the dependence among variables, hence increasing the power with respect to other standard solutions (e.g. Bonferroni correction) which combine many univariate tests in an overall one. Examples are given of two applications to real data from psychological and ecological studies; a simulation study provides some insight into the unbiasedness of the tests and their power. The methods were implemented in the R package flip, freely available on CRAN.


Permutation tests Mixed models Multivariate inference 



The authors gratefully acknowledge Elisa Di Giorgio, Chiara Turati, Gianmarco Altoè and Francesca Simion for making their data available for the psychological example and Valerio Matozzo, Andrea Chinellato, Marco Munari, Monica Bressan and Maria Gabriella Marin for making their data available for the ecological example. The authors are also grateful to an associate editor and two referees for detailed comments that helped clarify the proofs and the presentation. LF was supported by grant from the University of Padua (Progetti di Ricerca di Ateneo 2011, project CPDA117517) and by the Cariparo Foundation Excellence grant 2011/2012.


  1. Anderson, M.J.: A new method for non-parametric multivariate analysis of variance. Aust. Ecol. 26(1), 32–46 (2001). doi: 10.1111/j.1442-9993.2001.01070.pp.x Google Scholar
  2. Basso, D.: Nonparametric estimation of random effect variance with partial information from the clusters. Working paper of the Dept of Statistical Science, University of Padua 12 (2011) Google Scholar
  3. Basso, D.: Nonparametric estimation of random effect variance-covariance matrix with partial information from the clusters. Working paper of the Dept of Statistical Science, University of Padua 7 (2012) Google Scholar
  4. Basso, D., Finos, L.: A permutation test for testing between effects in linear mixed models. In: Complex Data Modeling and Computationally Intensive Statistical Methods for Estimation and Prediction (2011) Google Scholar
  5. Basso, D., Finos, L.: Exact multivariate permutation tests for fixed effects in mixed-models. Commun. Stat., Theory Methods 41(16), 2991–3001 (2012) CrossRefMATHMathSciNetGoogle Scholar
  6. Breslow, N.E., Clayton, D.G.: Approximate inference in generalized linear mixed models. J. Am. Stat. Assoc. 88(421), 9–25 (1993). doi: 10.2307/2290687 MATHGoogle Scholar
  7. Commenges, D.: Transformations which preserve exchangeability and application to permutation tests. J. Nonparametr. Stat. 15, 171–185 (2003) CrossRefMATHMathSciNetGoogle Scholar
  8. Di Giorgio, E., Turati, C., Altoè, G., Simion, F.: Face detection in complex visual displays: an eye tracking study with 3- and 6-month-old infants and adults. J. Exp. Child Psychol. 113, 66–77 (2012) CrossRefGoogle Scholar
  9. Fieuws, S., Verbeke, G., Molenberghs, G.: Random-effects models for multivariate repeated measures. Stat. Methods Med. Res. 5(16), 387–397 (2007) CrossRefMathSciNetGoogle Scholar
  10. Mardia, K.V., Kent, J.T., Bibby, J.M.: Multivariate Analysis. Academic Press, London (1992) Google Scholar
  11. Matozzo, V., Chinellato, A., Munari, M., Finos, L., Bressan, M., Marin, M.G.: First evidence of immunomodulation in bivalves under seawater acidification and increased temperature. PLoS ONE 7(3), e33820 (2012). doi: 10.1371/journal.pone.0033820 CrossRefGoogle Scholar
  12. McCullagh, P., Nelder, J.A.: Generalized Linear Models, 2nd edn. Chapman & Hall/CRC Monographs on Statistics & Applied Probability. Chapman & Hall/CRC, London (1989) CrossRefMATHGoogle Scholar
  13. Montgomery, D.C.: Design and Analysis of Experiments. Wiley, New York (2004) Google Scholar
  14. Moscatelli, A., Mezzetti, M., Lacquaniti, F.: Modeling psychophysical data at the population-level: the generalized linear mixed model. J. Vis. 12(11), 26 (2012)., doi: 10.1167/12.11.26 CrossRefGoogle Scholar
  15. Neuhaus, J., Segal, M.R.: An Assessment of Approximate Maximum Likelihood Estimators in Generalized Linear Mixed Models. Modelling Longitudinal and Spatially Correlated Data. Springer, Berlin (1997) Google Scholar
  16. Nichols, T.E., Holmes, A.P.: Nonparametric permutation tests for functional neuroimaging: a primer with examples. Hum. Brain Mapp. 15(1), 1–25 (2002). doi: 10.1002/hbm.1058 CrossRefGoogle Scholar
  17. Pesarin, F.: Multivariate Permutation Test with Application to Biostatistics. Wiley, Chichester (2001) Google Scholar
  18. Shah, A., Laird, N., Schoenfeld, D.: A random-effects model for multiple characteristics with possibly missing data. J. Am. Stat. Assoc. 5(92), 775–779 (1997) CrossRefMathSciNetGoogle Scholar
  19. Westfall, P.H., Young, S.S.: Resampling-Based Multiple Testing: Examples and Methods for P-Value Adjustment. Wiley, New York (1993) Google Scholar
  20. Yee, T.W., Wild, C.J.: Vector generalized additive models. J. R. Stat. Soc. B 58(3), 481–493 (1996) MATHMathSciNetGoogle Scholar
  21. Zhu, J., Eickhoff, C.J., Yan, P.: Generalized linear latent variable models for repeated measures of spatially correlated multivariate data. Biometrics 61(3), 674–683 (2005) CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Department of Statistical SciencesUniversity of Padua35100Italy

Personalised recommendations