Skip to main content
Book cover

Modern Nonparametric, Robust and Multivariate Methods pp 69–90Cite as

Permutation Tests in Linear Regression

  • Chapter

Abstract

Exact permutation tests are available only in rather simple linear models. The problem is that, although standard assumptions allow permuting the errors of the model, we cannot permute them in practice, because they are unobservable. Nevertheless, the residuals of the model can be permuted. A proof is given here which shows that it is possible to approximate the unobservable permutation distribution where the true errors are permuted by permuting the residuals. It is shown that approximation holds asymptotically and almost surely for certain quadratic statistics as well as for statistics which are expressible as the maximum of appropriate linear functions. The result is applied to testing the significance of predictors as well as to diagnostic checking of heteroscedasticity, autocorrelation, change-points, and changing regression function. Also a simulation experiment is made in order to evaluate the performance of the proposed tests.

Keywords

  • Autocorrelation
  • Change-point problem
  • Control variable
  • Heteroscedasticity
  • Simulation

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-22404-6_5
  • Chapter length: 22 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   109.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-22404-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   149.00
Price excludes VAT (USA)
Hardcover Book
USD   159.99
Price excludes VAT (USA)
Learn about institutional subscriptions

References

  • Anderson, M.J., Robinson, J.: Permutation tests for linear models. Aust. N. Z. J. Stat. 43, 75–88 (2001)

    MathSciNet  CrossRef  MATH  Google Scholar 

  • Anderson, M.J., ter Braak, C.J.F.: Permutation tests for multi-factorial analysis of variance. J. Stat. Comput. Simul. 73, 85–113 (2003)

    MathSciNet  CrossRef  MATH  Google Scholar 

  • Billingsley, P.: Probability and Measure. Wiley, New York (1979)

    MATH  Google Scholar 

  • Chen, J.: Testing for a change point in linear regression models. Commun. Stat. Theory Methods 27, 2481–2493 (1998)

    MathSciNet  CrossRef  MATH  Google Scholar 

  • Cook, R.D., Weisberg, S.: Residuals and Influence in Regression. Chapman and Hall, New York (1982)

    MATH  Google Scholar 

  • Cook, R.D., Weisberg, S.: Diagnostics for heteroscedasticity in regression. Biometrika 70, 1–10 (1983)

    MathSciNet  CrossRef  MATH  Google Scholar 

  • Edgington, E.S.: Randomization Tests, 3rd edn. Marcel Dekker, New York (1995)

    MATH  Google Scholar 

  • Efron, B.: Bootstrap methods: another look at the jacknife. Ann. Stat. 7, 1–26 (1979)

    MathSciNet  CrossRef  MATH  Google Scholar 

  • Fisher, R.A.: Design of Experiments. Oliver and Boyd, Edinburgh (1935)

    Google Scholar 

  • Freedman, D., Lane, D.: A nonstochastic interpretation of reported significance levels. J. Bus. Econ. Stat. 1, 292–298 (1983)

    Google Scholar 

  • Good, P.: Permutation Tests, 3rd edn. Springer, New York (2005)

    MATH  Google Scholar 

  • Huh, M.-H., Jhun, M.: Random permutation testing in multiple linear regression. Commun. Stat. Theory Methods 30, 2023–2032 (2001)

    MathSciNet  CrossRef  MATH  Google Scholar 

  • Holbert, D.: A Bayesian analysis of a switching linear model. J. Econ. 19, 77–87 (1982)

    CrossRef  Google Scholar 

  • Imhof, J.P.: Computing the distribution of quadratic forms in normal variables. Biometrika 48, 419–426 (1961)

    MathSciNet  CrossRef  MATH  Google Scholar 

  • Kempthorne, O.: Randomization–II. In: Kotz, S., Johnson, N.L. (eds.) Encyclopedia of Statistical Sciences, vol. 7, pp. 519–524. Wiley, New York (1986)

    Google Scholar 

  • Kennedy, P.E.: Randomization tests in econometrics. J. Bus. Econ. Stat. 13 85–94 (1995)

    MathSciNet  Google Scholar 

  • Kim, H.-J., Siegmund, D.: The likelihood ratio test for a change-point in simple linear regression. Biometrika 76, 409–423 (1989)

    MathSciNet  CrossRef  MATH  Google Scholar 

  • Kim, H.-J., Fay, M.P., Feuer, E.J., Midthune, D.: Permutation tests for jointpoint regression with applications to cancer rates. Stat. Med. 19, 335–351 (2000)

    CrossRef  Google Scholar 

  • Koutsoyiannis, A.: Theory of Econometrics. Macmillan, London (1973)

    MATH  Google Scholar 

  • Lehmann, E.L., Romano, J.P.: Testing Statistical Hypotheses, 3rd edn. Chapman and Hall, New York (2005)

    MATH  Google Scholar 

  • LePage, R., Podgórski, K.: Resampling permutations in regression without second moments. J. Multivar. Anal. 57, 119–141 (1996)

    MathSciNet  CrossRef  MATH  Google Scholar 

  • Loève, M.: Probability Theory. Van Nostrand, New York (1963)

    MATH  Google Scholar 

  • Manly, B.F.J.: Randomization, Bootstrap, and Monte Carlo Methods in Biology, 2nd edn. Chapman and Hall, London (1997)

    MATH  Google Scholar 

  • Nyblom, J.: Testing for deterministic linear trend in time series. J. Am. Stat. Assoc. 81, 545–549 (1986)

    MathSciNet  CrossRef  MATH  Google Scholar 

  • Nyblom, J., Mäkeläinen, T.: Comparisons of tests for the presence of random walk coefficients in a simple linear model. J. Am. Stat. Assoc. 78, 856–864 (1983)

    MathSciNet  CrossRef  MATH  Google Scholar 

  • Oja, H.: On permutation tests in multiple regression and analysis of covariance problems. Aust. J. Stat. 29, 91–100 (1987)

    MathSciNet  CrossRef  MATH  Google Scholar 

  • Pesarin, F.: Multivariate Permutation Tests. Wiley, New York (2001)

    MATH  Google Scholar 

  • Pitman, E.J.G.: Significance tests which may be applied to samples from any populations. J. R. Stat. Soc. B 4, 119–130 (1937a)

    MATH  Google Scholar 

  • Pitman, E.J.G.: Significance tests which may be applied to samples from any populations. II. The correlation coefficient. J. R. Stat. Soc. B 4, 225–232 (1937b)

    MATH  Google Scholar 

  • Pitman, E.J.G.: Significance tests which may be applied to samples from any populations. III. The analysis of variance test. Biometrika 29, 322–335 (1938)

    MATH  Google Scholar 

  • Ryan, T., Joiner, B., Ryan, B.: Minitab Student Handbook. Duxbury, North Scituate (1976)

    Google Scholar 

  • Rao, C.R.: Linear Statistical Inference and Its Applications, 2nd edn. Wiley, New York (1973)

    CrossRef  MATH  Google Scholar 

  • Schmoyer, R.L.: Permutation tests for correlation in regression errors. J. Am. Stat. Assoc. 89, 1507–1516 (1994)

    MathSciNet  CrossRef  MATH  Google Scholar 

  • Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6, 461–464 (1978)

    MathSciNet  CrossRef  MATH  Google Scholar 

  • Serfling, R.J.: Approximation Theorems of Mathematical Statistics. Wiley, New York (1980)

    CrossRef  MATH  Google Scholar 

Download references

Acknowledgements

The author thanks two anonymous referees for their useful comments.

Author information

Authors and Affiliations

  1. Department of Mathematics and Statistics, University of Jyväskylä, 35, FIN-40014, Jyväskylä, Finland

    Jukka Nyblom

Authors
  1. Jukka Nyblom
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jukka Nyblom .

Editor information

Editors and Affiliations

  1. Department of Mathematics and Statistics, School of Health Sciences, University of Turku, Turku, Finland

    Klaus Nordhausen

  2. Department of Mathematics and Statistics, University of Jyväskylä, Jyväskylä, Finland

    Sara Taskinen

Rights and permissions

Reprints and Permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Nyblom, J. (2015). Permutation Tests in Linear Regression. In: Nordhausen, K., Taskinen, S. (eds) Modern Nonparametric, Robust and Multivariate Methods. Springer, Cham. https://doi.org/10.1007/978-3-319-22404-6_5

Download citation