Skip to main content
SpringerLink
Log in

Multiple Penalty Regression: Fitting and Extrapolating a Discrete Incomplete Multi-way Layout

  • Published:
Annals of the Institute of Statistical Mathematics Aims and scope Submit manuscript

Abstract

The d iscrete multi-way layout is an abstract data type associated with regression, experimental designs, digital images or videos, spatial statistics, gene or protein chips, and more. The factors influencing response can be nominal or ordinal. The observed factor level combinations are finitely discrete and often incomplete or irregularly spaced. This paper develops low risk biased estimators of the means at the observed factor level combinations; and extrapolates the estimated means to larger discrete complete layouts. Candidate penalized least squares (PLS) estimators with multiple quadratic penalties express competing conjectures about each of the main effects and interactions in the analysis of variance decomposition of the means. The candidate PLS estimator with smallest estimated quadratic risk attains, asymptotically, the smallest risk over all candidate PLS estimators. In the theoretical analysis, the dimension of the regression space tends to infinity. No assumptions are made about the unknown means or about replication.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Beran R. (2002). Improving penalized least squares through adaptive selection of penalty and shrinkage. Annals of the Institute of Statistical Mathematics 54, 900–917

    Article  MATH  MathSciNet  Google Scholar 

  • Beran R. (2005). ASP fits to multi-way layouts. Annals of the Institute of Statistical Mathematics 57, 201–220

    Article  MATH  MathSciNet  Google Scholar 

  • Beran R., Dümbgen L. (1998). Modulation of estimators and confidence sets. Annals of Statistics 26, 1826–1856

    Article  MATH  MathSciNet  Google Scholar 

  • Buja A., Hastie T., Tibshirani R. (1989). Linear smoothers and additive models (with discussion). Annals of Statistics 17, 453–555

    MATH  MathSciNet  Google Scholar 

  • Freeman H.A. (1942). Industrial statistics. Wiley, New York

    Google Scholar 

  • Heckman N.E., Ramsay J.O. (2000). Penalized regression with model-based penalties. Canadian Journal of Statistics 28, 241–258

    Article  MATH  MathSciNet  Google Scholar 

  • Hoerl A.E., Kennard R.W. (1970). Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12, 55–67

    Article  MATH  Google Scholar 

  • Kimeldorf G.S., Wahba G. (1970). A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Annals of Mathematical Statistics 41, 495–502

    MathSciNet  Google Scholar 

  • Kneip A. (1994). Ordered linear smoothers. Annals of Statistics 22, 835–866

    MATH  MathSciNet  Google Scholar 

  • Lin Yi (2000). Tensor product space ANOVA Fits. Annals of Statistics 28, 734–755

    Article  MATH  MathSciNet  Google Scholar 

  • Mallows C.L. (1973). Some comments on C p . Technometrics 15, 661–676

    Article  MATH  Google Scholar 

  • Neveu J. (1965). Mathematical foundations of the calculus of probability. Holden-Day, San Francisco

    MATH  Google Scholar 

  • Rice J. (1984). Bandwidth choice for nonparametric regression. Annals of Statistics 12, 1215–1230

    MATH  MathSciNet  Google Scholar 

  • Scheffé, H. (1959). The analysis of variance. New York: Wiley.

  • Sen A., Srivastava M. (1990). Regression analysis. Springer, Berlin Heidelberg New York

    MATH  Google Scholar 

  • Stein C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In: Neyman J. (eds). Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability. University of California Press, Berkeley, pp. 197–206

    Google Scholar 

  • Stein C. (1966). An approach to the recovery of inter-block information in balanced incomplete block designs. In: David F.N. (eds). Festschrift for Jerzy Neyman. Wiley, New York, pp. 351–364

    Google Scholar 

  • Wahba, G. (1990). Spline models for observational data. Philadelphia: Society for Industrial and Applied Mathematics.

  • Wahba G., Wang Y., Gu C., Klein R., Klein B. (1995) Smoothing spline ANOVA for exponential families with application to the Wisconsin epidemiological study of diabetic retinopathy. Annals of Statistics 23, 1868–1895

    MathSciNet  Google Scholar 

  • Wichura M.J. (1971). A note on the weak convergence of stochastic processes. Annals of Mathematical Statistics 42, 1769–1772

    MathSciNet  Google Scholar 

  • Wood S.N. (2000). Modelling and smoothing parameter estimation with multiple quadratic penalties. Journal of the Royal Statistical Society, Series B 62, 413–428

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, University of California at Davis, Davis, CA, 95616, USA

    Rudolf Beran

Authors
  1. Rudolf Beran
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rudolf Beran.

About this article

Cite this article

Beran, R. Multiple Penalty Regression: Fitting and Extrapolating a Discrete Incomplete Multi-way Layout. AISM 59, 171–195 (2007). https://doi.org/10.1007/s10463-006-0050-z

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10463-006-0050-z

Keywords

Search

Navigation