Skip to main content
SpringerLink
Log in

On the equivalence of the weighted least squares and the generalised least squares estimators, with applications to kernel smoothing

  • Published:
Annals of the Institute of Statistical Mathematics Aims and scope Submit manuscript

Abstract

This paper establishes the conditions under which the generalised least squares estimator of the regression parameters is equivalent to the weighted least squares estimator. The equivalence conditions have interesting applications in local polynomial regression and kernel smoothing. Specifically, they enable to derive the optimal kernel associated with a particular covariance structure of the measurement error, where optimality has to be intended in the Gauss-Markov sense. For local polynomial regression it is shown that there is a class of covariance structures, associated with non-invertible moving average processes of given orders which yield the Epanechnikov and the Henderson kernels as the optimal kernels.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aitken A.C. (1935) On the least squares and linear combinations of observations. Proceedings of the Royal Society of Edinburgh, A 55: 42–48

    Google Scholar 

  • Amemiya T. (1985) Advanced econometrics. Harvard University Press, Cambridge

    Google Scholar 

  • Anderson T.W. (1948) On the theory of testing serial correlation. Skandinavisk Aktuarietidskrift 31: 88–116

    Google Scholar 

  • Anderson T.W. (1971) The statistical analysis of time series. Wiley, New York

    MATH  Google Scholar 

  • Baksalary J.K., Kala R. (1983) On equalities between BLUEs, WLSEs and SLSEs. The Canadian Journal of Statistics 11: 119–123

    Article  MathSciNet  MATH  Google Scholar 

  • Baksalary J.K., Van Eijnsbergeren A.C (1988) A comparison of two criteria for ordinary-least-squares estimators to be best linear unbiased estimators. The American Statistician 42: 205–208

    Article  MathSciNet  Google Scholar 

  • Benedetti J.K. (1977) On the nonparametric estimation of regression functions. Journal of the Royal Statistical Society, Series B 39: 248–253

    MathSciNet  MATH  Google Scholar 

  • Cleveland, W. S. (1979). Robust locally weighted regression and smoothing scatterplots. Journal of the American Statistical Association, 64, 368, 829–836.

    Google Scholar 

  • Dagum E.B. (1980) The X-11-ARIMA seasonal adjustment method. Statistics Canada, Ottawa

    Google Scholar 

  • Epanechnikov V.A. (1969) Nonparametric estimation of a multivariate probability density. Theory of Probability and Applications 14: 153–158

    Article  Google Scholar 

  • Fan J., Gjibels I. (1996) Local polynomial modelling and its applications. Chapman and Hall, New York

    MATH  Google Scholar 

  • Findley D.F., Monsell B.C., Bell W.R., Otto M.C., Chen B. (1998) New capabilities and methods of the X12-ARIMA seasonal adjustment program. Journal of Business and Economic Statistics 16: 2

    Article  Google Scholar 

  • Grenander U., Rosenblatt M. (1957) Statistical analysis of stationary time series. Wiley, New York

    MATH  Google Scholar 

  • Gross J., Trenkler G. (1997) When do linear transforms of ordinary least squares and Gauss-Markov estimator coincide? Sankhyā 59-A(2): 175–178

    MathSciNet  Google Scholar 

  • Gross J., Trenkler G. (1998) On the equality of linear statistics in the general Gauss-Markov Models. In: Mukherjee S.P., Basu S.K., Sinha B.K. (eds) Frontiers in probability and statistics. Narosa Publishing House, New Delhi, pp 189–194

    Google Scholar 

  • Gross J., Trenkler G., Werner H.J. (2001) The equality of linear transforms of the ordinary least squares estimator and the best linear unbiased estimator. Sankhyā 63-A(1): 118–127

    MathSciNet  Google Scholar 

  • Hannan E.J. (1970) Multiple time series. Wiley, New York

    Book  MATH  Google Scholar 

  • Henderson R. (1916) Note on graduation by adjusted average. Transaction of the Actuarial Society of America 17: 43–48

    Google Scholar 

  • Horn R.A., Olkin I. (1996) When does A * A = B * B and why does one want to know? The American Mathematical Monthly 103(6), 470–482

    Article  MathSciNet  MATH  Google Scholar 

  • Hoskins, W. D., Ponzo, P. J. (1972). Some properties of a class of band matrices. Mathematics of Computation, 26, (118), 393–400.

    Google Scholar 

  • Jaeger A., Krämer W. (1998) A final twist on the equality of OLS and GLS. Statistical Papers 39: 321–324

    Article  MathSciNet  MATH  Google Scholar 

  • Kenny P.B., Durbin J. (1982) Local trend estimation and seasonal adjustment of economic and social time series. Journal of the Royal Statistical Society A 145(I): 1–41

    Google Scholar 

  • Krämer W. (1980) A note on the equality of ordinary least squares and Gauss-Markov estimates in the general linear model. Sankhyā, A 42: 130–131

    MATH  Google Scholar 

  • Krämer W. (1986) Least-squares regression when the independent variable follows an ARIMA process. Journal of the American Statistical Association 81: 150–154

    Article  MathSciNet  MATH  Google Scholar 

  • Krämer W., Hassler U. (1998) Limiting efficiency of OLS vs. GLS when regressors are fractionally integrated. Economics Letters 60(3): 285–290

    Article  MathSciNet  MATH  Google Scholar 

  • Kruskal W. (1968) When are Gauss-Markov and least squares estimators identical? A coordinate-free approach. The Annals of Mathematical Statistics 39: 70–75

    Article  MathSciNet  MATH  Google Scholar 

  • Ladiray, D., Quenneville, B. (2001). Seasonal adjustment with the X-11 method. Lecture Notes in Statistics. New York: Springer.

  • Loader C. (1999) Local regression and likelihood. Springer, New York

    MATH  Google Scholar 

  • Lowerre J. (1974) Some relationships between BLUEs, WLSEs and SLSEs. Journal of the American Statistical Association 69: 223–225

    Article  MathSciNet  MATH  Google Scholar 

  • Macaulay F.R. (1931) The smoothing of time series. National Bureau of Economic Research, New York

    Google Scholar 

  • Magnus J.R., Neudecker H. (2007) Matrix differential calculus with applications in statistics and econometrics (3rd ed.). Wiley, Chichester

    Google Scholar 

  • McAleer M. (1992) Efficient estimation: The Rao-Zyskind condition, Kruskal’s theorem and ordinary least squares. Economic Record 68: 65–72

    Article  Google Scholar 

  • McElroy F.W. (1967) A necessary and sufficient condition that ordinary least-squares estimators be best linear unbiased. Journal of the American Statistical Association 62: 1302–1304

    Article  MathSciNet  Google Scholar 

  • Meyer C.D. (2000) Matrix analysis and applied linear algebra. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  • Mukherjee, B. N., Maiti, S. S. (1988). On conditions for equality of OLSE, GLSE and MLE in the analysis of covariance structures. Calcutta Statistical Association Bullettin, 171–191.

  • Müller H.G. (1984) Smooth optimum kernel estimators of densities, regression curves and modes. The Annals of Statistics 12(2): 766–774

    Article  MathSciNet  MATH  Google Scholar 

  • Müller H.G. (1987) Weighted local regression and kernel methods for nonparametric curve fitting. Journal of the American Statistical Association 82: 231–238

    Article  MathSciNet  MATH  Google Scholar 

  • Nadaraya E.A. (1964) On estimating regression. Theory of Probability and its Applications 9: 141–142

    Article  Google Scholar 

  • Phillips P.C.B. (1992) Geometry of the equivalence of OLS and GLS in the linear model. Econometric Theory 8(1): 158–159

    Article  Google Scholar 

  • Phillips P.C.B., Park J.Y. (1988) Asymptotic equivalence of OLS and GLS in regressions with integrated regressors. Journal of the American Statistical Association 83: 111–115

    Article  MathSciNet  MATH  Google Scholar 

  • Priestley M.B., Chao M.T. (1972) Nonparametric function fitting. Journal of the Royal Statistical Society, Series B 34: 384–392

    MathSciNet  Google Scholar 

  • Puntanten S., Styan G.P.H. (1989) The equality of the ordinary least squares estimator and the best linear unbiased estimator. The American Statistician 43(3): 153–161

    Article  MathSciNet  Google Scholar 

  • Tian Y., Weins D.P. (2006) On equality and proportionality of ordinary least squares, weighted least squares and best linear unbiased estimators in the general linear model. Statistics and Probability Letters 76: 1265–1272

    Article  MathSciNet  MATH  Google Scholar 

  • Wallis K. (1983) Models for X-11 and X-11 forecast procedures for preliminary and revised seasonal adjustments. In: Zellner A. (eds) Applied time series analysis of economic data. Bureau of the Census, Washington DC, pp 3–11

    Google Scholar 

  • Wand, M. P., Jones, M. C. (1995). Kernel smoothing. In Monographs on Statistics and Applied Probability, 60. London: Chapman and Hall.

  • Watson G.S. (1964) Smooth regression analysis. Sankhyā, A 26: 359–372

    MATH  Google Scholar 

  • Watson G.S. (1967) Linear least squares regression. The Annals of Mathematical Statistics 38: 1679–1699

    Article  MATH  Google Scholar 

  • Zyskind G. (1967) On canonical forms, non-negative covariance matrices and best and simple least squares linear estimators in linear models. The Annals of Mathematical Statistics 38: 1092–1119

    Article  MathSciNet  MATH  Google Scholar 

  • Zyskind G. (1969) Parametric argumentations and error structures under which certain simple least squares and analysis of variance procedures are also best. Journal of the American Statistical Association 64: 1353–1368

    Article  MathSciNet  Google Scholar 

  • Zyskind G., Martin F.B. (1969) On best linear estimation and a general Gauss-Markoff theorem in linear models with arbitrary non-negative structure. SIAM Journal of Applied Mathematics 17: 1190–1202

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, University of Bologna, via Belle Arti 41, 40126, Bologna, Italy

    Alessandra Luati

  2. S.E.F. e ME.Q., University of Rome “Tor Vergata”, via Columbia 2, 00133, Rome, Italy

    Tommaso Proietti

Authors
  1. Alessandra Luati
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tommaso Proietti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alessandra Luati.

About this article

Cite this article

Luati, A., Proietti, T. On the equivalence of the weighted least squares and the generalised least squares estimators, with applications to kernel smoothing. Ann Inst Stat Math 63, 851–871 (2011). https://doi.org/10.1007/s10463-009-0267-8

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10463-009-0267-8

Keywords

Search

Navigation