Abstract
The ordinary least squares estimation is based on minimization of the squared distance of the response variable to its conditional mean given the predictor variable. We extend this method by including in the criterion function the distance of the squared response variable to its second conditional moment. It is shown that this “second-order” least squares estimator is asymptotically more efficient than the ordinary least squares estimator if the third moment of the random error is nonzero, and both estimators have the same asymptotic covariance matrix if the error distribution is symmetric. Simulation studies show that the variance reduction of the new estimator can be as high as 50% for sample sizes lower than 100. As a by-product, the joint asymptotic covariance matrix of the ordinary least squares estimators for the regression parameter and for the random error variance is also derived, which is only available in the literature for very special cases, e.g. that random error has a normal distribution. The results apply to both linear and nonlinear regression models, where the random error distributions are not necessarily known.
Similar content being viewed by others
References
Abarin T., Wang L. (2006). Comparison of GMM with second-order least squares estimator in nonlinear models. Far East Journal of Theoretical Statistics 20: 179–196
Amemiya T. (1973). Regression analysis when the dependent variable is truncated normal. Econometrica 41: 997–1016
Azzalini A., Capitanio A. (1999). Statistical applications of the multivariate skew normal distribution. Journal of the Royal Statistical Society B 61: 579–602
Boos D.D. (1987). Detecting skewed errors from regression residuals. Technometrics 29: 83–90
Gallant A.R. (1987). Nonlinear statistical models. New York, Wiley
Hansen L.P. (1982). Large sample properties of generalized method of moments estimators. Econometrica 50: 1029–1054
Hutson A. (2004). Utilizing the flexibility of the epsilon-skew-normal distribution for common regression problems. Journal of Applied Statistics 31: 673–683
Jennrich R.I. (1969). Asymptotic properties of non-linear least squares estimators. Annals of Mathematical Statistics 40: 633–643
Magnus J.R., Neudecker H. (1988). Matrix differential calculus with application in statistics and econometrics. New York, Wiley
Marazzi A., Yohai V.J. (2004). Adaptively truncated maximum likelihood regression with asymmetric errors. Journal of Statistical Planning and Inference 122: 271–291
Sahu S.K., Dey D.K., Branco M.D. (2003). A new class of multivariate skew distributions with application to Bayesian regression models. The Canadian Journal of Statistics 31: 129–150
Seber G.A., Wild C.J. (1989). Nonlinear regression. New York, Wiley
Theodossiou P. (1998). Fianacial data and the skewed generalized T distribution. Management Science 44: 1650–1661
Wang L. (2003). Estimation of nonlinear Berkson-type measurement error models. Statistica Sinica 13: 1201–1210
Wang L. (2004). Estimation of nonlinear models with Berkson measurement errors. Annals of Statistics 32: 2559–2579
Williams M.S. (1997). A regression technique accounting for heteroscedastic and asymmetric errors. Journal of Agricultural Biological and Environmental Statistics 2: 108–129
Wu C.-F. (1981). Asymptotic theory of nonlinear least squares estimation. Annals of Statistics 9: 501–513
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Wang, L., Leblanc, A. Second-order nonlinear least squares estimation. Ann Inst Stat Math 60, 883–900 (2008). https://doi.org/10.1007/s10463-007-0139-z
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10463-007-0139-z