Review of Ordinary Least Squares and Generalized Least Squares

• Thomas B. Fomby
• Stanley R. Johnson
• R. Carter Hill

Abstract

The purpose of this chapter is to review the fundamentals of ordinary least squares and generalized least squares in the context of linear regression analysis. The presentation here is somewhat condensed given our objective of focusing on more advanced topics in econometrics. The results presented, though brief in form, are important and are the foundation for much to come. In the next section we present the assumptions of the classical linear regression model. In the following section the Gauss-Markov theorem is proved and the optimality of the ordinary least squares estimator is established. In Section 2.4 we introduce the large sample concepts of convergence in probability and consistency. It is shown that convergence in quadratic mean is a sufficient condition for consistency and that the ordinary least squares estimator is consistent. In Section 2.5 the generalized least squares model is defined and the optimality of the generalized least squares estimator is established by Aitken’s theorem. In the next section we examine the properties of the ordinary least squares estimator when the appropriate model is the generalized least squares model. Finally, in Section 2.7 we summarize our discussion and briefly outline additional results and readings that are available.

Keywords

Covariance Matrix Characteristic Vector Unbiased Estimator Consistent Estimator Characteristic Root
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

1. Aitken, A. C. (1935). On least squares and linear combinations of observations. Proceedings of the Royal Society, Edinburgh, 55, 42–48.Google Scholar
2. Dhrymes, P. (1971). Distributed Lags: Problems of Estimation and Formulation. San Francisco: Holden-Day.Google Scholar
3. Dhrymes, P. J. (1974). Econometrics: Statistical Foundations and Applications. New York: Springer-Verlag.Google Scholar
4. Gauss, K. F. (1821-23), Theoria Combinationis Observationum Erroribus Minimis Obnoxiae. French translation by J. Bertrand under title Methode des moindres corres. Paris: Mallet-Bachelier (1855).Google Scholar
5. Goldberger, A. (1964). Econometric Theory. New York: Wiley.Google Scholar
6. Johnston, J. (1972). Econometric Methods. New York: McGraw-Hill.Google Scholar
7. Maeshiro, A. (1980). Small sample properties of estimators of distributed lag models. International Economic Review, 21, 721–733.
8. Markov, A. A. (1900). Wahrscheinlichkeitsrechnung. Leipzig: Tuebner.Google Scholar
9. Ramsey, J. B. (1969). Tests for specification errors in classical linear least squares regression analysis. Journal of Royal Statistical Society, B,31, 350–371.Google Scholar
10. Rao, C. R. (1973). Linear Statistical Inference and Its Applications, 2nd ed. New York: Wiley.
11. Schmidt, P. (1976). Econometrics. New York: Marcel Dekker.Google Scholar
12. Theil, H. (1971). Principles of Econometrics. New York: Wiley.Google Scholar
13. Wilks, S. S. (1962). Mathematical Statistics. New York: Wiley.Google Scholar

Authors and Affiliations

• Thomas B. Fomby
• 1
• Stanley R. Johnson
• 2
• R. Carter Hill
• 3
1. 1.Department of EconomicsSouthern Methodist UniversityDallasUSA
2. 2.The Center for Agricultural and Rural DevelopmentIowa State UniversityAmesUSA
3. 3.Department of EconomicsLouisiana State UniversityBaton RougeUSA