Regression models and linear functions

Abstract

Regression analysis proceeds from one basic assumption. Specifically, this technique assumes, at least initially, that the relationships between the variables in the analysis are “linear.“ For example, simple regression analysis assumes that one variable can be expressed, at least approximately, as a linear function of another variable. A linear function is simply one of many possible mathematical functions that one might employ to predict one variable using another variable, and it is not immediately apparent why we should favor a linear function over other functions. Perhaps the best reason for describing the relationship between two variables in terms of a linear function is its simplicity. Linear functions are less complex than most other mathematical functions, and the principle of parsimony in science suggests that, other things being equal, we should choose simple explanations over more complex ones. Linear functions are also often most appropriate because many theoretical statements in the social sciences can be readily translated into the form of a linear function. For example, stratification theory suggests that income increases directly with increases in education. This theoretical statement can easily be translated into the more formal proposition that income is a linear function of education. Of course, the most important consideration is simply how well a linear function fits the empirical data on education and income.