Abstract
We first provide comprehensive, but simple, access to essential regression knowledge by discussing how regression analysis works, the requirements and assumptions on which it relies, and how you can specify a regression analysis model that allows you to make critical decisions for your business, clients, or project. Each step involved in regression analysis is linked to its execution in SPSS. We show how to use a range of SPSS’s easy-to-learn statistical procedures that underlie regression analysis, which will allow you to analyze, chart, and validate regression analysis results and to assess your analysis’s robustness. Interpretation of SPSS output can be difficult, but we make this easier by means of an annotated case study. We conclude with suggestions for further readings on the use, application, and interpretation of regression analysis.
Electronic supplementary material
The online version of this chapter (https://doi.org/10.1007/978-3-662-56707-4_7) contains additional material that is available to authorized users. You can also download the “Springer Nature More Media App” from the iOS or Android App Store to stream the videos and scan the image containing the “Play button”.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Strictly speaking, the difference between the predicted and the observed y-values is \(\hat e.\)
- 2.
This only applies to the standardized βs.
- 3.
This is only a requirement if you are interested in the regression coefficients, which is the dominant use of regression. If you are only interested in prediction, collinearity is not important.
- 4.
A related measure is the tolerance , which is 1/VIF and calculated as 1/(1−R2).
- 5.
The VIF is calculated using a completely separate regression analysis. In this regression analysis, the variable for which the VIF is calculated is regarded as a dependent variable and all other independent variables are regarded as independents. The R2 that this model provides is deducted from 1 and the reciprocal value of this sum (i.e., 1/(1−R2)) is the VIF. The VIF is therefore an indication of how much the regression model explains one independent variable. If the other variables explain much of the variance (the VIF is larger than 10), collinearity is likely a problem.
- 6.
This term can be calculated manually, but also by using the function mmult in Microsoft Excel where \({x^T}x\) is calculated. Once this matrix has been calculated, you can use the minverse function to arrive at \({({x^T}x)^{ - 1}}\).
- 7.
The test also includes the predicted values squared and to the power of three.
- 8.
This hypothesis can also be read as that a model with only an intercept is sufficient.
- 9.
The AIC is specifically calculated as \(AIC = {\text{n}}\left[ {{\text{log}}\left( {\frac{{{\text{S}}{{\text{S}}_{\text{E}}}}}{{\text{n}}}} \right) + \:\frac{{2{\text{k}}}}{{\text{n}}}} \right]\), where SSE is the error sum of squares, n is the number of observations and k the number of independent variables, while the BIC is calculated as \(BIC = {\text{n}}\left[ {{\text{log}}\left( {\frac{{{\text{S}}{{\text{S}}_{\text{E}}}}}{{\text{n}}}} \right) + \:\frac{{{\text{k}} \cdot {\text{log}}\left( {\text{n}} \right)}}{{\text{n}}}} \right]\). Note that these formulations only hold in case of normally distributed residuals with constant variance (Burnham and Anderson 2013).
- 10.
Cohen’s (1994) classical article “The Earth is Round (p < .05)” offers an interesting discussion on significance and effect sizes.
- 11.
It is possible to compare regression coefficients statistically, avoiding the need to the subjectivity of “similar.” Strictly speaking, the test for comparing coefficients is z-distributed with \({\text{z}} = \frac{{{b_1} - {b_2}}}{{\sqrt {SE_1^2 + SE_2^2} }}\) (see Paternoster et al. 1998).
- 12.
Note that this only works, as shown in the lower left of Fig. 7.7, if the “Python essentials” are installed.
- 13.
Note that it is better to calculate if the R2 increase is significant (as for Ramsey’s RESET test) but this needs to be done manually and falls outside of the scope of this book.
- 14.
Note that a p-value is never exactly zero, but has values different from zero in later decimal places.
- 15.
We would like to thank Dr. D.I. Gilliland and AgriPro for making the data and case study available.
References
Aiken, L. S., & West, S. G. (1991). Multiple regression: testing and interpreting interactions. Thousand Oaks, CA: Sage.
Baum, C. F. (2006). An introduction to modern econometrics using Stata. College Station, TX: Stata Press.
Burnham, K. P., & Anderson, D. R. (2013). Model Selection and multimodel inference: A practical information-theoretic approach (2nd ed.). New York, NJ: Springer.
Cohen, J. (1994). The earth is round (p < .05). The American Psychologist, 49(912), 997–1003.
Cook, R. D., & Weisberg, S. (1983). Diagnostics for heteroscedasticity in regression. Biometrika, 70(1), 1–10.
Durbin, J., & Watson, G. S. (1951). Testing for serial correlation in least squares regression, II. Biometrika, 38(1–2), 159–179.
Fabozzi, F. J., Focardi, S. M., Rachev, S. T., & Arshanapalli, B. G. (2014). The basics of financial econometrics: tools, concepts, and asset management applications. Hoboken, NJ: John Wiley & Sons.
Field, A. (2013). Discovering statistics using SPSS (4th ed.). London: Sage.
Green, S. B. (1991). How many subjects does it take to do a regression analysis? Multivariate Behavioral Research, 26(3), 499–510.
Greene, W. H. (2011). Econometric analysis (7th ed.). Upper Saddle River, NJ: Prentice Hall.
Hair Jr., J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (8th ed.). Boston, MA: Cengage.
Hill, C., Griffiths, W., & Lim, G. C. (2011). Principles of econometrics (4th ed.). Hoboken, NJ: John Wiley & Sons.
Kelley, K., & Maxwell, S. E. (2003). Sample size for multiple regression: Obtaining regression coefficients that are accurate, not simply significant. Psychological Methods, 8(3), 305–321.
Mason, C. H., & Perreault Jr., W. D. (1991), Collinearity, power, and interpretation of multiple regression analysis. Journal of Marketing Research, 28(3), 268–280.
Mooi, E. A., & Frambach, R. T. (2009). A stakeholder perspective on buyer–supplier conflict. Journal of Marketing Channels, 16(4), 291–307.
O’brien, R. M. (2007). A caution regarding rules of thumb for variance inflation factors. Quality & Quantity, 41(5), 673–690.
Paternoster, R., Brame, R., Mazerolle, P., & Piquero, A. (1998). Using the correct statistical test for the equality of regression coefficients. Criminology, 36(4), 859–866.
Ramsey, J. B. (1969). Test for specification errors in classical linear least-squares regression analysis. Journal of the Royal Statistical Society, Series B, 31(2), 350–371.
Treiman, D. J. (2014). Quantitative data analysis: Doing social research to test ideas. Hoboken, NJ: John Wiley & Sons.
VanVoorhis, C. R. W., & Morgan, B. L. (2007). Understanding power and rules of thumb for determining sample sizes. Tutorials in Quantitative Methods for Psychology, 3(2), 43–50.
Further Reading
Echambadi, R., & Hess, J. D. (2007). Mean-centering does not alleviate collinearity problems in moderated multiple regression models. Marketing Science, 26(3), 438–445.
Iacobucci, D. (2008). Mediation analysis: Quantitative applications in the social sciences. Thousand Oaks, CA: Sage.
Shmueli, G. (2010). To explain or to predict? Statistical Science, 25(3), 289–310.
Spiller, S. A., Fitzsimons, G. J., Lynch Jr., J. G., & McClelland, G. H. (2013). Spotlights, floodlights, and the magic number zero: Simple effects tests in moderated regression. Journal of Marketing Research, 50(2), 277–288.
Zhao, X., Lynch, J. G., & Chen, Q. (2010). Reconsidering Baron and Kenny: Myths and truths about mediation analysis. Journal of Consumer Research, 37(2), 197–206.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer-Verlag GmbH Germany, part of Springer Nature
About this chapter
Cite this chapter
Sarstedt, M., Mooi, E. (2019). Regression Analysis. In: A Concise Guide to Market Research. Springer Texts in Business and Economics. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-56707-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-662-56707-4_7
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-56706-7
Online ISBN: 978-3-662-56707-4
eBook Packages: Business and ManagementBusiness and Management (R0)