Abstract
Experiments are becoming increasingly important in marketing research. Suppose a company has to decide which of three potential new brand logos should be used in the future. An experiment in which three groups of participants rate their liking of one of the logos would provide the necessary information to make this decision. The statistical challenge is to determine which (if any) of the three logos is liked significantly more than the others. The adequate statistical technique to assess the statistical significance of such mean differences between groups of participants is called analysis of variance (ANOVA). The present chapter provides an introduction to the key statistical principles of ANOVA and compares this method to the closely related t-test, which can alternatively be used if exactly two means need to be compared. Moreover, it provides introductions to the key variants of ANOVA that have been developed for use when participants are exposed to more than one experimental condition (repeated-measures ANOVA), when more than one dependent variable is measured (multivariate ANOVA), or when a continuous control variable is considered (analysis of covariance). This chapter is intended to provide an applied introduction to ANOVA and its variants. Therefore, it is accompanied by an exemplary dataset and self-explanatory command scripts for the statistical software packages R and SPSS, which can be found in the Web-Appendix.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The naming of the variables throughout the chapter follows the key characteristic of the respective experimental scenario. “_2” refers to the two factor levels employed in the present experiment. All variable names are constructed following the same logic.
- 2.
All barplots in this chapter were produced using the ggplot2-library in R (Wickham 2009).
- 3.
R denotes the p-value by “Pr(>F),” which refers to the probability of observing the empirical F-value given the null hypothesis. R uses exponential notation to show small numbers. Hence, the value 1.45e-05 in Fig. 3 is equivalent to 0.0000145.
- 4.
In real data collections, we would collect a second independent dataset from new participants. Please assume that although the data for the second experiment (and all further studies) are stored in the same dataset, these datasets are independent and come from different participants.
- 5.
Please note how the df of the ANOVA changed compared to Fig. 3 due to three rather than two factor levels.
- 6.
- 7.
For the example with 2 × 2 experimental cells provided in Table 4, dummy-coding Factor 1 (simple = 0; complex = 1) and Factor 2 (business = 0; leisure = 1) would mean that the effect of Factor 1 compares the cell denoted by {0,0} (i.e., “simple and business”) to the two cells for which Factor 1 has the value 1 (i.e., “complex and business” and “complex and leisure”). The cell “simple and leisure” would be omitted from the test of the main effect, which is an undesirable feature of dummy coding when applied to ANOVA models.
- 8.
It is important to note that the term “Type I” is used to denote more than just one statistical concept, which can be confusing. We already encountered the term in the context of the statistical p-value, where falsely rejecting the null hypothesis is called an alpha or Type I error. In the present context, “Type I” refers to a specific way of computing the sum of squares in an ANOVA model, which is completely unrelated to the “Type I error” in statistical hypothesis testing.
- 9.
Please note that the residual degrees of freedom (i.e., 116) for the simple effects are the same as in the initial factorial ANOVA. This is the reason why simple effects have higher statistical power than other post hoc approaches that would just compare the two means, such as an independent-samples t-test.
- 10.
The term demand artifact indicates that participants guess the hypothesis of an experiment and demonstrate behavior that is consistent with their guess instead of their natural behavior. Therefore, the occurrence of a demand artifact destroys the external validity of the observed effects. Sawyer (1975) provides an excellent discussion of this problem and potential solutions.
- 11.
A third possible approach would be an extension of the regression framework called linear mixed models (LMM; for an applied introduction, see West et al. 2015).
- 12.
For example, when the effect of funny vs. rational advertisement is examined, one usually shows several funny and several rational advertisements and compares the aggregated mean evaluations. The random variation between advertisements can be controlled by LMM.
- 13.
An excellent introduction to the use of effect size measures and a comparison of different approaches can be found in the referred article by Lakens (2013).
References
Beaujean, A.A. (2012). BaylorEdPsych: R package for Baylor University educational psychology quantitative courses. R package version 0.5.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: Lawrence Erlbaum Associates.
Cumming, G., & Finch, S. (2005). Inference by eye: Confidence intervals and how to read pictures of data. American Psychologist, 60(2), 170–180.
Field, A. (2013). Discovering statistics using R (4th ed.). Los Angeles: Sage.
Field, A., Miles, J., & Field, Z. (2012). Discovering statistics using R. Los Angeles: Sage.
Fisher, R. A. (1935). The design of experiments. Edinburgh: Oliver & Boyd.
Fox, J., & Weisberg, S. (2011). An {R} companion to applied regression (2nd ed.). Thousand Oaks: Sage.
Greenhouse, S. W., & Geisser, S. (1959). On methods in the analysis of profile data. Psychometrika, 24(2), 95–112.
Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2), 65–70.
Huynh, H., & Feldt, L. S. (1976). Estimation of the box correction for degrees of freedom from sample data in randomized block and split-plot designs. Journal of Educational Statistics, 1(1), 69–82.
Klockars, A. J., & Sax, G. (1986). Multiple comparisons. Newbury Park: Sage.
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2013.00863.
Lawrence, M.A. (2015). ez: Easy analysis and visualization of factorial experiments. R package version 4.3.
Levene, H. (1960). Robust tests for equality of variances. In I. Olkin et al. (Eds.), Contributions to probability and statistics (pp. 278–292). Stanford: University Press.
Malhotra, N. K., Peterson, M., & Kleiser, S. B. (1999). Marketing research: A state-of-the-art review and directions for the twenty-first century. Journal of the Academy of Marketing Science, 27(2), 160–183.
Miller, G. A., & Chapman, J. P. (2001). Misunderstanding analysis of covariance. Journal of Abnormal Psychology, 110(1), 40–48.
Richardson, J. T. E. (2011). Eta squared and partial eta squared as measures of effect size in educational research. Educational Research Review, 6(2), 135–147.
Rodger, R. S., & Roberts, M. (2013). Comparison of power for multiple comparison procedures. Journal of Methods and Measurements in the Social Sciences, 4(1), 20–47.
Rutherford, A. (2001). Introducing ANOVA and MANOVA: A GLM approach. London: Sage.
Sawyer, A. G. (1975). Demand artifacts in laboratory experiments in consumer research. Journal of Consumer Research, 1(4), 20–30.
West, B. T., Welch, K. B., & Galecki, A. T. (2015). Linear mixed models: A practical guide using statistical software (2nd ed.). Boca Raton: Chapman & Hall.
Westfall, J., Kenny, D. A., & Judd, C. M. (2014). Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. Journal of Experimental Psychology: General, 143(5), 2020–2045.
Wickham, H. (2009). ggplot2: Elegant graphics for data analysis. New York: Springer.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Electronic Supplementary Material
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this entry
Cite this entry
Landwehr, J.R. (2022). Analysis of Variance. In: Homburg, C., Klarmann, M., Vomberg, A. (eds) Handbook of Market Research. Springer, Cham. https://doi.org/10.1007/978-3-319-57413-4_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-57413-4_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-57411-0
Online ISBN: 978-3-319-57413-4
eBook Packages: Business and ManagementReference Module Humanities and Social SciencesReference Module Business, Economics and Social Sciences