Experiments in Market Research
The question of how a certain activity (e.g., the intensity of communication activities during the launch of a new product) influences important outcomes (e.g., sales, preferences) is one of the key questions in applied (as well as academic) research in marketing. While such questions may be answered based on observed values of activities and the respective outcomes using survey and/or archival data, it is often not possible to claim that the particular activity has actually caused the observed changes in the outcomes. To demonstrate cause-effect relationships, experiments take a different route. Instead of observing activities, experimentation involves the systematic variation of an independent variable (factor) and the observation of the outcome only. The goal of this chapter is to discuss the parameters relevant to the proper execution of experimental studies. Among others, this involves decisions regarding the number of factors to be manipulated, the measurement of the outcome variable, the environment in which to conduct the experiment, and the recruitment of participants.
KeywordsExperimental design Laboratory experiment Data collection Cause-effect relationship Manipulation Experimental units
- Aaker, D. A., Kumar, V., Day, G. S., & Leone, R. P. (2011). Marketing research. Hoboken: Wiley.Google Scholar
- Anderson, E. T., & Simester, D. (2011). A step-by-step guide to smart business experiments. Harvard Business Review, 89(3), 98–105.Google Scholar
- Arnold, V. (2008). Advances in accounting behavioral research. Bradford: Emerald Group Publishing.Google Scholar
- Baum, D., & Spann, M. (2011). Experimentelle Forschung im Marketing: Entwicklung und zukünftige Chancen. Marketing – Zeitschrift für Forschung und Praxis, 33(3), 179–191.Google Scholar
- Camerer, C. F. (2011). The promise and success of lab-field generalizability in experimental economics: A critical reply to levitt and list. Available at SSRN 1977749.Google Scholar
- Christian, B. (2012). The a/b test: Inside the technology that’s changing the rules of business. http://www.wired.com/business/2012/04/ff_abtesting. Accessed 15 Mar 2018.
- Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale: Lawrence Erlbaum Associates.Google Scholar
- Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale: Lawrence Erlbaum Associates.Google Scholar
- Cox, D. R. (1992). Planning of experiments. Hoboken: Wiley.Google Scholar
- Eriksson, L., Johansson, E., Kettaneh-Wold, N., Wikström, C., & Wold, S. (2008). Design of experiments: Principles and applications. Stockholm: Umetrics AB, Umeå Learnways AB.Google Scholar
- Evans, A. N., & Rooney, B. J. (2013). Methods in psychological research. Los Angeles: Sage.Google Scholar
- Festinger, L. A. (1957). Theory of cognitive dissonance. Stanford: Stanford University Press.Google Scholar
- Harrison, G. W., & List, J. A. (2003). What constitutes a field experiment in economics? Working paper. Columbia: Department of Economics, University of South Carolina http://faculty.haas.berkeley.edu/hoteck/PAPERS/field.pdf. Accessed 15 Mar 2018.Google Scholar
- Kalkoff, W., Youngreen, R., Nath, L., & Lovaglia, M. J. (2014). Human participants in laboratory experiments in the social sciences. In M. Webster Jr. & J. Sell (Eds.), Laboratory experiments in the social sciences (pp. 127–144). Amsterdam/Heidelberg: Elsevier.Google Scholar
- Kuipers, K. J., & Hysom, S. J. (2014). Common problems and solutions in experiments. In M. Webster Jr. & J. Sell (Eds.), Laboratory experiments in the social sciences (pp. 127–144). Amsterdam/Heidelberg: Elsevier.Google Scholar
- Larsen, R. J., & Fredrickson, B. L. (1999). Measurement issues in emotion research. In D. Kahneman, E. Diener, & N. Schwarz (Eds.), Well-being: Foundations of hedonic psychology (pp. 40–60). New York: Russell Sage.Google Scholar
- Laugwitz, B. (2001). A web-experiment on colour harmony principles applied to computer user interface design. Lengerich: Pabst Science.Google Scholar
- Maxwell, S. E., & Delaney, H. D. (2004). Designing experiments and analyzing data: A model comparison perspective. Mahwah: Lawrence Erlbaum Associates.Google Scholar
- Meyvis, T., & Van Osselaer, S. M. J. (2018). Increasing the power of your study by increasing the effect size. Journal of Consumer Research, 44(5), 1157–1173.Google Scholar
- Montgomery, D. C. (2009). Design and analysis of experiments. New York: Wiley.Google Scholar
- Nielsen, J. (2000). Why you only need to test with 5 users. https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users. Accessed 15 Mar 2018.
- Nielsen, J. (2012). How many test users in a usability study. https://www.nngroup.com/articles/how-many-test-users. Accessed 15 Mar 2018.
- Nisbett, R. E. (2015). Mindware: Tools for smart thinking. New York: Farrar, Straus and Giroux.Google Scholar
- Rashotte, L. S., Webster, M., & Whitmeyer, J. M. (2005). Pretesting experimental instructions. Sociological Methodology, 35(1), 151–175.Google Scholar
- Remler, D. K., & Van Ryzin, G. G. (2010). Research methods in practice: Strategies for description and causation. Thousand Oaks: Sage.Google Scholar
- Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.Google Scholar
- Singer, E., Van Hoewyk, J., Gebler, N., & McGonagle, K. (1999). The effect of incentives on response rates in interviewer-mediated surveys. Journal of Official Statistics, 15(2), 217–230.Google Scholar
- Stuart, E. A., & Rubin, D. B. (2007). Best practices in quasi-experimental designs: Matching methods for causal inference. In J. Osborne (Ed.), Best practices in quantitative methods (pp. 155–176). New York. Thousand Oaks, CA: Sage.Google Scholar
- Wetzel, C. G. (1977). Manipulation checks: A reply to kidd. Representative Research in Social Psychology, 8(2), 88–93.Google Scholar
- Zikmund, W., & Babin, B. (2006). Exploring marketing research. Mason: Thomson South-Western.Google Scholar