Skip to main content
SpringerLink
Log in

Field Experiments

  • Reference work entry
  • First Online:
Handbook of Market Research

Abstract

Digitalization of value chains and company processes offers new opportunities to measure and control a firm’s activities and to make a business more efficient by better understanding markets, competitors, and consumers’ behaviors. Among other methodologies, field experiments conducted in online and offline environments are rapidly changing the way companies make business decisions. Simple A/B tests as well as more complex multivariate experiments are increasingly employed by managers to inform their marketing decisions.

This chapter explains why field experiments are a reliable way to reveal and to prove that a business action results in a desired outcome and provides guidelines on how to perform such experiments step by step covering issues such as randomization, sample selection, and data analysis. Various practical issues in the design of field experiments are covered with the main focus on causal inference and internal and external validity. We conclude the chapter with a practical case study as well as a brief literature review on recent published articles employing field experiments as a data collection method, providing the reader with a list of examples to consider and to refer to when conducting and designing a field experiment.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 649.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 699.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Edward W. Deming was an eminent engineer, statistician, professor, and management consultant for more than half a century. His work on statistical process control and other strategies for data-driven decision making continues to be relavent today.

  2. 2.

    For an interesting discussion on the choice of participants for an experiment and the questionability of employing students, refer to Koschate-Fisher and Schandelmeier (2014).

References

  • Aaker, D. A., Kumar, V., Day, G. S., & Leone, R. P. (2011). Marketing research. Hoboken: Wiley.

    Google Scholar 

  • Anderson, E. T., & Simester, D. (2013). Advertising in a competitive market: The role of product standards, customer learning, and switching costs. Journal of Marketing Research, 50(4), 489–504.

    Article  Google Scholar 

  • Andrews, M., Luo, X., Fang, Z., & Ghose, A. (2015). Mobile Ad effectiveness: Hyper-contextual targeting with crowdedness. Marketing Science, 35(2), 1–17.

    Google Scholar 

  • Ascarza, E., Iyengar, R., & Schleicher, M. (2016). The perils of proactive churn prevention using plan recommendations: Evidence from a field experiment. Journal of Marketing Research, 53(1), 46–60.

    Article  Google Scholar 

  • Bawa, K., & Shoemaker, R. (2004). The effects of free sample promotions on incremental brand sales. Marketing Science, 23(3), 345–363.

    Article  Google Scholar 

  • Blake, T., Nesko, C., & Tadelis, S. (2015). Consumer heterogeneity and paid search effectiveness: A large scale field experiment. Econometrica, 83(1), 155–174.

    Article  Google Scholar 

  • Campbell, D. T. (1957). Factors relevant to the validity of experiments in social settings. Psychological Bulletin, 54(4), 297–312.

    Article  Google Scholar 

  • Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin.

    Google Scholar 

  • Chen, H., Marmorstein, H., Tsiros, M., & Rao, A. R. (2012). When more is less: The impact of base value neglect on consumer preferences for bonus packs over price discounts. Journal of Marketing, 76(4), 64–77.

    Article  Google Scholar 

  • Crook, T., Brian, F., Ron, K., & Roger, L. (2009). Seven pitfalls to avoid when running controlled experiments on the web. In Proceedings of the 15th ACM SIGKDD international conference on knowledge discovery and data mining.

    Google Scholar 

  • DubĂ©, J.-P., Luo, X., & Fang, Z. (2015). Self-signaling and pro-social behavior: A cause marketing experiment. Fox school of business research paper no. 15-079. Available at SSRN: http://ssrn.com/abstract=2635808 or https://doi.org/10.2139/ssrn.2635808

  • Dunning, T. (2012). Natural experiments in the social sciences: A design-based approach. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Eisenberg, B., & Quarto-von Tivadar, J. (2009). Always be testing: The complete guide to Google website optimizer. New York: Wiley.

    Google Scholar 

  • Fisher, R. A. (1925). Statistical methods for research workers. London: Oliver and Boyd.

    Google Scholar 

  • Gerber, A. S., & Green, D. P. (2008). Field experiments and natural experiments. In J. M. Box-Steffensmeier, H. E. Brady, & D. Collier (Eds.), The Oxford handbook of political methodology, Oxford handbooks online (pp. 357–381). Oxford: Oxford University Press.

    Google Scholar 

  • Gerber, A. S., & Green, D. P. (2012). Field experiments. Design, analysis, and interpretation. New York: Norton.

    Google Scholar 

  • Gneezy, A. (2017). Field experimentation in marketing research. Journal of Marketing Research, 46, 140–143.

    Article  Google Scholar 

  • Goos, P., & Jones, B. (2011). Optimal design of experiments: A case study approach. New York: Wiley.

    Book  Google Scholar 

  • Goldfarb, A., & Tucker, C. E. (2011a). Online display advertising: Targeting and obtrusiveness. Marketing Science, 30(3), 389–404.

    Article  Google Scholar 

  • Goldfarb, A., & Tucker, C. E. (2011b). Privacy regulation and online advertising. Management Science, 57(1), 57–71.

    Article  Google Scholar 

  • Goldfarb, A., & Tucker, C. E. (2011c). Advertising bans and the substitutability of online and offline advertising. Journal of Marketing Research, 48(2), 207–227.

    Article  Google Scholar 

  • Harrison, G. W., & List, J. A. (2004). Field experiments. Journal of Economic Literature, 42(4), 1009–1055.

    Article  Google Scholar 

  • Hoban, P. R., & Bucklin, R. E. (2015). Effects of internet display advertising in the purchase funnel: Model-based insights from a randomized field experiment. Journal of Marketing Research, 52(3), 375–393.

    Article  Google Scholar 

  • Homburg, C. (2015). Marketingmanagement. Strategie – Instrumente – Umsetzung – UnternehmensfĂĽhrung. Lehrbuch. Wiesbaden: Springer Gabler.

    Google Scholar 

  • Homburg, C., Kuester, S., & Krohmer, H. (2013). Marketing management. A contemporary perspective. London: McGraw-Hill Higher Education.

    Google Scholar 

  • Iacobucci, D., & Churchill, G. A. (2010). Marketing research. Methodological foundations. Mason: South-Western Cengage Learning.

    Google Scholar 

  • Imbens, G. W., & Rubin, D. B. (2015). Causal inference for statistics, social and biomedical sciences: An introduction. New York: Cambridge University Press.

    Book  Google Scholar 

  • Kalyanam, K., McAteer, J., Marek, J., Hodges, J., & Lin, L. (2015). Cross channel effects of search engine advertising on brick and mortar retail sales: Meta analysis of large scale field experiments on Google.com. Working paper.

    Google Scholar 

  • Kohavi, R., Longbotham, R., Sommerfield, D., & Henne, R. M. (2009). Controlled experiments on the web: Survey and practical guide. Data Mining and Knowledge Discovery, 18(1), 140–181.

    Article  Google Scholar 

  • Koschate-Fischer, N., & Schandelmeier, S. (2014). A guideline for designing experimental studies in marketing research and a critical discussion of selected problem areas. Journal of Business Economics, 84, 793–826.

    Article  Google Scholar 

  • Landsberger, H. A. (1958). Hawthorne revisited. Ithaca: Cornell University.

    Google Scholar 

  • Lambrecht, A., & Tucker, C. E. (2013). When does retargeting work? Information specificity in online advertising. Journal of Marketing Research, 50(5), 561–576.

    Article  Google Scholar 

  • Ledolter, J., & Swersey, A. J. (2007). Testing 1-2-3. Experimental design with applications in marketing and service operations. Standford: Standford University Press.

    Google Scholar 

  • Levitt, S. D., & List, J. A. (2009). Field experiments in economics: The past, the present, and the future. European Economic Review, 53(1), 1–18.

    Article  Google Scholar 

  • Lewis, R. A., & Rao, J. M. (2015). The unfavourable economics of measuring the returns to advertising. The Quarterly Journal of Economics, 1941–1973.

    Google Scholar 

  • List, J. A. (2004). The nature and extent of discrimination in the marketplace: Evidence from the field. Quarterly Journal of Economics, 119(1), 48–89.

    Article  Google Scholar 

  • List, J. A. (2011). Why economists should conduct field experiments and 14 tips for pulling one off. Journal of Economic Perspectives, 25(3), 3–16.

    Article  Google Scholar 

  • McFarland, C. (2012). Experiment! Website conversion rate optimization with A/B and multivariate. Berkeley: New Riders.

    Google Scholar 

  • Montaguti, E., Neslin, S. A., & Valentini, S. (2016). Can marketing campaigns induce multichannel buying and more profitable customers? A field experiment. Marketing Science, 35(2), 201–217.

    Article  Google Scholar 

  • Neyman, Jerzy. (1923[1990]). On the application of probability theory to agricultural experiments: Essay on principles. Section 9. Statistical Science, 5 (4), 465–472. Translated by Dabrowska, Dorota M. and Terence P. Speed.

    Google Scholar 

  • Rush, K. (2012a). Meet the Obama campaign’s $250 million fundraising platform. Blog post 27 Nov 2012.

    Google Scholar 

  • Rush, K. (2012b). Optimization at the Obama campaign: a/b testing. Blog post 12 Dec 2012.

    Google Scholar 

  • Sahni, N., Dan, Z., & Pradeep, C. (2015). Do targeted discount offers serve as advertising? Evidence from 70 field experiments. Stanford University Graduate School of Business research paper no. 15-4. Available at SSRN: http://ssrn.com/abstract=2530290 or https://doi.org/10.2139/ssrn.2530290

  • Sándor, Z., & Wedel, M. (2001). Designing conjoint choice experiments using managers’ prior beliefs. Journal of Marketing Research, 38(4), 430–444.

    Article  Google Scholar 

  • Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Belmont: Wadsworth Cengage Learning.

    Google Scholar 

  • Simonov, A., Nosko, C., & Rao J. M. (2015). Competition and crowd-out for brand keywords in sponsored search. Available at SSRN: http://ssrn.com/abstract=2668265 or https://doi.org/10.2139/ssrn.2668265

  • Siroker, D., & Pete K. (2013). A/B testing. Wiley.

    Google Scholar 

  • Teele, D. L. (2014). Field experiments and their critics: Essays on the uses and abuses of experimentation in the social sciences. New Haven & London: Yale University Press.

    Google Scholar 

  • Vaver, J., & Koehler, J. (2011). Measuring ad effectiveness using geo experiments. Google Research working paper.

    Google Scholar 

  • Yang, S., & Ghose, A. (2010). Analyzing the relationship between organic and sponsored search advertising: Positive, negative, or zero interdependence? Marketing Science, 29(4), 602–623.

    Article  Google Scholar 

  • Zantedeschi, D., McDonnell Feit, E., & Bradlow, E. T. (2016). Modeling multi-channel advertising response with consumer-level data. Management Science, Articles in Advance. http://pubsonline.informs.org/doi/abs/10.1287/mnsc.2016.2451

Download references

Author information

Authors and Affiliations

  1. University of Mannheim, Mannheim, Germany

    Veronica Valli & Florian Stahl

  2. LeBow College of Business, Drexel University, Philadelphia, PA, USA

    Elea McDonnell Feit

Authors
  1. Veronica Valli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Florian Stahl
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Elea McDonnell Feit
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Veronica Valli .

Editor information

Editors and Affiliations

  1. Department of Business-to-Business Marketing, Sales, and Pricing, University of Mannheim, Mannheim, Germany

    Christian Homburg

  2. Department of Marketing & Sales Research Group, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany

    Martin Klarmann

  3. Marketing & Sales Department, University of Mannheim, Mannheim, Germany

    Arnd Vomberg

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Valli, V., Stahl, F., Feit, E.M. (2022). Field Experiments. In: Homburg, C., Klarmann, M., Vomberg, A. (eds) Handbook of Market Research. Springer, Cham. https://doi.org/10.1007/978-3-319-57413-4_3

Download citation

Publish with us

Policies and ethics

Search

Navigation