Online ads and offline sales: measuring the effect of retail advertising via a controlled experiment on Yahoo!
- 3.8k Downloads
A randomized experiment with 1.6 million customers measures positive causal effects of online advertising for a major retailer. The advertising profitably increases purchases by 5%. 93% of the increase occurs in brick-and-mortar stores; 78% of the increase derives from consumers who never click the ads. Our large sample reaches the statistical frontier for measuring economically relevant effects. We improve econometric efficiency by supplementing our experimental variation with non-experimental variation caused by consumer browsing behavior. Our experiment provides a specification check for observational difference-in-differences and cross-sectional estimators; the latter exhibits a large negative bias three times the estimated experimental effect.
KeywordsOnline advertising Display advertising Advertising effectiveness Field experiment Difference in differences
JEL ClassificationCodes: C93 - Field Experiments M37 - Advertising D12 - Consumer Economics: Empirical Analysis
We thank Meredith Gordon, Sergiy Matusevych, and especially Taylor Schreiner for their work on the experiment and data. Yahoo! Inc. provided financial and data assistance and guaranteed academic independence prior to our analysis so that the results could be published no matter how they turned out. We acknowledge the helpful comments of Manuela Angelucci, David Broockman, JP Dubé, Liran Einav, Glenn Ellison, Matt Gentzkow, Jerry Hausman, Kei Hirano, Garrett Johnson, Larry Katz, John List, Preston McAfee, Sendhil Mullainathan, Justin Rao, Paul Ruud, Michael Schwarz, Pai-Ling Yin, and many others, including attendees at many conferences and seminars.
- Aaker, D. A., & Carman, J. M. (1982). Are you overadvertising? a review of advertising-sales studies. Journal of Advertising Research, 22(4), 57–70.Google Scholar
- Abraham, M. M. (2008). The off-line impact of online ads. Harvard Business Review, 86(4), 28.Google Scholar
- Ackerberg, D. A. (2001). Empirically distinguishing informative and prestige effects of advertising. RAND Journal of Economics, 316–333.Google Scholar
- Ackoff, R. L., & Emshoff, J. R. (1975). Advertising research at anheuser-busch, inc. (1963-68). Sloan Management Review (pre-1986), 16(2), 1–1. http://search.proquest.com/docview/206793115?accountid=12861.Google Scholar
- Allaire, Y. (1975). A multivariate puzzle: A comment on advertising research at anheuser-busch, inc.(1963-68). Sloan Management Review,(Spring), 91, 94.Google Scholar
- Berndt, E. R. (1991). The Practice of Econometrics: Classic and Contemporary. MA: Addison-Wesley Reading.Google Scholar
- Blake, T., Nosko, C., Tadelis, S. (2013). Consumer heterogeneity and paid search effectiveness: A large scale field experiment. NBER Working Paper, 1–26.Google Scholar
- Eastlack, J., & Rao, A. (1989). Advertising experiments at the campbell soup company. Marketing Science, 57–71.Google Scholar
- Holahan C., & Hof R.D. (2007). So many ads, so few clicks, Bloomberg Businessweek. http://www.businessweek.com/stories/2007-11-11/so-many-ads-so-few-clicks.
- Lewis RA, & Rao JM (2013). On the near impossibility of measuring the returns to advertising. Working paper.Google Scholar
- Lewis, R. A. & Reiley D.H. (2014). Advertising effectively influences older users, How field experiments can improve measurement and targeting. Review of Industrial Organization forthcoming. http://rd.springer.com/article/10.1007/s11151-013-9403-y.
- Lewis, R. A., Rao, J. M., Reiley, D. H. (2011). Here, there, and everywhere: correlated online behaviors can lead to overestimates of the effects of advertising. Proceedings of the 20th international conference on World wide web. pp 157-166.Google Scholar
- Lewis, R. A., & Reiley, D. H. (2012). Ad attributes and attribution: Large-scale field experiments measure online customer acquisition. Working Paper.Google Scholar
- Lodish, L. M., Abraham, M., Kalmenson, S., Livelsberger, J., Lubetkin, B., Richardson, B., Stevens, M. E. (1995a). How tv advertising works: A meta-analysis of 389 real world split cable tv advertising experiments. Journal of Marketing Research, 125–139.Google Scholar
- Lodish, L. M., Abraham, M. M., Livelsberger, J., Lubetkin, B., Richardson, B., Stevens, M. E. (1995b). A summary of fifty-five in-market experimental estimates of the long-term effect of tv advertising. Marketing Science, 14(3 supplement), G133—G140.Google Scholar
- Meland, M. (1999). Banner click-throughs continue to fall. Forbes. http://www.forbes.com/1999/05/11/mu8.html.
- Shein, E. (2012). Banner ads: Past, present, and... future? CMOcom. http://www.cmo.com/content/cmo-com/home/articles/2012/4/24/banner-ads-past-present-and--future.html.