Quantitative Marketing and Economics

, Volume 16, Issue 1, pp 1–42 | Cite as

Cross channel effects of search engine advertising on brick & mortar retail sales: Meta analysis of large scale field experiments on

  • Kirthi KalyanamEmail author
  • John McAteer
  • Jonathan Marek
  • James Hodges
  • Lifeng Lin


We investigate the cross channel effects of search engine advertising on on sales in brick and mortar retail stores. Obtaining causal and actionable estimates in this context is challenging: Brick and mortar store sales vary widely on a weekly basis; offline media dominate the marketing budget; search advertising and demand are contemporaneously correlated; and estimates have to be credible to overcome agency issues between the online and offline marketing groups. We report on a meta-analysis of a population of 15 independent field experiments, in which 13 well-known U.S. multi-channel retailers spent over $4 Million in incremental search advertising. In test markets category keywords were maintained in positions 1-3 for 76 product categories with no search advertising on these keywords in the control markets. Outcomes measured include sales in the advertised categories, total store sales and Return on Ad Spending. We estimate the average effect of each outcome for this population of experiments using a Hierarchical Bayesian (HB) model. The estimates from the HB model provide causal evidence that increasing search engine advertising on broad keywords on had a positive effect on sales in brick and mortar stores for the advertised categories for this population of retailers. There also was a positive effect on total store sales. Hence the increase in sales in the advertised categories was incremental to the retailer net of any sales borrowed from non-advertised categories. The total store sales increase was a meaningful improvement compared to the baseline sales growth rates. The average Return on Ad Spend (ROAS) is positive, but does not breakeven on average although several retailers achieved or exceeded break-even based only on brick and mortar sales. We examine the robustness of our findings to alternative assumptions about the data specific to this set of experiments. Our estimates suggest online and offline are linked markets, that media planners should account for the offline effects in the planning and execution of search advertising campaigns, and that these effects should be adjusted by category and retailer. Extensive replication and a unique research protocol ensure that our results are general and credible.


Search engine advertising Cross channel impact Field experiments Bayesian meta analysis Retail marketing Advertising Retail sales Replication 

JEL Classification

M31 M37 L86 C39 C21 C11 L81 


  1. Agarwal, A., Hosanagar, K., & Location, M.S. (2008). Location, location: An analysis of profitability of position in online advertising markets. Journal of Marketing Research, 46(6), 1057–1073.Google Scholar
  2. Bajari, P., & Hortacsu, A. (2003). The winner’s curse, reserve prices, and endogenous entry: Empirical insights from ebay auctions. RAND Journal of Economics, 329–355.Google Scholar
  3. Berndt, E.R. (1991). The practice of econometrics: classic and contemporary. Addison-Wesley Reading MA.Google Scholar
  4. Berry, D.A., Berry, S.M., McKellar, J., & Pearson, T.A. (2003). Comparison of the dose-response relationships of 2 lipid-lowering agents: a bayesian meta-analysis. American Heart Journal, 145(6), 1036–1045.CrossRefGoogle Scholar
  5. Blake, T., Nosko, C., & Tadelis, S. (2013). Consumer heterogeneity and paid search effectiveness: A large scale field experiment. NBER Working Paper, 1–26.Google Scholar
  6. Brucks, M. (1985). The effects of product class knowledge on information search behavior. Journal of Consumer Research, 12(1), 1–16.Google Scholar
  7. Brynjolfsson, E., Dick, A.A., & Smith, M.D. (2010). A nearly perfect market?. QME, 8(1), 1–33.Google Scholar
  8. DerSimonian, R., & Kacker, R. (2007). Random-effects model for meta-analysis of clinical trials: an update. Contemporary clinical trials, 28(2), 105–114.CrossRefGoogle Scholar
  9. Eastlack, J.O. Jr, & Rao, A.G. (1989). Advertising experiments at the campbell soup company. Marketing Science, 8(1), 57–71.CrossRefGoogle Scholar
  10. Farley, J.U., & Lehmann, D.R. (1986). Generalizing about market response models: Meta-analysis in marketing. Lexington: Lexington Books.Google Scholar
  11. Farley, J.U., Lehmann, D.R., & Sawyer, A. (1995). Empirical marketing generalization using meta-analysis. Marketing Science, 14(3_supplement), G36–G46.CrossRefGoogle Scholar
  12. FitzGerald, D. (2013). Retail sales on thanksgiving, black friday rose 2.3 reports.
  13. Fulgoni, G.M., & Morn, M.P. (2009). Whither the click? how online advertising works. Journal of Advertising Research, 49(2), 134.CrossRefGoogle Scholar
  14. Greenland, S. (1994). Invited commentary: a critical look at some popular meta-analytic methods. American Journal of Epidemiology, 140(3), 290–296.CrossRefGoogle Scholar
  15. Hong, H., & Shum, M. (2006). Using price distributions to estimate search costs. The RAND Journal of Economics, 37(2), 257–275.CrossRefGoogle Scholar
  16. Imbens, G.W., & Rubin, D.B. (2015). Causal inference in statistics, social, and biomedical sciences. Cambridge University Press.Google Scholar
  17. Ioannidis, J.P.A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124.CrossRefGoogle Scholar
  18. Joo, M., Wilbur, K.C., Bo, C., & Yi, Z. (2013). Television advertising and online search. Management Science, 60(1), 56–73.CrossRefGoogle Scholar
  19. Levitt, S.D., & List, J.A. (2007). Viewpoint: On the generalizability of lab behaviour to the field. Canadian Journal of Economics/Revue canadienne d’é,conomique, 40(2), 347–370.CrossRefGoogle Scholar
  20. Lewis, R.A., & Reiley, D.H (2014). Online ads and offline sales: measuring the effect of retail advertising via a controlled experiment on Yahoo! Quantitative Marketing and Economics, 12.3, 235–266.
  21. Lieber, E., & Syverson, C. (2012). Online versus offline competition. Oxford Handbook of the Digital Economy, 189–223.,5&q=Lieber,+E.,+%26+Syverson,+C.+(2012).+Online+vs+offline+competition.+Oxford+Handbook+of+the+Digital+1020+Economy,+189%E2%80%93223.
  22. Lodish, L.M., Abraham, M., Kalmenson, S., Livelsberger, J., Lubetkin, B., Richardson, B., & Stevens, M.E. (1995). How tv advertising works: a meta-analysis of 389 real world split cable tv advertising experiments. Journal of Marketing Research, 125–139.Google Scholar
  23. Maniadis, Z., Tufano, F., & List, J.A. (2014). One swallow doesn’t make a summer: New evidence on anchoring effects. The American Economic Review, 104 (1), 277–290.CrossRefGoogle Scholar
  24. Naik, P.A., & Peters, K. (2009). A hierarchical marketing communications model of online and offline media synergies. Journal of Interactive Marketing, 23(4), 288–299.CrossRefGoogle Scholar
  25. Narayanan, S., & Kalyanam, K. (2015). Position effects in search advertising and their moderators: A regression discontinuity approach. Marketing Science, 34.3, 388–407.Google Scholar
  26. Ratchford, B.T., Lee, M.-S., & Talukdar, D. (2003). The impact of the internet on information search for automobiles. Journal of Marketing Research, 193–209.Google Scholar
  27. Rutz, O.J., & Bucklin, R.E. (2011). From generic to branded: a model of spillover in paid search advertising. Journal of Marketing Research, 48(1), 87–102.CrossRefGoogle Scholar
  28. Sahni, N.S (2015). Effect of temporal spacing between advertising exposures: Evidence from online field experiment. Quantitative Marketing and Economics, 13.3, 203–247.Google Scholar
  29. Sahni, N.S (2016). Advertising spillovers: evidence from online field experiments and implications for returns on advertising. Journal of Marketing Research, 53(4), 459–478.Google Scholar
  30. Sethuraman, R., Tellis, G.J., & Briesch, R.A. (2011). How well does advertising work? generalizations from meta-analysis of brand advertising elasticities. Journal of Marketing Research, 48(3), 457– 471.CrossRefGoogle Scholar
  31. Shapiro, S., MacInnis, D.J., & Heckler, S.E. (1997). The effects of incidental ad exposure on the formation of consideration sets. Journal of consumer research, 24 (1), 94–104.CrossRefGoogle Scholar
  32. Sutton, A.J., & Higgins, J. (2008). Recent developments in meta-analysis. Statistics in medicine, 27(5), 625–650.CrossRefGoogle Scholar
  33. Tellis, G.J. (1988). The price elasticity of selective demand: A meta-analysis of economic models of sales. Journal of Marketing Research (JMR) 25(4).Google Scholar
  34. Varian, H.R. (2007). Position auctions. International Journal of Industrial Organization, 25(6), 1163– 1178.CrossRefGoogle Scholar
  35. Verhoef, P.C., Neslin, S.A., & Vroomen, B. (2007). Multichannel customer management: Understanding the research-shopper phenomenon. International Journal of Research in Marketing, 24(2), 129– 148.CrossRefGoogle Scholar
  36. Yang, S., & Ghose, A. (2010). Analyzing the relationship between organic and sponsored search advertising: Positive, negative, or zero interdependence?. Marketing Science, 29(4), 602–623.CrossRefGoogle Scholar
  37. Yusuf, S., Collins, R., & Peto, R. (1984). Why do we need some large, simple randomized trials?. Statistics in medicine, 3(4), 409–420.CrossRefGoogle Scholar
  38. Zettelmeyer, F., Morton, F.S., & Silva-Risso, J. (2006). How the internet lowers prices Evidence from matched survey and automobile transaction data. Journal of Marketing Research, 43(2), 168–181.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Kirthi Kalyanam
    • 1
    Email author
  • John McAteer
    • 2
  • Jonathan Marek
    • 3
  • James Hodges
    • 4
  • Lifeng Lin
    • 4
  1. 1.Santa Clara UniversitySanta ClaraUSA
  2. 2.Google Inc.Menlo ParkUSA
  3. 3.Applied Predictive TechnologiesArlingtonUSA
  4. 4.Division of BiostatisticsUniversity of MinnesotaMinneapolisUSA

Personalised recommendations