Skip to main content

Firms’ reactions to public information on business practices: The case of search advertising

Abstract

We use five years of bidding data to examine the reaction of advertisers to widely disseminated press on the lack of effectiveness of brand search advertising (queries that contain the firm’s name) found in a large experiment run by eBay (Blake et al. 2015). We estimate that 11% of firms that did not face competing ads on their brand name keywords, matching the case of eBay, discontinued the practice of brand search advertising. In contrast, firms did not react to the information pertaining to the high value and ease of running experiments—we observe no change in the experiment-like variation in advertising levels. Further, while 72% of firms had sharp changes in advertising suitable for estimating causal effects, we find no correlation between firm-level advertising effects and the propensity to advertise in the future. We discuss how a principal-agent problem within the firm would lead to these learning dynamics.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Notes

  1. In our analysis, we focus on keywords that contain only the brand’s name or brand’s website (e.g. “eBay” or “ebay.com”), not more broad brand-related searches (e.g. “eBay shoes”). Such definition restricts us to the type of branded queries for which the results of BNT have the strongest implications (cannibalization should be the highest since searchers do not specify any objective other than going to the focal brand’s website), these queries are easy to attribute to particular brands (e.g. it is unclear to which brand to attribute a query like “retailers ebay amazon”), and they correspond to the vast majority of branded search.

  2. The ads above the organic links account for the vast majority of search engine revenue.

  3. Simonov et al. (2018) estimates that competitors on average “steal” around 18 percent points of clicks when they are in the top position (first paid link), while they steal only 2-3 percent points when they are in position 2–4. Hence, removing the own-brand ad would on average lead to the loss of 15 percent points of clicks.

  4. Throughout the paper, we use the “level of advertising” term to refer to the frequency of ad appearance in the top paid positions and not to the amount of dollars spent on advertising.

  5. We note that the advertising level changes that we describe above can reflect firms’ experimentations that resulted in shutting down brand advertising, meaning that the BNT coverage has triggered some experimentation. However, we can conclude that the volume of these incremental experimentations was not high enough to stand out among the normal changes in the frequency of brand advertising.

  6. We define an “ad effectiveness” study as one that measures the causal impact of advertising, calibrated in terms of dollars, versus the cost of the media. Given that we do not observe the profitability of each customer to the focal brands, we assume that clicks on the focal brand’s weblink have similar profitability and proxy for profits with the overall volume of traffic navigating to the focal brand’s website.

  7. Reporting incentives have been previously studied theoretically. An example is the persuasion game of Shin (1994). The analog here would be a marketing manager opting to continue to report nominal CPC and CTR, not incremental traffic induced by the ad, or to selectively report experimental results.

  8. Bandiera et al. (2011) note that such experiments often have a large impact on profits when the experimental findings are implemented by the firm.

  9. Particularly relevant, the authors conclude that changing incorrect beliefs about important practices is the main driver of the effect.

  10. The pricing rule of the GSP rewards high CTR ads with low CPCs, which makes sense since the opportunity cost of the search engine is impressions, see Edelman et al. (2007) for more details. Simonov et al. (2018) show that in the absence of competitors ad, own brand advertisement increases the probability to get a click by 0.014 while a naive measure would estimate a 0.4 probability increase.

  11. A quick web search reveals practitioners’ guides that warn of the problem of click crowd-out and others that recommend advertising on own-brand keywords. On the academic side, until March 2013, there was only one paper examining the interdependence of paid and organic traffic (Yang and Ghose 2010), which used observational data to conclude, counter-intuitively, that clicks on paid links actually increased clicks on organic links (crowd-in).

  12. Titles of the media coverage are listed in Appendix A.1.

  13. In Fig. 3 and throughout the paper, we restrict our attention to keywords that contain only the brand’s name or brand’s website (e.g. “eBay” or “ebay.com”), not more broad brand-related searches (e.g. “eBay shoes”).

  14. LOESS of second degree.

  15. Appendix A.3 presents the estimates for different specifications of the time window around the information dissemination event as well as placebo tests.

  16. Figure 14 in Appendix A.4 confirms these results by presenting the non-parametric relationship between a change in the focal brand’s advertising level after the BNT disseminations and the level of competition for all firms in the sample.

  17. Based on specification (2). For robustness, we also check another measure of the level of advertisement. Figure 15 in Appendix A.5 plots the fraction of companies with own brand ad in Mainline 1 > 90% of the time for “treatment” and “control” group. Results are the same.

  18. Indeed, such behavior was widely observed in the early days of search advertising when first-price auctions were used (Edelman and Ostrovsky 2007).

  19. We regress I(Competitor in ML1 > 50% or Competitor in ML2 > 50%) on I(Focal Brand in ML1 < 10%), standard errors clustered at the focal brand level, focal brand and date fixed effects included. The design of the dependent variable ensures that the absence of the focal brand’s ad does not affect the measure of competitors’ advertising mechanically. Both 10% and 50% thresholds are chosen arbitrarily; the results are robust to the deviations from these thresholds.

  20. As before, we define this competitor’s response as

    $$\text{C. Response} = \text{Pr(C. Advertise}|\text{Focal Brand in ML1}\!<\!\!10\%) - \text{Pr(C. Advertise}|\text{Focal Brand in ML1}\!>\!10\%),$$

    where

    $$\text{C. Advertise} = \text{Pr(Competitor in ML1}>\text{50\% or Competitor in ML2}>\text{50\%}).$$
  21. We note that our estimates of competitive entry are based on correlations. To confirm that these competitive reactions are causal, we examine changes in the competition level right before and after the focal brand’s change in advertising, which are the discontinuities used in Section 4.1. We confirm that competitors start to advertise right after the focal brand stops. Such fast reaction is consistent with the algorithmic response story that we have discussed above.

  22. Recall that all results for this section are based on weekly data. If we use daily data, results would be different in this case: we find that demand changes explain around 2% of all advertising level changes in daily data. This is explained by some firms treating weekdays to weekends (lower query volume) differently.

  23. While firms might evaluate ad effectiveness in terms of the changes in profit, we proxy the ad effectiveness with the total traffic because (1) we do not observe profit changes for the studied firms and (2) incremental profits are driven by incremental traffic (e.g. based on BNT and a case of eBay incremental profits and traffic are highly correlated).

  24. We know there is heterogeneity across companies from Simonov et al. (2018), and thus part of the difference could be due to firm composition.

  25. Figure 17 in Appendix A.7 presents the histogram of number of experiments per company

  26. To conduct these tests, we normalize the distribution by mean and standard deviation, and perform a series of test: Shapiro-Wilk, Jarque-Bera, D’Agostino, Lilliefors, etc.

  27. Figure 19 in Appendix A.9 shows the relationship between estimates of advertising effect of probability to get a click and post-experimentation decision. Also, Fig. 18 corresponds to the company for which advertising effect is estimated to be negative. As we can see, the company keeps advertising in post-experimentation period.

References

  • Ashraf, N., Bandiera, O., Jack, B.K. (2014). No margin, no mission? A field experiment on incentives for public service delivery. Journal of Public Economics, 120, 1–17.

    Article  Google Scholar 

  • Bandiera, O., Barankay, I., Rasul, I. (2005). Social preferences and the response to incentives: evidence from personnel data. The Quarterly Journal of Economics, 917–962.

  • Bandiera, O., Barankay, I., Rasul, I. (2007). Incentives for managers and inequality among workers: evidence from a firm-level experiment*. The Quarterly Journal of Economics, 122(2), 729–773.

    Article  Google Scholar 

  • Bandiera, O., Barankay, I., Rasul, I. (2011). Field experiments with firms. The Journal of Economic Perspectives, 25(3), 63–82.

    Article  Google Scholar 

  • Bartelsman, E.J., & Doms, M. (2000). Understanding productivity: lessons from longitudinal microdata. Journal of Economic literature, 38(3), 569–594.

    Article  Google Scholar 

  • Blake, T., Nosko, C., Tadelis, S. (2015). Consumer heterogeneity and paid search effectivness: a large scale field experiment. Econometrica, 83(1), 155–174.

    Article  Google Scholar 

  • Bloom, N., & Van Reenen, J. (2010). Why do management practices differ across firms and countries? The Journal of Economic Perspectives, 24(1), 203–224.

    Article  Google Scholar 

  • Bloom, N., Eifert, B., Mahajan, A., McKenzie, D., Roberts, J. (2013). Does management matter? Evidence from India. The Quarterly Journal of Economics, 128(1), 1–51.

    Article  Google Scholar 

  • Chandra, A., & Staiger, D.O. (2007). Productivity spillovers in healthcare: evidence from the treatment of heart attacks. The Journal of Political Economy, 115, 103.

    Article  Google Scholar 

  • Edelman, B., & Ostrovsky, M. (2007). Strategic bidder behavior in sponsored search auctions. Decision support systems, 43(1), 192–198.

    Article  Google Scholar 

  • Edelman, B., Ostrovsky, M., Schwarz, M. (2007). Internet advertising and the generalized second-price auction: selling billions of dollars worth of keywords. The American Economic Review, 97(1), 242–259.

    Article  Google Scholar 

  • Fiedler, M.A. (2013). Essays on provider behavior in health care markets. Harvard Department of Economics.

  • Foster, L., Haltiwanger, J., Syverson, C. (2008). Reallocation, firm turnover, and efficiency: selection on productivity or profitability? American Economic Review, 98(1), 394–425.

    Article  Google Scholar 

  • Haynes, R.B., Devereaux, P.J., Guyatt, G.H. (2002). Clinical expertise in the era of evidence-based medicine and patient choice. Evidence Based Medicine, 7 (2), 36–38.

    Article  Google Scholar 

  • Lazear, E.P. (2000). Performance pay and productivity. American Economic Review, 1346–1361.

  • Lewis, R.A., & Rao, J.M. (Forthcoming). The unfavorable economics of measuring the returns to advertising. Quarterly Journal of Economics.

  • Lewis, R.A., Rao, J.M., Reiley, D.H. (2015). Chapter measuring the effects of advertising: the digital frontier. NBER Press.

  • Ostrovsky, M., & Schwarz, M. (2011). Reserve prices in internet advertising auctions: a field experiment. In Proceedings of the 12th ACM conference on electronic commerce (pp. 59–60). ACM.

  • Scharfstein, D.S., & Stein, J.C. (2000). The dark side of internal capital markets: divisional rent-seeking and inefficient investment. The Journal of Finance, 55(6), 2537–2564.

    Article  Google Scholar 

  • Shearer, B. (2004). Piece rates, fixed wages and incentives: evidence from a field experiment. The Review of Economic Studies, 71(2), 513–534.

    Article  Google Scholar 

  • Shin, H.S. (1994). News management and the value of firms. The RAND Journal of Economics, 58–71.

  • Simonov, A., Nosko, C., Rao, J.M. (2018). Competition and crowd-out for brand keywords in sponsored search. Marketing Science, 37(2), 200–215.

    Article  Google Scholar 

  • Yang, S., & Ghose, A. (2010). Analyzing the relationship between organic and sponsored search advertising: positive, negative, or zero interdependence? Marketing Science, 29(4), 602–623.

    Article  Google Scholar 

Download references

Acknowledgments

We thank Matthew Gentzkow and Matt Goldman for useful comments and suggestions. All opinions represent our own and not those of our current or past employers. All remaining errors are our own.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Andrey Simonov.

Appendix: Press coverage of Blake et al. (2015)

Appendix: Press coverage of Blake et al. (2015)

Major press includes:

  • Harvard Business Review, 3/11/2013. Did eBay Just Prove That Paid Search Ads (shown below)

  • Business Insider, 3/14/2013 EBay Slams Google Ads As A Waste Of Money

  • BBC, 3/13/2013. Google advertising value questioned by eBay.

  • The Economist, 7/13/2013. Simple tests can overstate the impact of search advertising.

  • The Atlantic, 4/13/2014. A dangerous question: Does internet advertising work at all?

A.1 Frequency of competitors’ ads in mainline 2

Fig. 11
figure 11

Histogram of Competitor’s Ad Frequency in Mainline 2

A.2 Advertising level reaction timing and placebo tests

Fig. 12
figure 12

Estimates of the “treated” firms reaction to the BNT coverage, by time window around the information event. The point estimates are the “Treatment * After” estimates from the specification (1) in Table 2 under different time windows

Fig. 13
figure 13

Placebo estimates of the “treated” firms reaction to the BNT coverage. The point estimates are the “Treatment * After” estimates from the specification (1) in Table 2 under different time windows. The information event is moved back the same number of days as in the specified event window

A.3 Advertising level reaction by level of competition

Fig. 14
figure 14

Own advertisement level for companies by the level of competition. Based on 1148 firms in the sample. Each dot represents a firm. The dotted line represents the fitted local polynomial regression of degree 2. The relationship between the change in the level of advertising and the level of competition is significant at 5% level, with the slope coefficients in a linear regression of a 0.077 (s.e. of 0.026) and b− 0.073(0.0289). Subfigure a corresponds to the results in Fig. 5, Subfigure b – to the results in Fig. 6

A.4 Probability to advertise more than 90% of the time

Fig. 15
figure 15

Own advertisement level for “treatment” and “control” companies: fraction of companies advertising more than 90% of the time. Fraction of companies with own brand ad in Mainline 1 > 90% of the time

A.5 Competitors’ entry

Fig. 16
figure 16

Competitors are more likely to advertise when the focal brand does not. The histogram is across focal brands. Low, medium and high competition groups are defined based on the histogram Fig. 11. Competitive entry is defined as competitors advertising in paid position 1 or 2 at least 50% of the time. The probability of competitors’ advertising is computed on a daily basis and averaged within a focal brand and days when the focal brand advertisers in paid position one more/less than 10% of the times

A.6 Histogram of the number of experiments per focal brand

Fig. 17
figure 17

Histogram of the number of experiments per focal brand

A.7 Example of an experimenting company

In Fig. 18 we show an unnamed firm’s advertising levels on own-brand keywords. This firm displayed a pattern of experimentation in the end of 2011 and beginning of 2012. Based on this experimentation, we find that advertising effect small and statistically insignificant. Nevertheless, this firm continued to advertise after the “experimentation” period.

Fig. 18
figure 18

Company X advertising frequency over time

A.8 Estimated effects versus post-experimentation behavior

Fig. 19
figure 19

Relationship between estimated effects for companies and their decision in the post-experimentation period. Done for companies with similar experimentation patterns as in the case of eBay

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Rao, J.M., Simonov, A. Firms’ reactions to public information on business practices: The case of search advertising. Quant Mark Econ 17, 105–134 (2019). https://doi.org/10.1007/s11129-018-9203-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11129-018-9203-7

Keywords

  • Advertising effectiveness
  • Search advertising
  • Firms’ behavior

JEL Classification

  • D83
  • L21
  • L81
  • M37