1 Introduction

The number of products that consumers can purchase has steadily increased in the past years (Iyengar, 2010). Several firms are growing their assortment because they know that they can capture more market share if they offer a greater variety (Kekre & Srinivasan, 1990). Moreover, consumers regard brands that provide a more extensive assortment of products as having better quality (Berger et al., 2007). Therefore, companies spend time and effort developing new products in an increasingly competitive market. If we focus on what happens inside a physical retailer, the story becomes interesting. The retailer has to decide the assortment it will offer to the consumers depending on the inventory and the agreements it has with its suppliers (Cachon et al., 2005). We also know that shelf space is limited, and the manager has to decide the combination of items that will maximize the store’s revenue. This means that products will be added or removed from the shelf depending on competition and availability. Companies have an incentive to release more new products to give a sense of high quality and attract consumers (Moreno and Terwiesch, 2017). How does the retailer select the optimal assortment if firms are becoming more competitive and want their items on the shelf? We know that at the industry level, the relationship between innovation and concentration follows an inverted-U shape (Aghion et al., 2005). However, the literature has overlooked the effect of competition on the removal of products in physical retailers. Therefore, I analyze the relationship between market concentration and product removal in a physical retailer setting.

I use scanner data from 1107 stores located in 32 cities from 2007 until 2010. Specifically, I focus on removed products from the beer aisle after the joint venture between SABMiller and Molson Coors was approved. Since the merger was unexpected and the markets are heterogeneous, I estimate the level of market concentration using an instrumental variable based on pre-merger market shares (Ashenfelter et al., 2015). Furthermore, I use Poisson regressions with the 2-Stage Residual Inclusion method (Terza et al., 2008), and I find that there is a negative relationship between market concentration and product removal. Therefore, the paper shows that in competitive markets, a larger number of items are removed. Due to the nature of the empirical setting, we confirm that retailers decide to remove products faster when competition rises. This is a natural response to the increase in product assortment that grocery stores have seen in the past three decades (Consumer Reports, 2014). Furthermore, retailers know that offering a dynamic assortment creates an incentive for consumers to return to the store (Bernstein & Martínez-de-Albéniz, 2017). However, space on the shelf is limited, and if the store decides to offer new items, older ones have to be removed. In other words, the retailer act as a gatekeeper and decides which new products enter the market and which items are removed from the assortment.

This paper contributes to the Marketing literature by studying how product removal is affected by market concentration. I hope that further research is done to understand in depth the reasoning behind these decisions. With the rise of new technologies and data analytics, I expect firms to be more active in the development of new products and thus in the removal of older items.

2 Data and methods

I use the IRI Marketing Data Set, which provides weekly sales of individual products for a set of grocery stores located across the USA (Bronnenberg et al., 2008). I focus my analysis on the beer product category, which includes national and imported beers. I decided not to include ciders, cocktails, or malted beverages because they represent less than 5% of sales in this product category. I use data from 2007 until 2010, and I drop the stores that are in states that regulate the sale of alcoholic beverages in supermarkets (Rojas, 2008). Each product is identified with a Universal Product Code (UPC), which allows me to identify the firm that manufactures the item. In the end, there are 1107 stores located in 32 Metropolitan Statistical Areas (MSA), which I also define as markets. Finally, to control for stock-outs, I collapsed the data set at a quarter/market level (Hong et al., 2016), and I obtained macroeconomic information through the Bureau of Labor Statistics to control for economic trends.

To code when a product disappeared from the store, I followed each UPC over the 4 years of data, and I marked the last date on which they were sold. If any given item was last sold between January 1, 2007, and December 31, 2009, I assume that the article was removed from the shelf. In other words, the “removed” item has not been sold in more than 12 months. In the end, I summed the number of products that were removed at the firm level in any quarter/market combinationFootnote 1resulting in a total of 31,067 observations for 471 companies that produce or import beer across 32 MSAs.

In Fig. 1, I show how many different products were removed from the second quarter of 2008 until the fourth quarter of 2009. Before the merger, a total of 174 products were removed, and right after the merger, 146 products were removed. In the right axis of Fig. 1, I plot the average national Herfindahl–Hirschman Index (HHI)Footnote 2Note that this index gives a value between zero and one, with one representing a monopoly, and zero a perfectly competitive marketFootnote 3There is a large spike in concentration after the merger was approved, with the HHI going from 0.187 to 0.231.

Fig. 1
figure 1

Number of removed products and market concentration

Full size image

This increase in concentration happened because the merger entailed the second and the third largest brewers in the USA. In the data, starting from July 1, 2008, the name of the company changes to MillerCoors, which would entail the sum of SABMiller and Molson Coors’ portfolios. Note that SABMiller had 19.17% of the market share, and Molson Coors had 10.15% of the market in the second quarter of 2008. After the merger, MillerCoors had 29.89% of the market. Therefore, the increase in the HHI due to the joint venture of these two companies would be almost equal to two times the individual market shares (Chandra & Weinberg, 2018).

Another critical point when we analyze the merger is its timing and how the industry did not expect it to go through. The 2010 Horizontal Merger Guidelines written by the Department of Justice (DOJ) state that a change of more than 0.01 in the HHI in a market that is between 0.15 and 0.25 would raise significant concerns, and it would need to be studied in detail by the regulator (DOJ, 2010). The MillerCoors proposal was initially announced in October 2007, and after 8 months of revision, it was approved by the DOJ. The main argument was that the merger would generate efficiencies in transportation, and it would not promote the coordination among firms in the industry (Heyer et al., 2009). However, in two retrospective studies, researchers have found that the merger led to increases in prices of the most popular beer brands (Ashenfelter et al., 2015; Miller & Weinberg, 2017).

We can also argue that the merger did not affect the number of removed products by MillerCoors. In the second quarter of 2008, SABMiller had a total of 2920 different UPCs, whereas Molson Coors sold a total of 1205 different items. In the third quarter of 2008, MillerCoors sold a total of 4147 specific UPCs. Therefore, the company offered a similar assortment across the USA while reducing distribution costs. If we look at the average product offering per market, MillerCoors offered 129.59 items in the third quarter of 2008. In contrast, in the second quarter of 2008, SABMiller and Molson Coors offered 91.25 and 37.66 average items per market respectively.

Finally, if we focus on product removal, SABMiller removed 72 items, and Molson Coors removed 43 items across the 32 markets in the second quarter of 2008, whereas MillerCoors decided to remove 103 items in the third quarter of 2008. Therefore, the merger did not seem to affect product assortment nor product removal as the merging firm behaved like the sum of the two individual firms before the joint venture.

To test the connection between competition and removal, I define:

$${Y}_{jmt}={\beta }_{1}{HHI}_{mt}+{\beta }_{2}{{HHI}^{2}}_{mt}+{\gamma }_{jm}+{\delta }_{t}+{\sigma X}_{mt}+{\varepsilon }_{jmt}$$
(1)

where Yjmt is the number of products manufactured by parent company j that were removed by grocery stores in market m at time t. HHImt is the Herfindahl–Hirschman Index for market m during time t; to calculate this index, I get the sum of the squared market share for all the companies that were present in market m at time t. Moreover, I include a set of fixed effects to control for any variations across firm/market (γjm), and over time (δt). Finally, the vector Xmt includes macroeconomic controls for each market/quarter combination. In particular, I incorporate wages, workforce, and unemployment rate obtained through the Bureau of Labor Statistics.

Moreover, I cluster standard errors at the firm/market level because decisions regarding product removal may be related to location (Bertrand et al., 2004). Since I use count data, I run a set of Poisson regressions (Silva & Tenreyro, 2010). However, there are two concerns regarding this approach. The first is that the coefficients of interest, β1 and β2, could be biased because we do not exactly know what is going on. This endogeneity problem comes from reverse causality; we cannot claim that as market concentration goes up, the number of products goes down. This phenomenon could happen because the firm has established itself as the market leader and it is confident that it does not need to remove products to maintain its position. The same thing happens with prices. We cannot infer a causal claim from regressing prices on concentration because we can explain the phenomenon from both sides. Prices go down because the firm has more market power and it has obtained advantages in form of economies of scales, or perhaps this firm has a better cost structure and it decided to start a price war to kick out competitors (Ashenfelter et al., 2015). Therefore, we would have to create an instrumental variable that allows us to correctly estimate market concentration without being correlated with our dependent variable.

In a similar setting, Chandra and Weinberg (2018) decided to use the merger between SABMiller and Molson Coors as an exogenous shock to the market to study how advertising and concentration were related. The merger was also not responsible for the disappearance of products because this industry is highly dynamic, and products were moving regardless of the merger. Furthermore, MillerCoors did not decide to phase out existing products because the merger was proposed to create efficiencies in distribution (Heyer et al., 2009). Therefore, I define sim∆HHIm as the simulated increase in market concentration for market m.

$$sim\Delta {HHI}_{m}=2\times {PreMillerShare}_{m}\times {PreCoorsShare}_{m}$$
(2)

where PreMillerSharem and PreCoorsSharem are the market shares that SABMiller and Molson Coors had in market m in the first semester of 2008. These shares are multiplied times two because they represent the increase in concentration that would happen if the two companies become one in the HHI equation. In other words, it is a simple algebraic formulation that comes from the sum of squares. I point the reader to Dafny et al. (2012) for a more detailed explanation of the simulated increase in market concentration and the utilization of this instrument in the insurance industry. To finish building the instrumental variable, I need to interact sim∆HHIm with a Postt dummy that is equal to one after the merger had been approved:

$${HHI}_{mt}={\omega sim\Delta HHI}_{m}\times {Post}_{t}+{a}_{m}+{\delta }_{t}+{v}_{mt}$$
(3)

This instrumental variable allows me to estimate the shock that the merger would have across different markets. Since preferences in the USA are regional, we know that the merger would have different effects over the country (Ashenfelter et al., 2015). The second concern is to find an adequate estimator after running Poisson regressions using an instrumental variable with fixed effects. To solve this, I use the 2-Stage Residual Inclusion method (Terza et al., 2008). Therefore, I have to run the first stage that I defined in Eq. (3) using an OLS regression, and I have to save the residuals from the regression, which must be included in the second stage as an additional independent variable. This will control the endogenous variable’s endogeneity, which is also a variable in the regression. Hence, after running the model with this approach, I obtain unbiased coefficients that can be used to analyze the relationship between market concentration and product removal. Moreover, for the HHI2 case, I define:

$${HHI}_{mt}={\mu }_{1}{sim\Delta HHI}_{m}\times {Post}_{t}+{{\mu }_{2}{sim\Delta HHI}^{2}}_{m}\times {Post}_{t}+{a}_{m}+{\delta }_{t}+{\sigma X}_{mt}+{\theta }_{my}$$
(4)
$${{HHI}^{2}}_{mt}={\sigma }_{1}{sim\Delta HHI}_{m}\times {Post}_{t}+{\sigma }_{2}{sim\Delta {HHI}^{2}}_{m}\times {Post}_{t}+{a}_{m}+{\delta }_{t}+{\sigma X}_{mt}{\psi }_{mt}$$
(5)

In Table 1, I present the summary statistics, where the average number of removed products is equal to 0.251, with a standard deviation of 0.822. The average HHI is located at 0.252, indicating a moderately concentrated market, with values that range from 0.112 in Portland, Oregon, until 0.450 in Saint Louis, Missouri, where Anheuser-Busch’s headquarters are located. In Appendix Fig. 3, I plot the histogram of the number of removed products at a quarter/market level. Note that in the majority of combinations, retailers do not choose to remove products from the store.

Table 1 Summary statistics
Full size table

3 Results

In this section, I present the results of the study. First of all, I point the reader to the first-stage estimates in Appendix Table 3. Note that the instrument is a good predictor of market concentration across markets. In particular, Column (1) shows that a one-point increase in the predicted market concentration would represent an increase of 1.084 in actual concentration. Columns (2) and (3) show that the instrument for the quadratic concentration is also statistically significant and a good predictor for HHI2.

The main results are presented in Table 2. In Column (1), I regress the HHI on the number of removed products, and I find a negative and significant effect. This suggests that in competitive markets, there is a larger degree of product removal because firms decide to launch a larger number of new products, and space on the shelf is limited (Arrow, 1962; Dreze et al., 1994). In Column (2), I find a non-significant inverted-U relationship between product removal and market concentration.

Table 2 The impact of market structure on product removal (Poisson and 2SRI regressions)
Full size table

The problem with these Poisson regressions is that the coefficients could be biased. That is why I developed an instrumental variable, and I used the 2SRI method to have adequate estimators (Terza et al., 2008). In Column (3), I use the simulated increase in the HHI as an instrument to predict the HHI (Ashenfelter et al., 2015; Chandra & Weinberg, 2018; Dafny et al., 2012). Hence, I confirm that the relationship between product removal and market concentration is negative and significant. The coefficient means that if the HHI goes from 0 to 1, the predicted log count of the number of removed products decreases by 3.808. In other words, in the 25th percentile market, the estimated number of removed products would be equal to 26.43. In contrast, in the 75th percentile, the number of removed products would be expected to be 23.03. To get a sense of the economic significance of these values, we know that an average product in the 32 markets that we study sells $5219 in a quarter. Therefore, if 26 products are removed, it would represent a total of $135,694 removed from the market, which is a small number given the size of the segment, but still economically significant. This confirms my previous analysis, in which I stated that a more competitive market would see higher levels of product rotation. Previous research shows that firms obtain more significant profits when they offer more extensive assortments (Moreno & Terwiesch, 2016; Tan et al., 2017). However, in a physical retailer, space on the shelf is limited, and each store has to decide which products to introduce and remove (Borin et al., 1994; Frank & Massy, 1970). This finding tells us that competition is one possible driver of product dynamics in the sense that competition drives product launches (Barriola and Martínez de Albéniz, 2021), but it also drives product removal.

In Column (4), I include HHI2, and I find a non-significant inverted-U relationship between product removal and competition. In this case, the coefficient for the linear term loses significance because the original relationship is expected to be negative, and if we had statistical significance in this case, both models would be incongruent. I also tested for the uncentered Variance Inflation Factor and found no concerns for the HHI. Moreover, in Fig. 2, I plot a bin-scatter with the two variables of interest, and it is noticeable that the relationship between product removal and concentration is negative and would adequately fit a linear model.

Fig. 2
figure 2

The relationship between product removal and market concentration

Full size image

4 Conclusions

Competition ignites the release of new products (Plambeck & Wang, 2009). However, it is essential to understand how competition is affecting the removal of old items. In this paper, I used scanner data to show that there is a negative relationship between market concentration and product removal. I developed an instrumental variable that allowed me to estimate the degree of market concentration following the merger between SABMiller and Molson Coors that was approved in June 2008 (Ashenfelter et al., 2015; Chandra & Weinberg, 2018). Since I was using count data, I had to utilize the 2-Stage Residual Inclusion with Poisson regressions to obtain adequate estimators (Terza et al., 2008).

This paper is the first that establishes a connection between competition and product removal. Previous literature has focused on the relationship between innovation and competition, with Aghion et al. (2005) finding an inverted-U shape relationship between patents and market concentration. More recently, Plambeck and Wang (2009) showed that firms decide to release more products when newcomers choose to enter. Moreover, it has been shown that organizations that offer a larger number of products can obtain higher revenues (Tan et al., 2017). In the specific case of grocery stores, the number of products offered to the consumer has been increasing significantly over the past years. For example, Consumer Reports (2014) showed that in 1975 the average number of items available was 8498, whereas in 2008, the number increased to 47,000. Moreover, Walmart decided to shorten their assortment at the beginning of 2020 but quickly noticed that the consumers were spending less (Forbes, 2020). Hence, they backtracked and now offer more than 100,000 items per store (Pymnts, 2021).

There is thus an increase in the number of products provided, but space on the shelf of a physical retailer is limited. This paper contributes to the literature by showing that competition is one of the key drivers of product dynamics inside the store. As more companies decide to enter and offer new products, the retailer has to be strategic, and it has to remove older items from the shelf. This dynamic assortment creates an incentive for consumers to go back to the store to purchase different items (Bernstein & Martínez-de-Albéniz, 2017). The question that follows is how the manufacturer responds to this phenomenon? We know that competition will make the firm move faster to offer a new set of products over the year while it tries to obtain a premium location inside the store to capture a larger consumer base (Ren et al., 2011). In the brewing industry, the latest success was the rise of craft beers, which offered a new experience and are claiming more real estate in the beer aisle, which was previously owned by a small number of players (Clemons et al., 2006). With the rise of omnichannel retailing, manufacturers can take advantage and deliver their items to consumers directly (Fisher et al., 2019). The most important aspect is to connect with the consumer and offer the item even if it is not available in a physical store. Moreover, these companies need to develop adequate marketing strategies to publicize their new releases, which is becoming increasingly easier using social media (Felix et al., 2017; Narayanan & Manchanda, 2009).

From a policy-maker point of view, we know that these large mergers created a negative effect for consumers in terms of price increases (Ashenfelter et al., 2015; Miller and Weinberg, 2018), but it also led to the entry of smaller firms (Azar & Barriola, 2021). Therefore, in this setting, a rise in concentration created a great opportunity for a set of companies that had the chance to enter and change the landscape of one product category. We have also witnessed more mergers and acquisitions in this industry, with the noticeable agreement of $100 billion from AB InBev to acquire SABMiller (Forbes, 2016). These deals have not stopped the introduction and the removal of products in an industry that continues to grow (Gatrell et al., 2018).

We expect that this could also happen in other product categories and create new opportunities for firms to enter and for consumers to enjoy new experiences. Further research is needed to understand the decision-making process behind this phenomenon. It would be interesting to study how quickly new items enter and old ones are removed from the shelf when firms start a new marketing campaign. Additionally, it would be great to understand the effect of markdown pricing on this dynamic assortment strategy.