Effectiveness of In-Season Manager Changes in English Premier League Football

Football is very popular worldwide. In Europe and Latin-America, football has entertained crowds for more than one century. In other continents, interest has increased in the past decades. Top players now move to the football leagues of Australia, Japan and the United States, and, more recently, also to the league of the People’s Republic of China. Both clubs and national associations employ top-class managers from all around the globe to coach their squads. Furthermore, top clubs have an enormous global fan base.

The great interest in football is not restricted to fans seeking entertainment. Professional sports, in general, and professional football, in particular, have proven to be a fruitful soil for scientific research. Kahn (2000) and Szymanski (2003), for example, argue that professional sports offer interesting data to analyze labor market phenomena. In this respect, the high frequency of data obtained from controlled events is of particular interest. Results of football matches, for example, provide a straightforward and objective measure of performance (Ter Weel 2011). An element in football that has clear analogies both with business and economics is the ongoing debate about the effects of management on the performance of firms. Kuper and Szymanski (2010) question the influence of managers on the performance of professional football teams. In contrast, Anderson and Sally (2013) argue that this influence is non-negligible, as leadership appears to matter for history, in general, and for business, in particular. Pieper et al. (2014) argue, that football managers closely resemble managers in other branches of the economy with respect to personal characteristics, such as age and the capabilities to cope with stress, media attention and a large group of stakeholders.

Assuming that ‘management matters’, or, at least, that decision-makers, such as the supervisory board, suppose that ‘management matters’, one of the key decisions is the hiring and firing of managers (Ward et al. 2011). This holds in particular, if the decision to sack a manager is implemented prior to the expiration of her/his contract. Summary dismissals tend to be rather costly, so that one aims for improved performances in return. Both Pieper et al. (2014) and Van Ours and van Tuijl (2016) show that the decision to replace a manager is related to the difference between actual performance and expectations. Both studies use bookmaker odds to derive these expectations. The authors find an increased probability of replacement, if actual performances fall short of expectations. Thus, a sequence of (rather) bad results triggers clubs to replace the manager, hoping for better performances afterwards (Bruinshoofd and ter Weel 2003).

Much research has already been done on the effects of manager turnover in business. These studies mainly use stock prices, or data derived from financial statements that are only published with a lag, viz. on a quarterly or annual basis. These outcomes point at a statistically significant but small positive effect (Ter Weel 2011). Studies on the effectiveness of managerial changes in professional football have been done for a variety of European countries, for example, Belgium, England, Germany, Italy, the Netherlands and Spain (see for a recent overview Van Ours and van Tuijl 2016).

Two Belgian studies, Balduck et al. (2010a, (2010b), find no performance effects of a coach replacement. Studying English football, Poulsen (2000) finds no effects of a managerial change while Dobson and Goddard (2011) find a negative effect, just after the replacement of a manager. Analyzing data from German football, Salomo and Teichmann (2000) find negative effects of a trainer-coach dismissal, while Hentschel et al. (2012) conclude that a coach change may have a positive effect on homogeneous teams but no effect for heterogeneous teams. De Paola and Scoppa (2011) find similar conclusions for Italian football, just like Tena and Forrest (2007) for Spanish football. Koning (2003), Bruinshoofd and ter Weel (2003), Ter Weel (2011), Van Ours and van Tuijl (2016) study the effects of the replacement of head-coaches in the highest professional football league of the Netherlands. They all find that this does not lead to better performance of the teams involved.

We study the effects of managerial changes using data of the English Premier League. We apply the method initially used by Van Ours and van Tuijl (2016). Studying Dutch professional football, they account for potential selectivity of managerial changes by, first, correcting for the strength of the opponent and, second, by defining a counterfactual case with a similar development of performances prior to the hypothetical change, but without the managerial change actually taking place. The authors use the so-called cumulative surprise as an indicator of the difference between performance and expectations. The cumulative surprise is the sum of the differences between the actual number of points and the expected number of points, as based on bookmaker odds. Then, they use this cumulative surprise to match actual coach changes to counterfactual observations. In line with most previous studies, Van Ours and van Tuijl (2016) conclude that the development of performances around the time of the change in trainer-coach is subject to regression to the mean.

Our main finding is that, on average, an in-season replacement of the manager has no effect on in-season performances. In addition to the replication of the method of Van Ours and van Tuijl (2016) for the English Premier League, we also investigate whether there is heterogeneity in the effects and find that some changes have positive effects, while other changes are counterproductive, i.e. the effects of a managerial replacement on team performance are negative. To find out whether there is a pattern in this heterogeneity of the effects of a managerial change, we also study subsamples. These subsamples are based on the origin of the manager (British vs. non-British), his age, whether or not the manager ever played for a national team, whether the team was recently promoted to the Premier League and whether the team finished top-10 or bottom-10 in the preceding season. Our main finding, i.e. managerial replacements are ineffective, stands up to the scrutiny of these subsamples. To explore potential differences between successful and unsuccessful managerial changes we present three case-studies, from which we conclude that the efficacy of managerial turnover depends on specific highly unpredictable circumstances.

Our paper is organized as follows. In Sect. 2, we present our data and our research method. Subsequently, we discuss our results in Sect. 3. Next, we present three case-studies in Sect. 4. Finally, Sect. 5 concludes.

We use data from English Premier League (EPL) football for 15 seasons, from 2000/2001 to 2014/2015. Every season contains 20 clubs that compete according to a double round-robin format, resulting in 380 matches per season (5700 matches in total). For every match, the date, the home team, the away team and the final score are recorded. Furthermore, the dataset contains match-specific bookmaker data concerning the final result, as well as the managers in charge of the two teams per match.¹ Thus, information on in-season changes is included.² In case of a managerial change, we distinguish between forced ‘sackings’ and voluntary ‘resignations’.³ Finally, the dataset contains information on the final ranks of all clubs within the EPL in the preceding season.

In our sample, ten of the managerial changes occurred, either prior to the fifth match or posterior to the 34th match. These ten changes will be left out of our analysis. Thus, 74 managerial changes remain, of which 13 do not meet the criteria for matching with a valid counterfactual case to be used in the difference-in-differences approach. Our final sample thus consists of 61 managerial changes. Table 1 presents descriptive statistics of this sample. It shows, per season and in total, the number of changes to be considered in our analysis for the complete sample, dismissals only and the subsamples that are based on managerial characteristics. We define ‘British’ managers as managers from the United Kingdom and from the Republic of Ireland, thus making a sharp distinction between these two countries and the rest of the world. For age, we distinguish between managers aged over 50 and managers aged under 50 at the time of replacement. The age of 50 is the overall average and splits the sample in two more or less equal subsamples. Table 1 further shows that managerial replacements, on average, take place around the middle of the season (column W) with the cumulative surprise then being negative (column CS). Column FS shows the average cumulative surprise at the end of the season, indicating that, for some seasons we find improvements, while for others the cumulative surprise decreases. The last three columns show the average values for our counterfactual observations. By definition, the values for the rank number of the match and for cumulative surprise are rather similar for the treatment and control group. However, the improvement in cumulative surprise is larger for the control group than for the treatment group, given the values for MFS and FS. Table 5 in the Appendix presents a detailed overview including all single managerial changes.

Open image in new window

Figure 1 shows kernel densities for the cumulative surprises at the end of the season for the subsets of dismissals, resignations, as well as for the majority of the cases, in which no managerial change has taken place. The distribution of the cumulative surprise for the dismissals is somewhat different from the distribution of the cumulative surprise for quits. Nevertheless, they look fairly similar. However, there is a clear difference between the seasons with a managerial change compared to the seasons without a managerial change. At the end of the latter seasons there is a more positive cumulative surprise. In other words, seasons with managerial changes are seasons with worse performance than seasons without a managerial change.

In analyzing the effectiveness of in-season manager replacements, we first use all 61 changes for which we have found a valid counterfactual. Then, we focus on the subset of dismissals. The parameter estimates for all managerial changes are presented in the first column of Table 2. ‘Rank Opponent’ is a measure of the strength of the opponent, while ‘Home’ represents home advantage. The variable ‘Manager change’ measures the difference in performance before and after a managerial change. Without taking a control group into account, we can interpret the coefficients of this variable as treatment effects. A significant positive value indicates that performances improved, suggesting that changes were effective. However, interpreting this result as causal would be wrong, since one does not take into account the situation in which the manager would not have been replaced. Therefore, we include a dummy variable for the control group reflecting managerial replacement that did not take place. Significant and positive values for the related parameter indicate that performances went up after the ‘counterfactual’ change, i.e. the matched observation. The F-test for equality between the two managerial-change parameter shows whether there is indeed a causal effect, i.e. if the two parameters are not significantly different from each other there is no treatment effect. Table 2 also shows the number of observations in the treatment and control groups, both separately and combined. Differences in the number of observations between the treatment and control groups arise because some club-season combinations are a control group for multiple treatment groups.

The findings for dismissals are fairly similar. These results are presented in the second column of Table 2. Leaving out the 16 resignations, thus analyzing 45 dismissals, results in comparable values, significance and conclusions. In general, thus, we may conclude that there is no point in firing a manager after a sequence of bad results, since performances would have improved irrespective of the manager in charge.⁷ Again, these results are in line with previous studies.

Table 3 shows the results for multiple subsamples which are based on the characteristics of the replaced manager.⁸ In the first and second column, we distinguish between British (\(n=48\)) and non-British (\(n=13\)) coaches.⁹ Column three and four contain the results for subsamples of coaches aged over 50 (\(n=29\)) and aged under 50 (\(n=32)\), at the time they were replaced. Finally, the last two columns, five and six, report the results for those coaches who were capped as an active player (\(n=30\)) and those who did not play for their country (\(n=31\)). Without going into detail, the general result is that we find significant improvements in performance after a managerial change, which is also the case for the counterfactual managerial change. However, we do not find any significant differences between treatment group and control group. This leads us to conclude that on average, for none of the subsamples, performances improve after a managerial change.

Interestingly, the coefficient for the control group is positive and significant, but the F-test for equality reveals that there is no significant difference between the treatment and control groups, which might have to do with the small number of observations in this subsample. The other insignificant results are found for the top half of the league table (column 5). Here, both coefficients for the treatment and control group are positive, but insignificant, strengthening the idea that, for this subset of club-season combinations, performances develop irrespective of the manager in charge. The F-test reveals no significant difference, which is also the case for all other subsamples that do contain positive and significant results.

Our results in the previous section reveal that, on average, performances improve after the replacement of a manager, but the improvement is not causally related to the change. This is in line with previous studies. Nevertheless, there is a clear heterogeneity in the effects of a managerial change when we look at individual managerial changes. Figure 2 presents a scatterplot of all 61 changes included in our sample. The horizontal axis refers to the change in cumulative surprise after the managerial replacement. The vertical axis indicates the change in cumulative surprise for the control group.¹¹ For the sake of clarity, we added a diagonal that indicates equality of equal change in cumulative surprise for the treatment group and the control group. Observations above the line represent cases in which the control group did better than the treatment group, suggesting that the change was ineffective or even counterproductive. Observations below the line represent cases in which the managerial change was effective. Furthermore, the closer the observations are to the line, the more equal the development of the two groups is. Many observations are fairly close to the diagonal, which suggests that the managerial change was ineffective, thus supporting our average result. However, a substantial number of observations are at a fairly large distance from the diagonal, suggesting that some changes are quite effective, while others are counterproductive.

Open image in new window

To investigate whether there are particular reasons for effectiveness or ineffectiveness of a managerial change, we selected three managerial replacements to discuss in more detail. First, we look at Chelsea FC, with treatment season 2011/2012 and counterfactual 2010/2011. This observation is indicated as a ‘diamond’ in Fig. 2. The close proximity of the ‘diamond’ towards the diagonal line suggests hardly any effect at all. Second, we discuss Leeds United FC, with treatment season 2003/2004 and counterfactual 2000/2001. This observation is indicated as a ‘triangle’ in Fig. 2. The position of the ‘triangle’ suggests a strongly negative effect. Third, we examine Newcastle United FC, with treatment season 2005/2006 and counterfactual 2012/2013. This observation is indicated as a ‘circle’ in Fig. 2. The position of the ‘circle’ suggests a substantially positive effect. All three cases concern the dismissal of the manager.

4.1 Chelsea FC

André Villas-Boas moved from FC Porto to Chelsea FC in the summer of 2011.¹² The Portuguese manager, only 33 years old at the time, had just guided the ‘Dragões’ (Dragons) to victory in the UEFA Europa League. Rumour has it that the London club paid a transfer fee of approximately 15 million euro. Villas-Boas soon presented a 3-year plan to take the London club to the top of Europe. Yet, Chelsea-owner Roman Abramovich had already run out of patience after little more than 8 months. The Russian club-owner held the manager responsible for the disappointing results. Thus, on 4 March, 2012, Chelsea FC sacked their Portuguese manager. Former Italian midfielder Roberto Di Matteo, previously an assistant to Villas-Boas, took over, initially only as a caretaker. At the end of the season, Chelsea were sixth in the table. However, Di Matteo guided them to their first ever victory in the UEFA Champions League (UCL). Moreover, Chelsea also won the FA Cup under his supervision.¹³

Carlo Ancelotti became the Chelsea FC manager in the summer of 2009. The former Italian midfielder had guided AC Milan to two UCL-victories (2003, 2007). In the 2009/2010 season, he led Chelsea to the double, viz. both the EPL and the FA Cup. However, Chelsea lost both prizes in the next season. Abramovich sacked Ancelotti immediately posterior to the last match of the 2010/2011 season. One month earlier, rivals Manchester United FC had eliminated Chelsea FC in the quarter finals of the UCL, a trophy then still absent in the club’s boardroom. This has probably been a crucial element underlying this post-season sacking.

Figure 2 immediately makes clear that the difference between the control season (2010/2011) and the treatment season (2011/2012) is negligible. Moreover, the decline in cumulative surprise after the (hypothetical) change in manager is about equal for both seasons (see Fig. 3a). The efforts that resulted in winning two trophies probably explain the disappointing results in the EPL in the treatment season, despite replacing the manager, who apparently was a mismatch. After all, the importance of the FA Cup may have decreased in the twentyfirst century, but the UCL is, no doubt, the biggest prize in European club football.

Open image in new window

In English premier league football managers are replaced for various reasons, but predominantly because of poor performance (Audas et al. 1999; Dobson and Goddard 2011; Bachan et al. 2008; d’Addona and Kind 2014).¹⁶ In our paper, we investigate the effectiveness of in-season manager replacements, using data of 15 seasons from English Premier League football. When we compare the change in performance after managerial replacements with the change in performance of counterfactual replacements we find no difference. Although we find heterogeneity in the effects of managerial changes, the successfulness seems to be related to specific and highly unpredictable circumstances. This raises the question why managers are dismissed anyway. There are several potential reasons for this. The first possible reason is that some club-owners are good in recognizing that a managerial replacement might be effective, while other club-owners are not. The second possible reason is misperception. As performance after a managerial change is often better than before, the perception is that this change was successful. True or not, club-owners are probably not interested in counterfactuals. A before–after comparison without considering a counterfactual is misleading from a researchers’ point of view, but not in the perception of club-owners, fans and mass-media. The third possible reason is asymmetry in the perception of the relationship between decision and result. Deciding for a replacement and not have an improvement in results is better than deciding not to act and not have an improvement in results. In the first case, club-owners have at least tried to improve the performance, in the second case they failed to act. The fourth possible reason is that dismissal is simply the destiny of a manager. The position of a manager has once been invented such that a manager gets the blame for disappointing results and not the club-owner (Carter 2006, 2007).

Thus, managers seem to be sacked due to reasons outside of their influence, functioning as scapegoats. In management literature this is found to be the case after bad performances (e.g. Khanna and Poulsen 1995) and might be an optimal strategy, together with the appointment of an outside successor, in the aftermath of wrongdoing (Gangloff et al. 2016). In sports, scapegoating of managers is found as well (e.g. Bruinshoofd and ter Weel 2003). Consequently, their jobs are highly uncertain. The saying “you’re only as good as your last match” seems to be typically true for professional football managers. Therefore, they will ask for some compensation in return for this uncertainty. However, many qualified managers are available, who are all willing to work in the EPL. This makes it rather easy for clubs to find a suitable replacement. Therefore, one might expect marginal demands from their side. Although CEO-compensation is based on multiple factors, such as ability (Chang et al. 2010), the opposite seems to be true, as salaries seem to be sky-high, probably including a scapegoat premium as found by Ward et al. (2011) for the CEO of listed companies as well as for college American football coaches. We have found that performances develop irrespective of the manager in charge, which is in line with the doubts of Kuper and Szymanski (2010) about the influence of football manager. Apparently, extremely high salaries reflect the compensation for job uncertainty rather than the compensation for superior quality.