On the Incentive Structure of Tournaments: Evidence from the National Basketball Association’s Draft Lottery

Abstract

Tournament theory analyzes labor market outcomes where rewards are distributed on the basis of relative rank. An important factor in these outcomes is the likely return to additional effort. Using National Basketball Association game event data across two seasons, we estimate each team’s game player portfolio and find that teams who were in contention to win the draft lottery reduce their portfolio’s return differential during the 2017–2018 season but not for the 2018–2019 season. We attribute the change to the reduction in the probability of obtaining a higher pick for the 2019 draft.

Introduction

Tournament theory analyzes labor market outcomes where worker rewards are distributed on the basis of relative rank rather than on absolute output. In their initial treatment of tournaments, Lazear and Rosen (1981) argue that where promotion is determined by relative performance, a worker’s optimal performance is determined by how increased effort alters the probability of winning the tournament, the differential reward for winning, and on the marginal cost of effort. In these models, for example, a decreased reward differential, negatively affects the optimal return to effort and therefore the worker’s optimal level.

While much of the theoretical, as well as empirical, work on tournament theory has been interpreted within an individual setting, Frick (2009) argues that tournament theory is applicable in a team sport setting as both are at the intersection of compensation and effort. Moreover, North American professional team sport leagues are unique in that they typically contain two distinct tournaments. The first tournament is to determine which of the league’s teams will compete in a championship round, i.e., postseason, in addition to how those teams will be seeded, i.e., ranked. For this tournament, each team would desire to perform better in order to reach a higher seed, i.e., reward, due to the fact that these seeds would have an easier path, e.g., by being able to play at home or by competing against lower performing teams, to winning. The second tournament is a tournament to determine the order that teams, who do not make the championship round, will draft next season’s first-year players. For this tournament, each team would like to perform worse in order to be able to select potential first-year players before the other teams and therefore obtain a potentially more productive player. These divergent incentives are defined by Preston and Szymanski (2003) as a ‘dual incentive problem,’ and the latter is colloquially referred to as ‘tanking.’

In what follows, we examine the impact of a professional sports’ tournament and its nonlinear reward structure, on participants’ ‘effort.’ Specifically, we examine a change in the reward structure of the National Basketball Association’s (NBA) first-year player ‘lottery-style’ draft that occurred between the 2017–2018 and 2018–2019 seasons. The NBA consists of two fifteen-team divisions. The top, in terms of total wins, eight teams from each division move to the championship rounds, i.e., playoffs, at the end of the regular season. The remaining fourteen teams are called lottery teams, and each has a nonzero chance at winning the top pick in the draft.

For much of the past 25 years, the NBA allocated the team with the worst record a twenty-five percent chance of obtaining the first selection in the draft or winning the lottery. The probability of winning the top pick in the draft then decayed nonlinearly with the second worst team having a roughly nineteen percent chance of obtaining the top pick, the third having a slightly more than fifteen percent chance, etc. The fourteenth worst team had the smallest chance of winning the pick at 0.005 percent. For the 2019 first-year draft, the NBA reduced the nonlinearity of selection process by lowering the probability of winning the top slot to fourteen percent for each of the three worst-performing teams. Overall, the change was such that all lottery teams saw a reduction in the likelihood of getting a higher pick than their final record would have received.

For the NBA, the intent of the reward structure change was to decrease the amount of perceived tanking, and for tournament theory this would be the likely outcome. The overall reward restructuring reduced the probability of winning the tournament, and agents should choose actions that make winning less likely. We modify Schmidt (2021) approach and estimate game-by-game team efficiency frontiers using play-by-play data from each game. In the end, we find that for teams who are in contention for the top prizes in the 2017–2018 ‘lottery’ tournament their deviations from their possible maximum return increase as the season progresses. Put differently, as the regular season progresses and teams reveal themselves to be in contention to win the NBA’s second tournament, those teams’ chosen player combinations deviate away significantly from those that had been chosen earlier in the season, at least relative to their respective game-defined efficient frontier. As our player portfolios are those that exist for a given game, this result is not a function of players being temporary or permanently unavailable. For example, while teams may have injuries or may trade players to other teams in order to accumulate future assets, our game efficient frontiers would adjust in these circumstances as these players would not be considered as they were not available for that particular game.¹ Interestingly, this result disappears during the 2018–2019 season, as an association between a team’s game return deviation and relative standing no longer exists.

The outline of the paper is as follows: Sect. 2 provides a brief introduction to tournament theory and presents a rudimentary model. Sections 3 and 4 provide a background on the NBA’s second tournament and our methodological approach as well as well as a brief description of the associated data, respectively. Section 5 presents the empirical results, while Sect. 6 provides a discussion of the results. The final section provides a short conclusion.

Tournament Theory

Modeling of labor market tournaments typically begins with worker productivity being determined by how much effort, or investment, a worker is willing to put forth and a random component.² Following Lazear and Rosen (1981), let $x_{it}$ represent the productivity of worker i, in period t. Also let the individual worker’s production be a function of their effort ($e_{it}$) and a random component ($\epsilon _{it}$). Each worker’s output may then be represented as follows:

$$\begin{aligned} x_{it}= e_{it} + \epsilon _{it} \end{aligned}$$

(1)

where $x_{it}$ is assumed to be stochastic but positively related to $e_{it}$. Increased levels of effort, i.e., $e_{it}$, however, come at a cost to the worker. Specifically, worker i’s costs are assumed to be captured by $C(e_i)$ and that C() be strictly convex and increasing in $e_i$ such that $C^{'}() > 0$ and $C^{''}() < 0$.

Furthermore, in terms of labor rewards, it is assumed that each of the n workers receives a reward ($W_{it}$) that diminishes in value, i.e., $W_{1t}> W_{2t},> \ldots >W_{nt}$. As mentioned, the tournament aspect revolves around the fact that the awarding of these rewards is determined by the relative rankings of each worker’s output, i.e., on $x_{it}$. The probability that worker i performs better, and therefore wins a higher prize, in period t, is assumed to depend positively on the worker’s own action, i.e., $e_{it}$, negatively on the actions of all other workers, i.e., $e_{lt}$ ($l \ne i$) and upon the distribution of the random error terms, i.e., $\epsilon _{it}$ and $\epsilon _{lt}$.

If we assume a two-worker (j & k) tournament, where both workers are constrained to the same cost behavior, i.e., $C(e_t)$, the probability worker j wins the tournament ($W_{1t}$) in period t, i.e., $p_{jt}$, is given by:

$$\begin{aligned} p_{jt} = \text{prob}[x_{jt}> x_{kt}] = \text{prob}[e_{jt}- e_{kt} > \epsilon _{kt} - \epsilon _{jt}] \end{aligned}$$

(2)

in which case the expected payoff for worker j would then be:

$$\begin{aligned} p_{jt}[W_{1t} - C(e_t)] + (1-p_{jt})[W_{2t} - C(e_t)] \end{aligned}$$

(3)

or simplifying:

$$\begin{aligned} p_{jt}(W_{1t} - W_{2t}) + W_{2} - C(e_t) \end{aligned}$$

(4)

If one ignores concerns of risk aversion and holds worker k’s actions constant, worker j will then choose $e_{jt}$ in order to maximize (4) or:

$$\begin{aligned} \frac{\partial p_{jt}}{\partial e_{jt}} (W_{1t} - W_{2t}) - C^{'} (e_{jt}) = 0 \end{aligned}$$

(5)

Several results follow from (5): (1) $e_i$, and therefore performance, depends positively with how effort alters the probability of winning; (2) $e_i$ depends positively on the prize differential ($W_1 - W_2$) but are unchanged by modifying the absolute level of prizes, at least those that leave the relative differential unchanged; and (3) $e_i$ depends negatively upon the marginal cost of effort. More generally—assuming that all players maximize (5), treating other workers effort choices as fixed, and that the resulting Nash equilibrium exists, optimal design of a tournament requires that, in equilibrium, the marginal cost of effort, $C^{'}(e_{it})$, equals the marginal value of product of effort, i.e., $V_{it}$, where $V_{it} =\frac{\partial p_{it}}{\partial e_{it}} (W_{1t}- W_{2t})$.

These models, also, suggest that where tournaments contain only a single reward—for example, those of promotion—the optimal reward size is increasing in the number of workers. Essentially the argument is that as the number of competitors rises, the marginal impact of effort on the likelihood of winning, i.e., $p_{it}$, falls. As a result, the reward for winning must be higher to elicit the same level of effort. Analogously, as the number of possible rewards falls, reward differentials should rise in order to elicit optimal effort by participants.

Empirically, Lazear (1998) finds that pay differentials appear to increase as the number of opportunities for promotion decrease.³ Similarly, Eriksson (1999) examines executive pay levels of 2,600 Danish firms and also finds that pay differences rise as one moves up in the corporate hierarchy. Finally, Main (1993) examines the pay of CEOs for over two hundred publicly traded companies and finds that the ratio of pay between job levels increases markedly as one moves up the corporate hierarchy.

While these pay gaps may exist on the basis of differential productivity, Gibbs and Hendricks (2004) find that very little of the variation in employee pay within job grade is related to performance; rather most of the variation comes from promotion. Lazear (2000), however, finds that promotion mainly revolves around workers’ relative performance rather than their absolute level. Specifically, Lazear (2000) examines worker performance data from large financial service firms and finds that an employee’s change in performance rating, i.e., the difference between the worker’s (absolute) performance rating and their job-year mean—is more important than the worker’s absolute rated performance level in determining the probability of promotion.

Fittingly, Gregory-Smith and Wright (2019) argue that workers may, therefore, be willing to accept pay levels lower than their respective marginal products if prospects of promotion exist. Symmetrically, these same workers may require increased compensation to remain if they are passed up for said promotion and therefore experience a decrease in the likelihood of further promotion. Specifically, taking into account both the likelihood of executive exit once a change in CEO has occurred and the associated compensation of surviving executives, they find that being passed over for promotion increases the probability of firm exit by up to 24 percent. In addition, they find that executives that remain with the firm are rewarded by increases in pay somewhere between 3.5 and 4 percent.

Along similar lines, Ehrenberg and Bognanno (1990a, 1990b) provide evidence of workers’ marginal value of effort, i.e., $V_{it}$, positively impacting workers’ effort levels. Specifically, these papers examine professional golf data and find that professional golfers’ final round scoring is negatively related to the marginal monetary return to increased effort. The argument is that while additional effort would, on average, manifest itself in lower scoring, the possible return to said effort depends on the golfer’s current ranking due to the competition’s nonlinear payout of monetary rewards.⁴ Ehrenberg and Bognanno (1990a), for example, find that a golfer whose marginal value of effort is one standard deviation above the mean return would score between 1.0 and 1.7 strokes lower.

The NBA’s Reverse-Order Draft

The design of the NBA’s second tournament follows logically from its stated purpose, i.e., poorer performing teams would have the opportunity to choose from the best new players in the hopes of improving their level of competition and thereby the overall level of competition within the league. However, as was mentioned earlier its design can create perverse incentives where teams compete by losing; poorly performing teams, i.e., those at the bottom of the standings, may compete by intentionally losing games toward the end of the season. Therefore, when designing these tournaments, leagues must consider the conflicting goals of deterring competitors from intentionally losing, i.e., ‘shirking,’ and promoting competitive balance (Soebbing and Mason 2009).

While the championship tournament was first introduced in North American professional team sports in 1903 with the formation of Major League Baseball and the agreement between American and National League champions to play in the World Series, the second-type tournament was first introduced in the National Football League (NFL) following the 1935 season. The NFL introduced the draft in response to a five-team bidding war that eventually led to the Brooklyn Dodgers signing Heisman Trophy winner Stan Kostka to a contract worth $5000—a level higher than most players who were currently playing the NFL (Quirk and Fort 1992). In response, Bert Bell, owner of the last place Philadelphia Eagles, proposed a reverse order draft and the League ratified the draft prior to the 1936 season.

The National Basketball Association introduced a first-year player draft in 1949.⁵ Initially, however, the draft only awarded the first pick based on team ranking. For all others, teams could forfeit their first-round pick and select a player from their immediate geographical area, so called ‘territorial picks.’ It wasn’t until the 1966 NBA draft that a largely reverse order draft was installed. The 1966 draft format, however, introduced a level of uncertainty as the first pick was determined by a coin flip between the NBA’s Eastern and Western conferences’ worst-performing teams with the loser of the coin toss receiving the second pick. All subsequent picks were awarded on the basis of reverse order of finish.

This format was largely in place until after the 1984-1985 season when the tournament design was changed due to concerns of teams intentionally reducing effort, i.e., shirking, in order to be awarded a higher pick in the upcoming draft. Specifically, the NBA modified the design for the 1986 draft to give all teams who failed to reach the championship round the same probability of being awarded the first pick, thus reducing the marginal benefit of shirking to zero. Interestingly, Taylor and Trogdon (2002) find that teams who were eliminated from the championship round were no less likely to lose games at the end of the 1984-1985 regular season than teams who had not been eliminated.

Due to several drafts where higher seeded teams were awarded the first pick and were therefore able to obtain the most highly desired players, the NBA changed the format to a weighted lottery following the 1989–1990 season. Specifically, the system of ordering the draft was changed to award picks through a lottery-style ping-pong ball process. This design awards each of the lottery teams a defined number of ping-pong balls with poorer performing teams being rewarded more of these balls and the number of balls a team received decreasing from there.

For the 1990 NBA draft, the total number of ping-pong balls was 66 with the worst-performing team receiving eleven of these and the remaining teams receiving a total that decayed linearly from there. In which case, the worst-performing team was given a roughly seventeen percent chance of winning the top pick, the second worst team was given slightly greater than fifteen percent chance, and so forth. Following this change, the aforementioned (Taylor and Trogdon 2002) find that teams eliminated from playoff contention at the end of the 1989–1990 regular season were nineteen percent less likely to win.

Following the 1992–1993 season, the NBA altered the allocation of ping-pong balls such that the team with the worst record received a twenty-five percent chance of obtaining the first selection in the draft, the second worst team having a roughly nineteen percent chance, and so forth. This design remained largely in place until the 2019 first-year draft, where the NBA reduced the nonlinearity of the selection process highlighted in Fig. 1.

A Change for the 2018–2019 Season

For the 2019 first-year draft, the NBA reduced the nonlinearity of the ‘lottery-style’ selection process. Specifically, the NBA lowered the probability of winning the top slot to fourteen percent for each of the three worst-performing teams. Figure 1 highlights the change in the odds of winning the top pick, while Fig. 2 highlights the change in the expected outcome for each of the lottery teams. Essentially, the NBA reduced the likelihood of winning the top pick for the three worst finishing teams and reallocated the likelihood to the remaining lottery teams. Furthermore, the NBA altered the likelihood of winning any of the top fourteen picks. Tables 1 and 2 highlight these changes. Table 2, for example, reports that the tournament’s overall reward structure was changed such that it reduced the likelihood of getting a top-four pick (one of the first four picks in the draft) by nearly fifty percent for the worst-performing team.⁶ Finally, Table 3 further highlights that the modification to the reward probabilities was such that all lottery teams saw a reduction in the likelihood of getting a higher pick than their end of season seeding would receive.

It is this last change in design that we are interested in. Specifically, given equation 5, the ‘lottery’ nature of the first-year player draft may induce teams to underperform if the rewards, i.e., $W_{nt}$, are increasing in value. There is, however, a significant degree of uncertainty with respect to both which team will be awarded the first pick and the uncertainty of the relative talent of that pick, in which case competing teams’ marginal value, i.e., $V_{it}$, for this second tournament is be composed of two parts: (1) the probability of getting a given pick in the draft, i.e., $\frac{\partial p_{it}}{\partial e_{it}}$, and (2) the benefit that the pick provides, i.e., $(W_{1t} - W_{2t})$. The marginal value of underperforming is then positively related to the probability of getting the next highest pick in the draft and to the marginal benefit of the next highest pick in the draft. It is precisely the change in the probability of getting the first pick between the 2017–2018 and 2018–2019 seasons that we examine here.⁷

Methodological Approach

Our scope of analysis is similar to Taylor and Trogdon (2002), Price et al. (2010), and Fornwagner (2019). Fornwagner (2019), for example, examines the National Hockey League’s (NHL) draft lottery and presents similar evidence to the earlier cited (Taylor and Trogdon 2002) findings. Specifically, over a nearly 30-year period, NHL teams, once they are eliminated from qualifying for the championship tournament, they find are more likely to lose. In addition and consistent with our findings below, eliminated teams appear to alter their player playing time in such a way as to produce lower quality output. Here, they find that higher quality players (measured by plus/minus) play significantly less (as measure by time by time on ice (TOI)) once a team is eliminated. Finally, while not their focus, Fornwagner (2019) does not find any systematic impact on the likelihood of losing from altering the reward structure.

As is recognized by Price et al. (2010), the previous approaches fail to account for teams strategically resting players in order to reduce the likelihood of winning or any injuries that may impact player performance. And while Fornwagner (2019) finds that eliminated teams reduce the minutes of more productive players, this approach would fail to capture injuries (nonstrategic) or teams distributing their players in less optimal combinations (strategic). Our approach is to examine the player combinations chosen by respective teams relative to those available to the team on game day. Moreover, we do not estimate or make assumptions on the quality of individual players, only how team expected performance would be impacted by the possible player allocations.

A Portfolio of Workers

Schmidt (2021) presents a model where player allocation decisions are akin to asset allocation decisions made by financial advisers, i.e., owners/managers put together a workforce with an eye to maximizing the firm’s potential return. They, however, must internalize any associated risk that come with these hiring/assignment decisions. Analogous to the typical portfolio analysis, any risk maybe be an outgrowth of worker’s own production variability, as well as any variability associated with the worker’s interaction with other workers.

Formally and following Schmidt (2021), let $r_{jt} = \Sigma _{i=1}^{n}r_{jit}$ represent a vector of productivity returns of each team j’s n players in game t and is thought to be iid with $r_{jt} \sim \text{N}(\mu _{jt}, \Sigma _{jt})$. Here, $\mu _{jt}$ and $\Sigma _{jt}$ capture the mean vector and covariance matrix of team j’s n players returns in game t. The team’s choice variable, i.e., the vector of asset weights, is denoted by $\xi _{jt} = \Sigma _{i=1}^{n}\xi _{jit}$, where $\xi _{jit}$ represents the proportion of playing time allocated by team j to player i in game t. Consistent with the typical usage, $\xi _{jt}$ is finite—in our case, a team has only a limited, say m, amount of game minutes to allocate. For our application, there exists a significant departure from the typically modeling in that team j cannot allocate its entire portfolio to a single player asset. The game of basketball requires five, and only five, players on the court at any point in time, in which case the largest possible weight is twenty percent. Therefore, $\xi _{jt}$ must follow $0 \le \xi _{jit} \le q$, where q captures this limit.

In order to estimate each team j’s individual game t efficiency frontiers, we simulate the possible game portfolios by randomly assigning $\xi _i$ to each team’s players following the constraints (1): $\xi ^{'}_{jit}e =1$, (2) $0 \le \xi _i \le q$ and (3) the $\xi _i$s are assumed to be distributed uniformly. In addition, each team’s efficient game frontier was calculated by conditioning on a given rate of return and then simulating 100,000 possible portfolios with the randomly assigned $\xi _i$—constrained as before—and assigning the portfolio with lowest risk to the frontier.⁸

Specifically, the frontier was calculated as follows:

$$\begin{aligned} \begin{array}{rrclcl} \min _{\xi } &{} {\frac{\text{k}}{2} \xi ^{'}\Sigma \xi } &{}&{}\\ \text{s.t.} &{} r^p _L \le &{} r^p&{} \le r^p_U \\ &{}\xi ^{'}e &{} = &{} 1 \\ &{} 0 \le &{} \xi _i &{} \le q &{} &{} i = 1, \ldots , n \\ \end{array} \end{aligned}$$

(6)

As is discussed in depth in the next section, we use the plus/minus performance measure to calculate the components of $r_{jt} = \Sigma _{i=1}^{n}r_{jit}$. Also, the actual percentage of total game minutes that the team allocated to each player is used to calculate each team’s actual game values of $\xi _i$. Finally, the vertical distance between these two calculations, i.e., the degree to which each team deviates from their respective unique game frontier, is our return differential (rd) measure for each team’s games across the season.

Data

Our data come from two seasons, i.e., 2017–2018 & 2018–2019, of NBA game.⁹ The total number of participants, i.e., teams, is thirty for both seasons with each team playing a total of 82 regular season games. Each NBA game typically lasts a total of 48 minutes.^10,11 In addition, each team is only allowed to have a total of twelve players available for each of these 82 games.¹² However, only five of the players on the roster are allowed to participate in a game—be on the floor—at any one time. This means that while, for a given game, a team can typically allocate 240 min ($m=240$) across its potential players, it can only allocate up to twenty percent of their game total to any one of their players. In our portfolio analogy, this means that, at most, an NBA team can only allocate twenty percent of their portfolio, i.e., $q=0.20$, to any of their potential player assets. One should note that while this is the maximum possible, the game of basketball is such that this would be an extreme outcome and not one that appears in the data. Bradley Beal of the Washington Wizards, for example, leads the NBA in minutes played for the 2018–2019 season with a total of 3028 minutes or roughly 15.4% of the total team minutes. That number was only two minutes above the previous year’s leader, LeBron James. Finally, one should further note that the results that follow would be nearly identical if one imposes the constraint $q=0.15$.

Finally, our goal is to use the play-by-play data to calculate each players combination’s expected game return and its associated game risk. In order to calculate their expected return and risk, we desire a variable that captures the relevant measure of asset (team) productivity, i.e., winning. In the end, placement in the two tournaments is a function of the total number of wins a team earns. Winning, in the end, revolves around scoring more points than the opposing team. We therefore incorporate the plus/minus measure for our expected return measure. The plus/minus measure tracks how the potential outcome of the game changes while the player was on the court. For example, if during the play the player was on the court, the score increased to the player’s team’s favor than the measurement would record a ‘$+$’ or a positive number. In contrast, if the score decreased relative to the team’s score line than this would record a ‘−’ or a negative. To complete the circle, if the score remains unchanged, the value would be zero.

The use of plus/minus as a measure of player productivity in the NBA has come under considerable debate, e.g., Berri (2012). These concerns generally revolve around the interpretation of the statistic as a measure of individual quality, i.e., is player A better than player B. Here, while we are concerned about the individual quality of the player, we are only concerned in how the individual asset quality impacts the group. Specifically, we are looking for a measure of what happens to the return of our portfolio, i.e., winning/losing, when a particular asset allocation is assumed.

To this point, if a team (portfolio) performs better when player A is joined with player B, then once we increase the weight of our allocation within the portfolio of player A and player B, we should see a higher return, i.e, increased plus/minus. This should be true regardless of how the player impacts the game, i.e., rebounds, assists, or even through their sheer personality. Moreover, we are not attempting to value player A or player B, only how the portfolio performance moves when we adjust their weights. To use a finance analogy, we are not specifically interested in the expected return of an individual asset but rather in how the performance of said asset impacts the performance of the portfolio overall.

Table 4 reports some descriptive statistics about our data. Our game-level event data contain over half a million observations for each of the two seasons. While there are minor differences in the totality of the data between the two data sets, the two seasons’ data are, in general, quite close. For example, in each of the seasons, roughly seventy-five percent of the observations contain plays where no team scored any points and the average number of points scored per play is roughly 0.470. Furthermore, both seasons return a similar number of unique player combinations as well as a similar amount of unique per game player combinations.

Empirical Results

Figure 3 provides a representative mapping of the estimates provided from (6) for a representative NBA team—the Chicago Bulls. The figure reports the mapping for an early season game–game number eight—and a late season game–game number 68—for both the 2017–2018 and 2018–2019 seasons. For the 2017–2018 season, game number 68 was played on March 15, 2018 and, at that point, the Bulls had the sixth worst record; for the 2018–2019 season, game number 68 was played on March 10th, 2019 and, at that point, the Bulls had the fourth worst record in the NBA. Game 8 was similarly played five days later in 2018–2019 as compared to 2017–2018. The dots, which populate the inner part of the frontier, represent the mapping of the expected return and risk of the possible portfolios the team could have chosen. The largish black X represents the team’s actual allocation.

A feature of Fig. 3 is that the Bulls failed to reach their respective game-defined efficient frontier in each of these games—although they were relatively close in game 68 of the 2018–2019 season. In fact, examination of all teams and all games finds that no team ever reached any of their game-defined efficient frontiers. It is the case that all teams, in all games, could have increased their returns, holding their risk constant, or could have obtained the same return with lower associated risk. However, as our analysis is ex post, this result is hardly surprising as there is no expectation that teams reach their game-defined efficient frontier—our analysis uses players’ actual returns, while teams would use their expected returns.

Given our earlier discussion, Fig. 3 also provides anecdotal evidence of the impact of the change in draft lottery structure. The Chicago Bulls would play both the 2017–2018 and 2018–2019 seasons largely inside the bottom six in terms of team record; for the 2017–2018 season, the Bulls would finish with the NBA’s sixth worst record; for the 2018–2019, they would finish with the NBA’s fourth worst record. As mentioned earlier, our interest is in the degree to which an NBA team deviates from its maximum return as a function of their respective draft position. Visual examination of the return differential—the vertical distance between the team’s X and its highest return portfolio—in Fig. 3 suggests that the Bulls’ departure was larger for game 68 of the 2017–2018 season as compared to the same game in 2018–2019.

As the data behind the mappings in Fig. 3 come from only one NBA team and from only a set of two games from these two seasons, they only provide limited evidence of any possible change. We, therefore, next turn to examine whether this behavior is consistent across teams and across games. Specifically we estimate the impact that a team’s draft position has on their chosen portfolio of players, at least relative to their game-defined efficient frontier, i.e., a team’s return differential, for all games and across both seasons. Specifically, we estimate the following regression:

$$\begin{aligned} rd_{ijt} = \beta _0 + \beta _1*dp_{ijt} + \epsilon _{ijt} \end{aligned}$$

(7)

where $rd_{ijt}$ represents the return differential and $dp_{ijt}$ represents the draft position for game i of team j in year t. Here, $i = 1, \ldots , 82$, $j = 1, \ldots , 30$, and $t =$ 2017–2018, 2018–2019. The top row of Table 5 reports the estimated impact of draft position, i.e., $\beta _1$, on return differential for all the games played during the 2017–2018 and 2018–2019 seasons, respectively. The $\beta _1$s were estimated using panel generalized least squares incorporating random effects. While our panel designations are team j for cross section and the game number i for the time variable, we incorporate random effects due to possible differences, e.g., fandom, coaching, across teams that may have some influence on the return differential that are not a function of draft position.¹³

Table 5 highlights that in general draft position does not appear to impact a team’s return differential. For example, the top panel of the table finds that when we include our entire sample of observations, i.e., all teams and their relative draft position across the entire season, the coefficient on draft position is insignificant, a finding consistent across both seasons. Moreover, the Wald $\chi ^2$ test fails to reject the $H_0$ that all the coefficients are zero. This result, however, is not very surprising as this sample joins two competing tournaments with some teams trying to reduce their respective distance (or at the very least maintaining it) in order to increase performance and improve the likelihood of winning a ticket to the championship round. At the same time, other teams may be attempting to reduce performance in order to gain a potentially higher draft pick.

We therefore estimated subsets of the data that separated teams by draft position based upon the likelihood of the team winning entry into each of the two tournaments. For example, the middle panel of Table 5 separated the data into two samples, one for teams currently qualified for the championship tournament, i.e., the top sixteen teams in terms of winning record, and those who were currently qualified for the NBA’s other tournament, i.e., the bottom fourteen teams in terms of winning record. One should note that in this context, currently denotes that at the time the game was played the team was currently in position to qualify for the given tournament, in which case we have teams that are marginally in both groups depending on when and which games are played. The Washington Wizards, for example, were seeded ninth in the lottery tournament after playing their 52 game, but finished in sixth place by the end of the season in 2018–2019.

The remaining panels estimate further subsets of the data. In the end the only group that responded to draft position were the teams in the bottom six and only for the 2017–2018 season.¹⁴ This result suggests that teams ranked in the bottom six of winning percentage increased their return differential for the 2017–2018 season—a result consistent with these teams shirking. This result, however, does not appear in the 2018–2019 season estimation.

While the estimates provided in Table 5 suggest that teams in the bottom six in terms of winning percentage tended to choose player combinations that were further from their respective game-defined efficient frontiers during the 2017–2018 season, one might suspect that including game observations from early in the season would mute any possible impact of draft position as teams may need time to assess their relative quality across the two tournaments. We therefore estimated (7) on a rolling basis using a 35-game window. Specifically, we estimated the equation incorporating only the first 35 games of the season and then for games 2 through 36, etc. These results are highlighted in Fig. 4 which provides the rolling estimates of $\beta _1$ where teams currently reside in either a top twenty-four position or a bottom six position. Dashed lines provide the associated 95 percent confidence intervals.

Overall, Fig. 4 continues to suggest that teams who were in the bottom six in terms of winning percentage incorporate player portfolios that were significantly worse, at least relative to those they had chosen earlier and those they could have chosen. Specifically, Fig. 4c finds, for the NBA’s 2017–2018 season, that once the NBA ventured past its halfway point, the bottom six teams in terms of record increased their return differential. This pattern is consistent with the assumption that teams optimally desire to compete in the championship tournament and are likely to put forth their best effort to determine whether they have a chance. However, once its revealed that they are not likely to be invited to this tournament, they turn their attention to winning the other tournament.

Moreover, NBA rules force teams to choose the tournament they intend to compete in by a certain point of the season. The NBA’s trading deadline, highlighted by the dashed vertical line in Fig. 4, is particularly important. Prior to this deadline, NBA teams are allowed to exchange players, draft picks, as well as cash. Typically, those teams that are competing for a seed or a better seed in the championship tournament will desire to obtain players who may increase their relative performance. In contrast, those teams that are in competition for a lower seeding in the draft tournament may desire to sell off players to increase their relative performance in this tournament. That date for both years fell on February 8th. On this date in both years, NBA teams, on average, would have completed two-thirds of the season games.¹⁵

Discussion

While the results above suggest that the change in the structure of the lottery which altered $\frac{\partial p_{it}}{\partial e_{it}}$ of the (5) for competing lottery tournament teams caused teams to reduce their level of shirking, this is not the only dimension of concern. As in Ehrenberg and Bognanno (1990a, 1990b) there exists a second dimension that may affect $\frac{\partial p_{it}}{\partial e_{it}}$, as it also depends on the relative distribution of lottery competing teams’ records. Given that competing in the tournament is costly, one would suspect that as the differential in total wins between competing teams widens, teams would be less likely to compete. Figure 5, therefore, reports the change in the coefficient of variation in team winning percentage for the bottom three and six teams as well as for all of the remaining teams on a rolling basis for the latter half of the NBA regular season.¹⁶¹⁷

The figure suggests that the two seasons varied in how the two tournament teams clustered. For the top twenty-four teams, the 2018–2019 season was slightly more condensed as compared to the 2017–2018 season. For the bottom teams, both the bottom six and bottom three, the behavior is reversed as the dispersion between these teams was on the order of ten to twenty percent more in 2018–2019. There exists a degree of endogeneity, however, as teams that are attempting to compete would likely all lose to teams competing in the championship tournament and their grouping, therefore, would likely be more condensed than if teams were not competing to win the draft lottery.

The Value of Winning

An additional area of concern in (5) is whether there are any possible changes in both the absolute and relative talent levels of the respective potential first-year players, i.e., ($W_{jt}-W_{kt}$), across the 2 years. Potential first-year draftees may differ in talent both in cross section and in time. In order to assess the relative talent level, we need to predict, on the basis of the information that teams have, the potential productivity of these future NBA players. At the time of the draft teams are limited in information on each potential player’s ability. They do, however, have information on players’ amateur, i.e., college, careers. We, therefore, next turn to incorporating potential first-year players’ college statistics to estimate their potential NBA productivity through machine learning regressions.

Rather than incorporating the usual regression approach to estimate the potential of first-year players, we incorporate a machine learning approach. As is described in Chalfin et al. (2016), in situations where workers vary in their potential productivity, using a machine learning approach rather than its alternatives may potentially improve outcomes. Moreover, while the goal of machine learning algorithms is similar to its alternative, i.e., to incorporate information from a set of independent variables, i.e., $x_{it}$, to produce predictions of a variable of interest, i.e., $y_{it}$, its focus is not on the individual estimated coefficients in order to highlight causal relationships (Mullainathan and Spiess 2017). In fact, if the question at hand requires good estimates of parameters so that one may understand the underlying relationship between the independent and dependent variables, machine learning is not an appropriate set of tools. Rather the tools of machine learning are specifically designed for prediction (Chalfin et al. 2016).

The question at hand is similar to recent studies by Jacob et al. (2018); Sajjadiani et al. (2019) and Kleinberg et al. (2018) who use machine learning to examine the outcomes of hiring decisions, of teachers and police, respectively, based upon a set of applicant characteristics. Sajjadiani et al. (2019), for example, highlight how machine learning can be used as an alternative way to screen work history. They examine the work history, as represented on resume and job market applications, of over 16,000 job applicants for public teaching positions. They use machine learning to predict subsequent work outcomes e.g., student evaluations, voluntary turnover, etc. In the end, they find that work experience relevance as well as a history of approaching better jobs is associated with more positive hiring outcomes. In contrast, a history of avoiding bad jobs was associated with more negative outcomes. These findings, they argue, suggest that a move toward machine learning may allow companies to improve their selection decisions while reducing their risk.

In our case, we are interested in predicting the NBA value of a potential first-year player,, i.e., $y_{it}$, from their observable college statistics, i.e., $x_{it}$, based upon a sample of n previous first-year draftees’ $(y_{it}, x_{it})$ for each of the two seasons. Generically, the machine learning approach incorporates a loss function, i.e., $L(\hat{y}_{it}, y_t)$, as an input and searches for a function $\hat{f}(x_{it})$ that is taken from a subsample of the data, i.e., called train data—that has a low expected prediction loss $E_{(y,x)}L(\hat{f}(x), y)$ when applied to another subsample of the data—called test data. In the end, the goal of machine learning is to find a $\hat{f}(x_{it})$ such that it has low bias and low variance. This in turn should return improved prediction performance.

While alternatives exist, we incorporated the random forest machine learning algorithm.¹⁸¹⁹ One of the advantages of the random forest approach, which extends the decision tree method of Breiman (1996), is that the selection of the relevant $x_{it}$s is chosen without any a priori assumptions on either their functional or distributional relationship to the dependent variable.²⁰ In general, the random forest approach is constructed by forming n decision trees which use some subset of $(x_{it}, y_{it})$. The n decision trees are independent of one another as each represents a bootstrapped subsample of the sample data taken with replacement and thereby reduces the associated variance without an associated increase in bias.²¹ At the end of the process, an average of the n $\hat{f}(x_{it})$s is calculated and used to estimate each potential first-year draftee’s NBA productivity for both the 2018 and 2019 drafts. Finally, in order to keep in line with the information available to NBA teams’ evaluators we estimated two separate random forests. The first used NBA players drafted between 2000 and 2017 to produce estimates for the 2018 draft first year. The second updates the sample to include the production of the 2018 draft class to estimate the NBA potential for the 2019 draft.

Table 6 reports summary statistics for data that underlie the potential draftee forecast ($y_{it}$). Specifically, the first and third columns of Table 6 summarize the mean college statistics ($x_{it}$) for all basketball players drafted between 2000 and 2017 who attended an US college.²² The second and fourth columns update the summary data to include the thirty-five players drafted in 2018.²³ These data are then compared to the NBA rookie data from each player drafted and used to predicted future values for potential 2018 and 2019 draftees.²⁴ For example, there exists a high degree of correlation (over 0.50) between a draftee’s three point efficiency in college and the draftee’s average point total in their rookie year. Similar correlation exists for overall field goal percentage and free throws attempts.

The data reported in Table 6 were used to model five basketball performance measures—expected rebounds, expected assists, expected steals, expected blocks, and expected points. In order to do so, we estimated five different machine leaning models to associate the strongest correlation between the underlying data, i.e., $x_{it}$, to the relevant NBA statistic—$y_{it}$, e.g., rebounding. We then used the estimated model, for each performance measure, to predict what one may expect for each eligible draftee, given their college performance, for both 2018 and 2019. These are reported in columns (2)–(6) of Table 7. One may note that across the two drafts, Zion Williamson was expected to have the highest return overall—largely driven by his expected rebounding.

Finally, we then combined these five expected values to calculate a total value measure, i.e., the NBA’s official fantasy points scoring format, to assess the player’s overall potential value. Specifically, we summed the expected values as follows:

$$\begin{aligned} x^{value}_{it} = 1.2 *reb^{e}_{it} + 1.5 * ast^{e}_{it} + 3 * stl^{e}_{it} + 3 * blk^{e}_{it} + pts^{e}_{it} \end{aligned}$$

(8)

where $x^{value}_{it}$ is the total value measure, $reb^{e}_{it}$ the total expected rebounds, $ast^{e}_{it}$ the total expected assists, $stl^{e}_{it}$, the total expected steals, $blk^{e}_{it}$ the total expected blocks, and $pts^{e}_{it}$ the total expected points for potential first-year player i in year t.

These results are reported in the final column of Table 7. Overall, Table 7 maps to the actual draft outcome rather closely. The correlation coefficient, for example, over both years, for the estimated top twelve and the actual top twelve draftees is 0.612. And while there is some variation in the actual location of the picks, of the top ten picks in the 2018 and 2019 NBA drafts, the model estimates all in its top twelve in terms of total value. This likely underestimates the true correlation as the model fails to correctly identify the third overall pick in the 2018 draft–Luka Donc̆ić–due to the fact that he played professionally in Spain, rather than in college. Finally, there were only two players, Gary Trent in 2018 and Brandon Clarke in 2019, drafted within the NBA’s top ten where our model estimated the value significantly lower.

In terms of overall potential value, the 2019 NBA included two players, Zion Williamson and Ja Morant, who were valued at more than two points higher than the highest first-year player in the 2018 draft, Trae Young. Furthermore, the average expected value for the top twelve in the 2019 draft was a full point higher than for the top twelve in the 2018 draft. Similarly, the 2019 standard deviation was ten percent lower as compared to the 2018 draft. Moreover, as there are thirty draft picks, Fig. 6 provides a histogram of the top thirty expected value from (8) and continues to suggest that the 2019 version of the draft was of greater quality. In totality, the winning of the 2019 lottery tournament was more valuable, at least as compared to the 2018 version.

The Cost of Winning

The final part of (5) revolves around the marginal cost, i.e., $C^{'} (e_{jt})$, of winning the tournament. In our case, this cost does not revolve around the physical or intellectual cost of additional effort as is typical. Here the costs are any impact on the team’s revenues both current and in the future. Fans, typically, have greater desire to attend, watch, or purchase associated paraphernalia when teams are competitive as opposed to when they are not. Moreover, Schmidt and Berri (2006) argue that attendance and wins have become more closely associated across time. And while surveys of fan support for their team tanking are mixed, that it leads to diminished interest is less so. Gong et al. (2021), for example, examine the impact of increased chatter, i.e., twitter tweets, of NBA teams tanking on game day attendance. While they find that increased chatter of home team tanking negatively impacts game attendance in the short run, they also find possible long-term negative impacts. The long-run impact continues to exist even when there exists little current chatter on the home team tanking. In this case, teams attempting to win the draft lottery will likely experience diminished team revenues.

Interestingly, Gong et al. (2021) fail to estimate an impact on home team attendance where the away team is thought to be tanking. One might suspect, then, that diminished performance would be more costly when a team is playing in front of their home crowd as opposed to playing away from home. Figure 7 repeats the earlier rolling estimate approach of Fig. 4c, d but splits the estimation between home and away games. Consistent with the earlier suggestion, the estimate of tanking are significantly larger on the road as opposed to when the team plays at home. Teams it appears to recognize the costs of playing poorly at home and therefore do most of their shirking on the road.

Conclusion

We examine the impact of the change in the ordering of the NBA Draft Lottery. For much of the past 25 years, the NBA allocated the team with the worst record a twenty-five percent chance of obtaining the first selection in the draft with the probability of winning the top pick in the draft then decayed nonlinearly to essentially zero for the last seeded team. For the 2019 first-year draft, the NBA reduced the nonlinearity of the ‘lottery-style’ selection process. Essentially, the NBA reduced the likelihood of winning the top pick for the bottom three finishing teams and reallocated the likelihood to the remaining ‘lottery’ teams. Moreover, the NBA altered the likelihood of winning any of the top fourteen picks. This reduction in the likelihood of winning a tournament, following Lazear and Rosen (1981), may cause competitors to reduce their efforts to win.

We, therefore, examine the player portfolio selections of NBA teams over their 82 regular season games for both the 2017–2018 and 2018–2019 seasons. Using NBA game event data, we estimate an individual player’s game productivity mean and the variance–covariance matrix of all players’ performances over a 48-minute game in order to capture individual player risk as well as any possible spillover effects by or onto other players within that game. Then, given each team’s potential game assets, we construct each team’s unique efficient game frontier. Finally, we calculate the degree to which each team deviates from their respective unique game frontier across the season.

In the end, we find that as the 2017–2018 regular season progressed and teams revealed themselves to be in contention to win the lottery, those teams’ chosen player portfolio deviates away significantly from their respective game-defined efficient frontier. As the player portfolios are only for those that play in a given game, this result is not a function of players being temporary or permanently removed from the owner/manger’s available set. For example, while teams may trade players to other teams, our game efficient frontiers would adjust in this circumstance. Interestingly, this result appears to disappear during the 2018–2019 season, as an association between a team’s game return deviation—from maximum return—and relative standing no longer exists.

In terms of future adjustments to the lottery structure, Lenten et al. (2018) argue that Leagues should distribute draft-pick order by allocating the first overall pick to the team eliminated first from the championship tournament. The remainder of picks should be distributed similarly. In this way, once eliminated, teams no longer have an incentive to lose. Finally, they provide a quasi-natural experiment incorporating for more than decade’s worth of Australian Football League regular-season data and find that the proposed allocation increased the likelihood of an already eliminated team’s probability of winning late-season matches by over twenty percent.

Fig. 1

Odds of winning the draft’s first overall pick: 2018 and 2019

Table 1 Odds of winning a draft pick: 2018 and 2019

Table 2 Change in odds between 2018 and 2019

Fig. 2

Expected draft position: 2018 and 2019

Table 3 Change in the odds of winning a specific draft pick: 2018 and 2019

Table 4 Sample descriptive statistics: 2017–2018 and 2018–2019

Fig. 3

Chicago Bulls efficient frontier

Table 5 Panel estimates of the impact of draft position and return differential: 2017–2018 and 2018–2019

Fig. 4

Impact of draft position on return differential

Fig. 5

Change in the spread in winning percentage by clusters of draft position

Table 6 Descriptive statistics: 2000–2001:2016–2017

Table 7 Machine learning (random forest) forecasts of first-year players: 2017–2018 and 2018–2019

Fig. 6

Estimated first-year player predicted values: Top 30

Fig. 7

Impact of draft position on return differential: Bottom Six Teams