Abstract
The reverse order college draft gives the worst teams in the National Football League (NFL) the opportunity to hire the best amateur talent. For it to work effectively, teams must be able to identify the “best” talent. Our study of NFL quarterbacks highlights problems with the draft process. We find only a weak correlation between teams’ evaluations on draft day and subsequent quarterback performance in the NFL. Moreover, many of the factors that enhance a quarterback’s draft position are unrelated to future NFL performance. Our analysis highlights the difficulties in evaluating workers in the uncertain environment of professional sports.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Quinn (2008) also reviewed research on the impact the draft has had on competitive balance. This research indicates there is little relationship between a reverse order draft and the level of competitive balance.
ESPN.com, as well as other web sites, reports the equation for the NFL’s quarterback’s rating.
Specifically, from Table One we see that a play that does not produce any yards will cost a team −2.7 points. An interception will cost a team 34.5 yards while the cost of losing a fumble is 36.4 yards. As noted in Berri (2007), the value of 30 for a turnover is chosen for simplicity.
The NFL and AFL merged before the start of the 1970 season. According to ProFootbalReference.com, from 1970 to 1976 the NFL’s draft consisted of 17 rounds and at least 442 picks. From 1978 to 1992 the draft was only 12 rounds (and from 330 to 336 picks). For the 1993 season the NFL draft was eight rounds and 224 picks. After the 1993 season the draft was only seven rounds. In 1994 there were only 222 picks. But after 1994 the number of picks exceeded 250 (but never exceeding 262 picks) in all but three seasons. Consequently we settled on a cut-off of pick 250 for our study. Quarterbacks chosen after 250 were not considered for this examination.
Our NFL performance data on quarterbacks was taken from sports.yahoo.com. (http://sports.yahoo.com/nfl/stats/byposition?pos=QB). The NFL has changed quite a bit since 1970. Consequently, to compare quarterbacks across this time period one has to adjust the numbers. Specifically, we calculated a quarterback’s relative performance with respect to each statistic. This calculation began by calculating the average performance in each statistic from 1970 to 2007. Then in each year we subtracted the average in that statistic from that season from each quarterback’s performance in that statistical measure. We then added the average performance across the entire period. For example, in 1975 Terry Bradshaw’s net points per play was 0.162. The average quarterback in 1975 posted a net points per play mark of 0.088 while the average mark from 1970 to 2007 was 0.144. Given these numbers, Bradshaw’s relative net points per play in 1975 was 0.220, or [(0.162−0.088) + 0.144]. It is these relative numbers that were used in our analysis of quarterbacks from 1970 to 2007.
The history of the NFL’s National Invitational Camp can be found at (http://www.nflcombine.net/?q=node/9).
Combine data from 1999 to 2008 can be found at nfldraftscout.com.
Information on the number of test questions, the time given for the test, and average score in the population was taken from an article published in The USA Today by Chappell (2006).
The Wonderlic scores we utilized were taken from NFL Quarterback Wonderlic Scores (http://www.macmirabile.com/wonderlic.htm). This is a website maintained by Mac Mirabile. As Mirabile notes, “… these results represent research and generally come from reliable sources, i.e., Notes from NFL scouts, newspaper articles. It is important to understand that scores cannot by “verified” since they are not released by the NFL, but rather leaded by teams or scouts.”.
Data on college quarterbacks since 2000 was taken from the NCAA’s website reporting Division I Football Statistics (http://web1.ncaa.org/d1mfb/mainpage.jsp?site=org). College data for quarterbacks selected in the 1999 and 2000 drafts was taken from CNNSI.com.
Data on 20 yard and 10 yard dash times, vertical jump, broad jump, and the shuttle and cone test was reported for fewer than 90 quarterbacks in our sample. Consequently these variables were not included in our study.
According to the Centers for Disease Control and Prevention (cdc.gov), the Body Mass Index is calculated by first dividing Weight (in pounds) by height (in inches) squared. This number is then multiplied by 703. A score of 18.5 indicates that a person’s weight is below normal. A score between 18.5 and 24.9 is considered normal. A BMI from 25.0 to 29.9 is indicates a person is overweight. And scores above 30.0 are indicative of an obese person. In our sample of NFL quarterbacks the average BMI score was 27.8, with a range from 24.4 to 31.5. The CDC notes that “highly trained athletes may have a high BMI because of increased muscularity rather than increased body fatness.” [http://www.cdc.gov/nccdphp/dnpa/healthyweight/assessing/bmi/adult_BMI/about_adult_BMI.htm#Interpreted].
This is true whether we measure performance with QB Score, Net Points, Wins Produced, or the NFL’s QB Rating. These results are available from the authors upon request. When we turn to per play measures of QB Score, Net Points, and Wins Produced, we do find that faster times in the 40 yard dash lead to reduced levels of per play performance (at the 10% level of significance). The adjusted R-squared from these regressions, though, is in the negative range and the F-statistic is statistically insignificant. Such results indicate that there is little relationship between the combine statistics and per play performance.
These results might also indicate that our Wins Produced measure of college performance is imperfect.
Career Plays is the number of plays a quarterback participated in throughout his college career. We only were able to collect career numbers on 105 quarterbacks taken from 2001 to the present. The inclusion of this variable was inspired by an article by David Lewin posted at ESPN.com [College Stats Don’t Lie (April 17, 2008): http://sports.espn.go.com/nfl/news/story?id=3350135]. Lewin argued in this article that NFL performance was influenced by only two statistics, games started in college and completion percentage. Lewin’s full results were not published, but he did indicate that his sample consisted of “highly drafted quarterbacks since 1996.” We did not have data on games started for all the quarterbacks selected since 1999, but we do think the number of career plays would be highly correlated with the number of games started in a quarterback’s career.
The NCAA groups teams into Division I-A (now called the Football Bowl Subdivision), Division I-AA (Football Championship Subdivision), Division II, and Division III. Of the 132 quarterbacks in our sample, only 16 did not come from a Division I-A school. Our results indicate that not playing in the Football Bowl Subdivision reduces your draft position by 56–63 slots, or nearly two rounds.
With respect to QB Rating, blacks posted an average mark of 100.6 versus 92.6 for whites. For QB Score per play, Net points per play, and Wins Produced per 100 play the differences were 3.676 versus 3.021, 0.311 versus 0.257, and 0.818 versus 0.673, respectively.
One potential issue is that there is very little variation in college performance. After all, only the quarterbacks who are considered the best in college get a chance to play in the NFL. To address this issue we looked at quarterbacks from 1998 to 2007 that were both drafted and logged at least 100 plays in a single NFL regular season. In all we had 215 NFL season observations. The standard deviation in QB Score per play, Net Points per play, Wins Produced per 100 plays, and QB Rating in the NFL was 1.054, 0.086, 0.231, and 14.45, respectively. For these quarterbacks the standard deviation for these same stats in college was 1.001, 0.081, 0.219, and 15.47. In sum, with respect to QB Score, Net Points, and Wins Produced we see slightly less variation in the college numbers. For QB Rating, though, the college numbers have a greater level of variation.
We also regressed PICK on just the college performance numbers. When we regress PICK on aggregate college performance numbers (QB Score, Net Points, or Wins Produced) we are able to explain 7% of the variation in draft position. When we consider the per play measures and career plays, our explanatory power rises to 9%.
There did not appear a simple way to present all of these regressions in a table. The results, though, are available from the authors upon request.
The Wonderlic score was statistically significant for 1 year of experience, but when this happened the sign was negative. In other words, higher Wonderlic scores were associated with lower levels of performance.
When we estimate Eq. (4) with completion percentage as the performance metric, we find that college completion percentage is statistically significant. But the model’s adjusted r-squared is only 0.18. In other words, much of the variation in an NFL’s quarterback’s completion percentage is not related to what that player did in college. Furthermore, the estimated elasticity of NFL completion percentage relative to college completion percentage is only 0.34. So a 10% increase in a player’s completion percentage in college only leads to a 3.4% improvement in what the player will do in the NFL. Such results suggest that although college completion percentage has statistical significance, the economic significance of this factor is quite small.
We did find that Wonderlic had a negative impact on touchdowns per attempt during the first year of experience.
We also considered NFL career performance after 3 years (to hold experience constant) as a dependent variable. We further experimented with college career plays as an explanatory variable. The results of estimation were little different.
References
Berri DJ (2007) Back to back evaluation on the gridiron. In: Albert JH, Koning RH (eds) Statistical thinking in sport. Chapman & Hall/CRC, Boca Raton
Berri DJ, Simmons R (2009) Race and the evaluation of signal callers in the National Football League. J Sports Econ 10(1):23–43. doi:10.1177/1527002508327383
Berri DJ, Schmidt MB, Brook SL (2006) The wages of wins: taking measure of the many myths in modern sport. Stanford University Press, Stanford CA
Blass AA (1992) Does the baseball labor market contradict the human capital model of investment? Rev Econ Stats 74(2):261–268
Chappell M (2006) Wondering about the Wonderlic? Try it. USA Today, 28 Feb. http://www.usatoday.com/sports/football/draft/2006-02-28-wonderlic_x.htm
Fort RD (2006) Sports economics, 2nd edn. Prentice Hall, Upper Saddle River
Hendricks W, Debrock L, Koenker R (2003) Uncertainty, hiring, and subsequent performance: the NFL draft. J Labor Econ 21(4):857–886. doi:10.1086/377025
Kahn LM (2000) The sports business as a labor market laboratory. J Econ Perspect 14(3):75–94
Krautmann AC (1999) What’s wrong with Scully-estimates of a player’s marginal revenue product? Econ Inq 37(2):369–381
Krautmann AC, von Allmen P, Berri DJ (2009) The underpayment of restricted players in North American sports leagues. Int J Sport Fin 4(3):161–175
Lazear EP (1986) Incentive contracts. National Bureau of Economic Research Working Paper No. W1917
Leeds MA, von Allmen P (2008) The economics of sports, 3rd edn. Addison Wesley, New York
Lehn K (1982) Property rights, risk sharing and player disability in Major League Baseball. J Law Econ 25(2):343–366. doi:10.1086/467020
Massey C, Thaler R (2005) Overconfidence vs. market efficiency in the National Football League. NBER Working Paper No. 11270
Quinn KG (2008) Player drafts in the major North American sports leagues. In: Humphreys BR, Howard D (eds) The business of sport, vol 3. Praeger, Westport
Quirk J, Fort R (1992) Pay dirt: the business of professional team sports. Princeton University Press, Princeton
Scully GW (1974) Pay and performance in Major League Baseball. Am Econ Rev 64(6):915–930
Simmons R, Berri DJ (2009) Gains from specialization and free agency: the story from the gridiron. Rev Ind Organ 34(1):81–98. doi:10.1007/s11151-009-9200-9
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Berri, D.J., Simmons, R. Catching a draft: on the process of selecting quarterbacks in the National Football League amateur draft. J Prod Anal 35, 37–49 (2011). https://doi.org/10.1007/s11123-009-0154-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11123-009-0154-6