Abstract
The paper tries to convey the idea that choosing a winner among a group of nominees or short-listed candidates may hurt those who bestow prizes, those who are selected, as well as those who base their own choices on the ranking. We base our observations on examples of contests (movies, literature, and music) in which winners often turn out not to be better than nominees. Our suggestion is therefore to select, say five candidates, and not to rank them, but reward all nominees equally.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Le Monde, September 7, 2010, p. 21.
See Masa Mizuno’s Music Directory, http://www.afn.org/~afn39483/ (accessed on October 14, 2012).
English (2005, p. 25).
See Courty and Pagliero (2013).
See http://gostbustere.tumblr.com/post/17158444595/speech-by-john-berger-on-accepting-the-booker-prize-for (consulted April 2, 2013).
The Oscars winning movies are guessed by every newspaper weeks before the ceremony; a session of the Queen Elisabeth piano competition takes a full month for those who reach the finals; names of possible winners of literary prizes are circulated during weeks by specialized literary newspapers before the winners are announced.
For a complete list of winners and nominees since 1929, see http://www.imdb.com/event/ev0000003/1929.
There were 4 between 1929 and 1931 and from 1945 to the present days. This number was 7 in 1932, 9 in 1934 and between 1937 and 1944, and 11 in 1935 and 1936.
This is related to Ginsburgh (2003), who looked at movies produced between 1950 and 1980, three Top Movie Lists only and was not interested in comparing Oscars and nominated movies.
It may happen that no nominated movie is of quality larger than 0. This is indicated by the number of titles that were nominated (usually four), but did not achieve more than 0 quality. In 1930 (1), there were no other nominations than the Oscar.
See also the notes at the bottom of the table.
In 1929, two movies were awarded the Oscar, but we took only the better one (Sunrise) and ignored the other (Wings) in our calculations. There were two ceremonies in 1930, but in 1930 (1), there were no official nominees and the data were discarded as well. There was no ceremony in 1933.
Note that if Oscars are excluded, the average for nominees is 4.5, that is higher that the one for winners.
There is of course a long tail of movies of quality 1 and 0.
In one case (1931), no movie made it to any list.
According to the 2010 Yearbook of European Audiovisual Observatory, this number varies between 600 and 900 during the period 1999–2009.
For complete rules see http://www.oscars.org/awards/academyawards/rules/85/rule01.html.
A rough guess at this can be made as follows. Assume there are 100 eligible movies that are candidates for Oscars. Over the 66 years, this amounts to 6,600 movies. The 15 best movie lists contain a little less than 500 movies (of which 310 are of quality less than 3).
Hodgson (2009) was able to do this since he could get hold of the number of competitions in which the same wines were competing, during the same year.
In nine occasions, it would have selected movies that ended up being of identical quality. This would have put Gone With the Wind on equal footing with The Wizard of Oz in 1940, or Annie Hall with Star Wars in 1978.
See Flores and Ginsburgh (1996) for the details of the analysis. The authors also find that the effect of order of appearance on ranking is much weaker in the Queen Elisabeth violin contest.
For the full list of one or two winners and the three to five additional shortlisted writers, see en.wikipedia.org/wiki/List_of_winners_and_shortlisted_authors_of_the_Booker_Prize_for_Fiction.
We can unfortunately not exclude that the number of copies in each re-edition of winning titles is larger than that for short-listed titles.
For having sometimes been present as the prize list was announced, one winner at a time, it was horrible to follow the decreasing length of clapping as the announcements of names got beyond the first three winners.
References
Awards, honors and prizes (2005). Farmington Hills, MI: Thomson Gale.
Bouton, L., & Kirchsteiger, G. (2011). Good rankings are bad. Why reliable rankings can hurt consumers, CEPR Discussion Paper 8702, December.
Coupé, T. (2013). Peer review versus citations: An analysis of best paper prizes. Research and Policy, 42, 295–301.
Coupé, T., Ginsburgh, V., & Noury, A. (2010). Are leading papers of better quality. Oxford Economic Papers, 61, 1–11.
Courty, P., & Pagliero, M. (2013). The pricing of art and the art of pricing: Pricing styles in the concert industry. In V. Ginsburgh & D. Throsby (Eds.), The handbook of the economics of art and culture (Vol. 2). Amsterdam/New York: Elsevier. forthcoming.
Danziger, S., Lev, J., & Avnaim-Pesso, L. (2010). Extraneous factors in judicial decisions. Proceedings of the National Academy of Sciences, 108, 6889–6892.
de Piles, R. (1708). In J. Thuillier (Ed.), Cours de peinture par principes (p. 1989). Paris: Gallimard.
English, J. (2005). The economy of prestige. Cambridge: Harvard University Press.
English, J. (2013). The economics of cultural awards. In V. Ginsburgh & D. Throsby (Eds.), The handbook of the economics of art and culture (Vol. 2). Amsterdam/New York: Elsevier. forthcoming.
Flores, R., & Ginsburgh, V. (1996). The Queen Elisabeth musical competition, how fair is the final ranking. Journal of the Royal Statistical Society (The Statistician), 45, 97–104.
Frey, B. (2005). Knight fever: Towards an economics of awards. Working Paper 239, University of Zurich, Institute for Empirical Research in Economics.
Frey, B. (2006). Giving and receiving awards. Perspectives on Psychological Science, 1, 377–388.
Frey, B. (2007). Awards as compensation. European Management Review, 4, 6–14.
Ginsburgh, V. (2003). Awards, success and aesthetic quality in the arts. Journal of Economic Perspectives, 17, 99–111.
Ginsburgh, V., & Van Ours, J. (2003). Expert opinion and compensation: Evidence from a musical competition. American Economic Review, 93, 73–90.
Ginsburgh, V., & Weyers, S. (2008). On the contemporaneousness of Roger de Piles’ Balance des Peintres. In J. Amariglio, S. Cullenberg, & J. Childers (Eds.), The sublime economy. London: Routledge.
Glejser, H., & Heyndels, B. (2001). The ranking of finalists in the Queen Elisabeth international music competition. Journal of Cultural Economics, 25, 109–129.
Graddy, K. (2013). Taste endures! The rankings of Roger de Piles and three centuries of art prices. Journal of Economic History, 73, 766–791.
Hodgson, R. (2009). An analysis of the concordance among 13 wine competitions. Journal of Wine Economics, 4, 1–9.
Kahneman, D. (2011). Thinking fast and slow. New York: Farrar, Straus and Giroux.
Lampsonius, D. (1572). Les effigies des peintres célèbres des Pays-Bas (p. 1956). Brussels: Desclée de Brouwer.
Lee, J. (2008). Outlier aversion in evaluating performance: Evidence from figure skating. Journal of Sports Economics, 9, 141–159.
Meehl, P. (1996). Clinical versus statistical prediction. Northvale, NJ/London: Jason Aronson Inc.
Pokorny, M., & Sedgwick, J. (Eds.). (2005). An economic history of film. London: Routledge.
Redelmeier, D., & Baxter, S. (2009). Rainy weather and medical school admission interviews. Canadian Medical Association Journal, 181, 933.
van Mander, K. (1604). Het schilder-boeck. Haerlem: Paschier Van Wesbusch.
Vasari, G. (1568). In A. Chastel (Ed.), Les vies des meilleurs peintres, sculpteurs et architectes (p. 1981). Paris: Berger Levrault.
Author information
Authors and Affiliations
Corresponding author
Additional information
The authors are grateful to two anonymous referees and to Kathryn Graddy for comments that much improved the paper. We are also grateful to Victor Fernandez-Blanco who always made himself available to answer our many questions on movies. The part of the paper on the Oscars as well as the general discussion on whether we need rankings is new. To show that what is discussed about movies may also apply in other fields of the arts; we, however, draw on previous work on musical competitions and the Booker Prize.
Appendix: Fifteen top 100 movie lists
Appendix: Fifteen top 100 movie lists
-
1.
Rolling Stone Magazine, 100 Maverick Movies of the Last 100 Years (1999)
-
Established by the Magazine and critic Peter Travers
-
-
2.
Los Angeles Daily News Readers’ Poll, Greatest American Films (1999)
-
Readers’ poll organized on the basis of a list of 400 movies established by the American Film Institute
-
-
3.
Video Detective, Top 100 Films of all Time (1997)
-
A guide written by Jim Riffel
-
-
4.
Time Out Film Guide, Top 100 Films (1998)
-
Compiled from readers who submitted their all-time top ten film lists
-
-
5.
Time Out Film Guide, Top 100 Films (1995)
-
Directors, producers, actors, programmers and critics were polled to name their top ten films
-
-
6.
Village Voice Critics’ Poll, 100 Best Films of the twentieth century (2000)
-
Fifty or more distinguished film critics, including Molly Haskell, Jonathan Rosenbaum, Andrew Sarris and others, to vote in their film poll http://www.filmsite.org/villvoice.html
-
-
7.
FilmFour, 100 Greatest Films of all Time (no date)
-
List established by UK’s Channel 4
-
-
8.
Guinness Book of Film, The Top 100 Films by Genre Type (1999)
-
One hundred films extracted among the 1,000 films listed in the Guinness Book of Film
-
http://www.filmsite.org/guiness.html (no longer available in October 2012)
-
-
9.
Entertainment Weekly’s, 100 Greatest Movies of All Time (1999)
-
A guide written by senior editor Ty Burr
-
-
10.
Mr. Showbiz’s Critics Picks, 100 Best Movies of all Time (2003?)
-
A list made on the Mr. Showbiz Web site, a full year and a half before the American Film Institute announced their own list of 100 Greatest American Movies
-
-
11.
Mr. Showbiz’s Readers Picks, 100 Best Movies of all Time (2003?)
-
see list no. 10
-
-
12.
Leonard Maltin, 100 Must-See Films of the twentieth century (2000)
-
List established by American film critic Leonard Maltin
-
-
13.
American Film Institution, American Greatest Movies (2005)
-
Movies selected by AFI’s blue-ribbon panel of more than 1,500 leaders of the American movie community
-
-
14.
American Multi Cinema, 100 Greatest Films (no date)
-
See comments on the website
-
-
15.
Movieline Magazine’s 100 Best Movies ever Made (1995)
-
List complied by Virginia Campbell and Edward Margulies
-
Most websites give details about how their lists are compiled. Last accessed in 2011.