Modeling Change and Variation in U.S. News & WorldReport College Rankings: What would it really take to be in the Top 20?
University administrators may invest significant time and resources with the goal of improving their U.S. News& World Report ranking, but the real impact of these investments is not well known since, as other universities make similar changes, rankings become a moving target. This research removes the mystique of the U.S. News ranking process by producing a ranking model that faithfully recreates U.S. News outcomes and quantifies the inherent “noise” in the rankings for all nationally ranked universities. The model developed can be a valuable tool to institutional researchers and university leaders by providing detailed insight into the U.S. News ranking process. It allows the impact of changes to U.S. News subfactors to be studied when variation between universities and within subfactors is present. Numerous simulations were run using this model to understand the effect of each subfactor individually and to determine the amount of change that would be required for a university to improve its rank or move into the top 20. Results show that for a university ranked in the mid-30 s it would take a significant amount of additional resources, directed in a very focused way, to become a top-ranked national university, and that rank changes of up to ± 4 points should be considered “noise”. These results can serve as a basis for frank discussions within a university about the likelihood of significant changes in rank and provide valuable insight when formulating strategic goals.
KeywordsU.S. News Rankings Modeling Variation National Universities
- Adams, W. (2012). A predictable scandal. Inside Higher Education. Accessed Oct 12, 2012, from http://www.insidehighered.com/views/2012/02/07/essay-rankings-scandal-claremont-mckenna.
- Betsinger, A. (2009). One ranking to rule them all: modeling U.S. News & World Report’s predicted graduation rate, 2008 Research Grant Final Project Report. Accessed Aug 12, 2012, from http://www3.airweb.org/images/final_research_paper_betsinger_2008.pdf.
- Clarke, M. (2004). Weighing things up: A closer look at U.S. News & World Report’s ranking formulas. College and University, 79(3), 3–9.Google Scholar
- Ehrenberg, R. (2003b). Method or Madness? Inside USNWR College Rankings. Cornell Higher Education Research Institute (CHERI). Paper 39. Accessed Sept 26, 2012, from http://digitalcommons.ilr.cornell.edu/workingpapers/42.
- Farrell, E., & Van Der Werf, M. (2007), The Chronicle of Higher Education. Accessed Sept 26, 2012, from http://www.chronicle.com/article/Playing-the-Rankings-Game/4451.
- Jaschik, S. (2007). Should U.S. News Make Presidents Rich? Inside Higher Education. Accessed Sept 26, 2012, from http://insidehighered.com/news/2007/03/19/usnews.
- Lederman, D. (2009). ‘Manipulating,’ Er, Influencing ‘U.S. News.’ Inside higher education. Accessed June 5, 2009, from http://www.insidehighered.com/news/2009/06/03/rankings.
- Monks, J., & Ehrenberg, R. (1999). The impact of U.S. News & World Report college rankings on admission outcomes and pricing policies at selective private institutions. Cornell higher education research institute (CHERI). Paper 1. http://digitalcommons.ilr.cornell.edu/cheri/1.
- Morse, R., & Flanigan S. (2011). How we calculate the rankings. Accessed Jan 17, 2012, from http://www.usnews.com/education/best-colleges/articles/2011/09/12/how-us-news-calculates-the-college-rankings-2012.
- Morse, R., Flanigan, S., & Tolis, D. (2011). Association for Institutional Research Forum. May 25. U.S. News & World Report’s America’s Best Colleges: What Happened in the 2011 Edition and What’s Ahead for the New Ranking?Google Scholar
- National Association for College Admission Counseling (2011). Report of the NACAC Ad Hoc Committee on U.S. News & World Report Rankings. Accessed July 11, 2012, from http://www.nacacnet.org/about/Governance/Comm/Pages/NACACUSNewsAdHocCommittee.aspx.
- Porter, S. (2000). The robustness of the graduation rate performance indicator used in the U.S. News & World Report college rankings. The Case International Journal of Educational Advancement, 1(2), 145–164.Google Scholar
- Schmidt, P. (2008). Most colleges chase prestige on a treadmill, researchers find. The chronicles of higher education. Accessed Nov 10, 2012, from http://chronicle.com/article/Colleges-Chase-of-Prestige/1315.
- Sponsler, B. (2009). The role and relevance of rankings in higher education policymaking. Institute for Higher Education Policy Issue Brief, IB06200909, 1–24.Google Scholar
- Yablon, M. (2001). Test flight: The scam behind SAT bashing. New Republic., 30, 24–25.Google Scholar