Research in Higher Education

, Volume 55, Issue 8, pp 761–779

Modeling Change and Variation in U.S. News & WorldReport College Rankings: What would it really take to be in the Top 20?

  • Shari L. Gnolek
  • Vincenzo T. Falciano
  • Ralph W. Kuncl
Article

Abstract

University administrators may invest significant time and resources with the goal of improving their U.S. News& World Report ranking, but the real impact of these investments is not well known since, as other universities make similar changes, rankings become a moving target. This research removes the mystique of the U.S. News ranking process by producing a ranking model that faithfully recreates U.S. News outcomes and quantifies the inherent “noise” in the rankings for all nationally ranked universities. The model developed can be a valuable tool to institutional researchers and university leaders by providing detailed insight into the U.S. News ranking process. It allows the impact of changes to U.S. News subfactors to be studied when variation between universities and within subfactors is present. Numerous simulations were run using this model to understand the effect of each subfactor individually and to determine the amount of change that would be required for a university to improve its rank or move into the top 20. Results show that for a university ranked in the mid-30 s it would take a significant amount of additional resources, directed in a very focused way, to become a top-ranked national university, and that rank changes of up to ± 4 points should be considered “noise”. These results can serve as a basis for frank discussions within a university about the likelihood of significant changes in rank and provide valuable insight when formulating strategic goals.

Keywords

U.S. News Rankings Modeling Variation National Universities 

References

  1. Adams, W. (2012). A predictable scandal. Inside Higher Education. Accessed Oct 12, 2012, from http://www.insidehighered.com/views/2012/02/07/essay-rankings-scandal-claremont-mckenna.
  2. Bastedo, M., & Bowman, N. (2010). U.S. News & World Report college rankings: Modeling institutional effects on organizational reputation. American Journal of Education, 116(2), 163–183.CrossRefGoogle Scholar
  3. Betsinger, A. (2009). One ranking to rule them all: modeling U.S. News & World Report’s predicted graduation rate, 2008 Research Grant Final Project Report. Accessed Aug 12, 2012, from http://www3.airweb.org/images/final_research_paper_betsinger_2008.pdf.
  4. Bowman, N., & Bastedo, M. (2009). Getting on the front page: Organizational reputation, status signals, and the impact of U.S. News and World Report on student decisions. Research in Higher Education,. doi:10.1007/s11162-009-9129-8.Google Scholar
  5. Clarke, M. (2002). News or noise? An analysis of U.S. News and World Report’s ranking scores. Educational Measurement: Issues and Practice, 21(4), 39–48.CrossRefGoogle Scholar
  6. Clarke, M. (2004). Weighing things up: A closer look at U.S. News & World Report’s ranking formulas. College and University, 79(3), 3–9.Google Scholar
  7. Dichev, I. (2001). News or noise? Estimating the noise in the U.S. News university rankings. Research in Higher Education, 42(3), 237–266.CrossRefGoogle Scholar
  8. Ehrenberg, R. (2003a). Reaching for the brass ring: The U.S. News & World Report rankings and competition. Review of Higher Education, 26(2), 145.CrossRefGoogle Scholar
  9. Ehrenberg, R. (2003b). Method or Madness? Inside USNWR College Rankings. Cornell Higher Education Research Institute (CHERI). Paper 39. Accessed Sept 26, 2012, from http://digitalcommons.ilr.cornell.edu/workingpapers/42.
  10. Farrell, E., & Van Der Werf, M. (2007), The Chronicle of Higher Education. Accessed Sept 26, 2012, from http://www.chronicle.com/article/Playing-the-Rankings-Game/4451.
  11. Grewel, R., Dearden, J., & Llilien, G. (2008). The university rankings game: Modeling the competition among universities for ranking. The American Statistician, 62(3), 232–237.CrossRefGoogle Scholar
  12. Griffith, A., & Rask, K. (2007). The influence of the U.S. News and World Report collegiate rankings on the matriculation decision of high-ability students: 1995–2004. Economics of Education Review, 26, 244–255.CrossRefGoogle Scholar
  13. Jaschik, S. (2007). Should U.S. News Make Presidents Rich? Inside Higher Education. Accessed Sept 26, 2012, from http://insidehighered.com/news/2007/03/19/usnews.
  14. Lederman, D. (2009). ‘Manipulating,’ Er, Influencing ‘U.S. News.’ Inside higher education. Accessed June 5, 2009, from http://www.insidehighered.com/news/2009/06/03/rankings.
  15. Machung, A. (1998). Playing the rankings game. Change, 30(4), 12–16.CrossRefGoogle Scholar
  16. McDonough, P., Antonio, A., Walpole, M., & Perez, L. (1998). College rankings: Democratized college knowledge for whom? Research in Higher Education, 39(5), 513–517.CrossRefGoogle Scholar
  17. Meredith, M. (2004). Why do universities compete in the ratings game? An empirical analysis of the effects of the U.S. News and World Report college rankings. Research in Higher Education, 45(5), 443–461.CrossRefGoogle Scholar
  18. Monks, J., & Ehrenberg, R. (1999). The impact of U.S. News & World Report college rankings on admission outcomes and pricing policies at selective private institutions. Cornell higher education research institute (CHERI). Paper 1. http://digitalcommons.ilr.cornell.edu/cheri/1.
  19. Morse, R., & Flanigan S. (2011). How we calculate the rankings. Accessed Jan 17, 2012, from http://www.usnews.com/education/best-colleges/articles/2011/09/12/how-us-news-calculates-the-college-rankings-2012.
  20. Morse, R., Flanigan, S., & Tolis, D. (2011). Association for Institutional Research Forum. May 25. U.S. News & World Report’s America’s Best Colleges: What Happened in the 2011 Edition and What’s Ahead for the New Ranking?Google Scholar
  21. National Association for College Admission Counseling (2011). Report of the NACAC Ad Hoc Committee on U.S. News & World Report Rankings. Accessed July 11, 2012, from http://www.nacacnet.org/about/Governance/Comm/Pages/NACACUSNewsAdHocCommittee.aspx.
  22. Porter, S. (2000). The robustness of the graduation rate performance indicator used in the U.S. News & World Report college rankings. The Case International Journal of Educational Advancement, 1(2), 145–164.Google Scholar
  23. Sauder, M., & Lancaster, R. (2006). Do rankings matter? The effects of U.S. News & World Report Rankings on the admissions process of law schools. Law & Society Review, 40(1), 105–134.CrossRefGoogle Scholar
  24. Schmidt, P. (2008). Most colleges chase prestige on a treadmill, researchers find. The chronicles of higher education. Accessed Nov 10, 2012, from http://chronicle.com/article/Colleges-Chase-of-Prestige/1315.
  25. Sponsler, B. (2009). The role and relevance of rankings in higher education policymaking. Institute for Higher Education Policy Issue Brief, IB06200909, 1–24.Google Scholar
  26. Webster, D. (1992). Rankings of undergraduate education in U.S. News & World Report and money: are they any good? Change, 24(2), 18–31.CrossRefGoogle Scholar
  27. Webster, T. (2001). A principal component analysis of the U.S. News & World Report tier rankings of colleges and universities. Economics of Education Review, 20, 235–244.CrossRefGoogle Scholar
  28. Yablon, M. (2001). Test flight: The scam behind SAT bashing. New Republic., 30, 24–25.Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Shari L. Gnolek
    • 1
  • Vincenzo T. Falciano
    • 2
  • Ralph W. Kuncl
    • 3
  1. 1.Scannell & KurzPittsfordUSA
  2. 2.University of RochesterRochesterUSA
  3. 3.University of RedlandsRedlandsUSA

Personalised recommendations