Constructing a ranking of higher education institutions based on equity: is it possible or desirable?

Higher Education volume 80, pages605–624(2020)Cite this article

Abstract

This paper presents findings from a research project which aimed to rank Australian higher education institutions on their ‘equity performance’; that is, the extent to which they were accessible for, supportive of and benefiting students traditionally under-represented in higher education. The study comprised a conceptual consideration of how higher education equity might be defined and empirically measured, drawing on extant scholarly research as well as observations from key stakeholders, including equity practitioners, researchers, policymakers and higher education executives and institutional planners. Based on these findings, a theoretical framework for higher education equity performance was constructed, and performance indicators identified and subjected to systematic assessment for real-world application. The ensuing ranking system was populated with institutional data from the 37 public universities in Australia. The findings from this analysis indicate that a ranking system may not be the optimal method for assessing higher education equity performance and highlights the subjective nature of both higher education equity and higher education ranking systems.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

US$ 39.95

Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. Austin, K., & Heath, J. (2010). Using DEMO to evaluate and enhance schools outreach programs: an example from the South Coast of New South Wales. Paper presented at the 2nd Annual Student Equity In Higher Education National Conference, Australia.

  2. Bastedo, M. N., & Gumport, P. J. (2003). Access to what? Mission differentiation and academic stratification in U.S. public higher education. Higher Education, 341–359.

  3. Bexley, E. (2008). The first-year experience at La Trobe University. Melbourne, Australia: Centre for the Study of Higher Education.

    Google Scholar 

  4. Boyadjieva, P., & Ilieva-Trichkova, P. (2018). From conceptualisation to measurement of higher education as a common good: challenges and possibilities. Higher Education, 1–17.

  5. Bradley, D., Noonan, P., Nugent, H., & Scales, B. (2008). Review of Australian higher education: final report. Canberra: Department of Education, Employment and Workplace Relations.

    Google Scholar 

  6. Britton, J., Dearden, J., Shephard, N., & Vignoles, A. (2016). How English domiciled graduate earnings vary with gender, institution attended, subject and socioeconomic background. Retrieved from http://www.ifs.org.uk/uploads/publications/wps/wp201606.pdf

  7. Cabinet Office Strategy Unit. (2009). Unleashing aspiration: the final report of the panel on fair access to the professions. Retrieved from http://webarchive.nationalarchives.gov.uk/20100407163903/http://www.cabinetoffice.gov.uk/strategy/work_areas/accessprofessions.aspx

  8. Cameron, S., Daga, R., & Outhred, R. (2018). Setting out a conceptual framework for measuring equity in learning. In UNESCO (Ed.), Handbook for meausring equity in education (pp. 16–44). UNESCO Institute for Statistics: Montreal.

    Google Scholar 

  9. Cherastidtham, I., & Norton, A. (2018). University attrition: what helps and what hinders university completion? Retrieved from https://grattan.edu.au/wp-content/uploads/2018/04/University-attrition-background.pdf

  10. Chien, C.-L., & Huebler, F. (2018). Introduction. In Handbook on measuring equity in education (pp. 11–15). Montreal, Quebec: UNESCO Institute for Statistics.

    Google Scholar 

  11. CollegeNet. (2017). 2017 Social mobility index. Retrieved from http://www.socialmobilityindex.org/

  12. Collyer, F. (2013). The production of scholarly knowledge in the global market arena: university ranking systems, prestige and power. Critical Studies in Education, 54(3), 245–259. https://doi.org/10.1080/17508487.2013.788049.

    Article  Google Scholar 

  13. Conner, T. W., & Rabovsky, T. M. (2011). Accountability, affordability, access: a review of the recent trends in higher education policy research. Policy Studies Journal, 39, 93–112.

    Article  Google Scholar 

  14. Department of Education. (2014). Completion rates of domestic bachelor students 2005–2012: a cohort analysis. Canberra: Department of Education.

    Google Scholar 

  15. Department of Education. (2015). Completion rates of domestic bachelor students—a cohort analysis: 2005–2013. Canberra: Department of Education.

    Google Scholar 

  16. Department of Education and Training. (2018). Selected Higher Education Statistics, 2017. Retrieved from https://www.education.gov.au/selected-higher-education-statistics-2017-student-data

  17. Department of Education Employment and Workplace Relations. (2010). Regional participation: The role of socioeconomic status and access. Retrieved from http://pandora.nla.gov.au/pan/120403/20100705-1509/www.deewr.gov.au/HigherEducation/Programs/Equity/Documents/RegionalParticipation_report.pdf

  18. Department of Employment Education and Training. (1990). A fair chance for all. Canberra: Australian Governmen Printing Service.

    Google Scholar 

  19. Dill, D. D., & Soo, M. (2005). Academic quality, league tables, and public policy: a cross-national analysis of university ranking systems. Higher Education, 49(4), 495–533. https://doi.org/10.1007/s10734-004-1746-8.

    Article  Google Scholar 

  20. Doran, G. (1981). There's a S.M.A.R.T. way to write management's goals and objectives. Management Review, 70(11), 35–36.

    Google Scholar 

  21. Dow, K., Adams, B., Dawson, J., & Phillips, D. (2010). Report advising on the development of the Victorian Tertiary Education Plan. Retrieved from http://www.education.vic.gov.au/about/research/Pages/vteplan.aspx

  22. Equality of Opportunity Project. (2017). Data and Replication Code. Retrieved from http://www.equality-of-opportunity.org/data/

  23. European Commission/EACEA/Eurydice. (2015). The European Higher Education Area in 2015: Bologna Process Implementation Report. Retrieved from https://eacea.ec.europa.eu/sites/eacea-site/files/european_higher_education_area_bologna_process_implementation_report.pdf.

  24. Gale, T., Hattam, R., Parker, S., Comber, B., Bills, D., & Tranter, D. (2010). Interventions early in school as a means to improve higher education outcomes for disadvantaged (particularly low SES) students. Retrieved from http://dro.deakin.edu.au/view/DU:30040776

  25. Gale, T., Parker, S., Rodd, P., Stratton, G., Sealey, T., & Moore, T. (2013). Student aspirations for higher education in Central Queensland: a survey of school students’ navigational capacities. Retrieved from https://www.ncsehe.edu.au/publications/student-aspirations-central-queensland/

  26. Gemici, S., Bednarz, A., & Karmel, T. (2014). The factors affecting the educational and occupational aspirations of young Australians. Retrieved from https://www.ncver.edu.au/research-and-statistics/publications/all-publications/the-factors-affecting-the-educational-and-occupational-aspirations-of-young-australians

  27. Gemici, S., Lim, P., & Karmel, T. (2013). The impact of schools on young people's transition to university. Longitudinal Surveys of Australian Youth. Research Report 61. Adelaide: National Centre for Vocational Education Research Ltd.

  28. Australia, G. C. (2014). Graduate salaries 2013: a report on the earnings of new Australian graduates in their first full-time employment. Melbourne: Graduate Careers Australia Ltd..

    Google Scholar 

  29. Hahn, A., Leavitt, T., & Aaron, P. (1994). Evaluation of the quantum opportunities program (QOP). Did the program work? A report on the post secondary outcomes and cost-effectiveness of the QOP program (1989–1993). Retrieved from http://ies.ed.gov/ncee/wwc/study/29918

  30. Higher Education Standards Panel. (2017). Improving retention, completion and success in higher education. Retrieved from https://docs.education.gov.au/node/50816

  31. Hodges, B., Bedford, T., Hartley, J., Klinger, C., Murray, N., O'Rourke, J., & Schofield, N. (2013). Enabling retention: processes and strategies for improving student retention in university-based enabling programs. Sydney: Office for Learning and Teaching.

    Google Scholar 

  32. Hsu, A., & Zomer, A. (2016). Environmental performance index’. Wiley StatsRef: Statistics Reference Online. doi:https://doi.org/10.1002/9781118445112.stat03789.pub2.

  33. Johnes, J., & Taylor, J. (1989). Undergraduate non-completion rates: differences between UK universities. Higher Education, 18(2), 209–225. https://doi.org/10.1007/BF00139181.

    Article  Google Scholar 

  34. Koshy, P. (2016). Equity policy in Australian higher education: Past, present and prospective. In Hill, M., Hudson, A., Mckendry, S., Raven, N., Saunders, D., Storan, J. and Ward, T. (Eds.) Closing the Gap: Bridges for Access and Lifelong Learning, Forum for Access and Continuing Education: London: forum for access and continuing education.

  35. Koshy, P., Seymour, R., & Dockery, A. M. (2016). Are there institutional differences in the earnings of Australian higher education graduates? Economic Analysis and Policy, 51, 1–11.

    Article  Google Scholar 

  36. Lewis, D. R., Johnson, D. R., Erikson, R. N., & Bruininks, R. H. (1994). Multiattribute evaluation of program alternatives within special education. Journal of Disability Policy Studies, 5(1), 77–112.

    Article  Google Scholar 

  37. Li, I., & Dockery, A. M. (2015). Does school socio-economic status influence university outcomes? Australian Journal of Labour Economics, 18(1), 75–94.

    Google Scholar 

  38. Lim, P., Bednarz, A., & Karmel, T. (2014). The impact of school academic quality on low socioeconomic status students. Retrieved from https://www.ncver.edu.au/research-and-statistics/publications/all-publications/the-impact-of-school-academic-quality-on-low-socioeconomic-status-students

  39. Liu, A. (2011). Unraveling the myth of meritocracy within the context of US higher education. Higher Education, 62(4), 383–397.

    Article  Google Scholar 

  40. Lomax-Smith, J., Watson, L., & Webster, B. (2011). Higher education base funding review: final report. Retrieved from https://www.voced.edu.au/content/ngv%3A49506

  41. Luzeckyj, A., King, S., Scutter, S., & Brinkworth, R. (2011). The significance of being first: a consideration of cultural capital in relation to “first in family” student’s choices of university and program. A practice report. The International Journal of the First Year in Higher Education and Culture, 2(2), 91–96. https://doi.org/10.5204/intjfyhe.v2i2.89.

    Article  Google Scholar 

  42. Marginson, S. (2011). Equity, status and freedom: a note on higher education. Cambridge Journal of Education, 41(1), 23–36.

    Article  Google Scholar 

  43. Marginson, S., & Van der Wende, M. (2007). To rank or to be ranked: the impact of global rankings in higher education. Journal of Studies in International Education, 11(3–4), 306–329.

    Article  Google Scholar 

  44. Office for Fair Access. (2014). National strategy for access and student success in higher education: department for Business. Innovation and: Skills.

    Google Scholar 

  45. Perna, L., & Swail, W. (2001). Pre-college outreach and early intervention. Thought & Action, 17(1), 99–110.

    Google Scholar 

  46. Phillimore, J., & Koshy, P. (2010). Meeting the Rudd government’s equity targets for universities: Three scenarios. People and Place, 18(1), 1–18.

    Google Scholar 

  47. Pitman, T. (2014). Unlocking the gates to the peasants: are policies of ‘fairness’ or ‘inclusion’ more important for equity in higher education? Cambridge Journal of Education, 1–13. https://doi.org/10.1080/0305764X.2014.970514.

  48. Pitman, T., Trinidad, S., Devlin, M., Harvey, A., Brett, M., & McKay, J. (2016). Pathways to higher education: the efficacy of enabling and sub-bachelor pathways for disadvantaged students. In National Centre for student equity in higher education (NCSEHE). Perth: Curtin University.

    Google Scholar 

  49. Pitman, T., Roberts, L., Bennett, D., & Richardson, S. (2017). An Australian study of graduate outcomes for disadvantaged students. Journal of Further and Higher Education, 1–13. https://doi.org/10.1080/0309877X.2017.1349895.

  50. Podinovskii, V. V. (1994). Criteria importance theory. Mathematical Social Sciences, 27(3), 237–252. https://doi.org/10.1016/0165-4896(93)00737-F.

    Article  Google Scholar 

  51. Poister, T. H., Aristigueta, M. P., & Hall, J. L. (2014). Managing and measuring performance in public and nonprofit organizations: an integrated approach (2 ed.). San Francisco: John Wiley.

  52. QS. (2014). QS World University Rankings: Methodology. Retrieved from http://www.topuniversities.com/university-rankings-articles/world-university-rankings/qs-world-university-rankings-methodology

  53. Raphael, D. D. (1946). Equality and equity. Philosophy, 21(79), 118–132. https://doi.org/10.1017/S0031819100005301.

    Article  Google Scholar 

  54. Reay, D., Crozier, G., & Clayton, J. (2009). 'Strangers in paradise'?: working-class students in elite universities. Sociology, 43(6), 1103–1121.

    Article  Google Scholar 

  55. Saaty, T. L. (1990). How to make a decision: the analytic hierarchy process. European Journal of Operational Research, 48(1), 9–26.

    Article  Google Scholar 

  56. Soh, K. (2017). The seven deadly sins of world university ranking: a summary from several papers. Journal of Higher Education Policy and Management, 39(1), 104–115. https://doi.org/10.1080/1360080X.2016.1254431.

    Article  Google Scholar 

  57. Southgate, E., Douglas, H., Scevak, J., M, S., Rubin, M., & Lindell, C. (2014). The academic outcomes of first-in-family in an Australian university: an exploratory study. International Studies in Widening Participation, 1(2), 31–45.

    Google Scholar 

  58. Stolz, I., Hendel, D. D., & Horn, A. S. (2010). Ranking of rankings: benchmarking twenty-five higher education ranking systems in Europe. Higher Education, 60(5), 507–528. https://doi.org/10.1007/s10734-010-9312-z.

    Article  Google Scholar 

  59. Strayhorn, T. L. (2010). When race and gender collide: social and cultural capital’s influence on the academic achievement of African American and Latino males. The Review of Higher Education, 33(3), 307–332.

    Article  Google Scholar 

  60. The Pell Institute, & PennAhead. (2015). Indicators of higher education equity in the United States. Retrieved from http://www.pellinstitute.org/downloads/publications-Indicators_of_Higher_Education_Equity_in_the_US_45_Year_Trend_Report.pdf

  61. UN Economic and Social Council. (1999). General comment no. 13: The right to education (Art. 13 of the covenant). Retrieved from http://www.refworld.org/docid/4538838c22.html

  62. Washington Monthly. (2016). 2016 College Guide and Rankings. Retrieved from https://washingtonmonthly.com/college_guide

  63. Wellings, P., Black, R., Craven, G., Freshwater, D., & Harding, S. (2019) Performance-based funding for the commonwealth grant scheme, report for the Minister for Education (June) retrieved from.https://docs.education.gov.au/node/52995

  64. Zeleny, M. (1982). Multiple criteria decision making. New York: McGraw-Hill.

    Google Scholar 

Download references

Acknowledgements

Funding for the research underpinning this paper was provided by the Australian Government Department of Education and Training.

Author information

Affiliations

  1. Curtin University, PO Box U1987, Perth, 6845, Australia

    Tim Pitman & Paul Koshy

  2. Australian Council for Educational Research, Camberwell, VIC, 3124, Australia

    Daniel Edwards, Liang-Cheng Zhang & Julie McMillan

Authors
  1. Tim Pitman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel Edwards
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liang-Cheng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Paul Koshy
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Julie McMillan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tim Pitman.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Pitman, T., Edwards, D., Zhang, LC. et al. Constructing a ranking of higher education institutions based on equity: is it possible or desirable?. High Educ 80, 605–624 (2020). https://doi.org/10.1007/s10734-019-00487-0

Download citation

Keywords

  • University ranking
  • Higher education equity
  • Comparative
  • Performance measurement