Constructing a ranking of higher education institutions based on equity: is it possible or desirable?

Pitman, Tim; Edwards, Daniel; Zhang, Liang-Cheng; Koshy, Paul; McMillan, Julie

doi:10.1007/s10734-019-00487-0

Published: 03 January 2020

Constructing a ranking of higher education institutions based on equity: is it possible or desirable?

Tim Pitman ORCID: orcid.org/0000-0002-4237-2203¹,
Daniel Edwards²,
Liang-Cheng Zhang²,
Paul Koshy¹ &
Julie McMillan²

Higher Education volume 80, pages 605–624 (2020)Cite this article

1611 Accesses
5 Citations
11 Altmetric
Metrics details

Abstract

This paper presents findings from a research project which aimed to rank Australian higher education institutions on their ‘equity performance’; that is, the extent to which they were accessible for, supportive of and benefiting students traditionally under-represented in higher education. The study comprised a conceptual consideration of how higher education equity might be defined and empirically measured, drawing on extant scholarly research as well as observations from key stakeholders, including equity practitioners, researchers, policymakers and higher education executives and institutional planners. Based on these findings, a theoretical framework for higher education equity performance was constructed, and performance indicators identified and subjected to systematic assessment for real-world application. The ensuing ranking system was populated with institutional data from the 37 public universities in Australia. The findings from this analysis indicate that a ranking system may not be the optimal method for assessing higher education equity performance and highlights the subjective nature of both higher education equity and higher education ranking systems.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

£ 29.95

Price includes VAT (United Kingdom)
Tax calculation will be finalised during checkout.

References

Austin, K., & Heath, J. (2010). Using DEMO to evaluate and enhance schools outreach programs: an example from the South Coast of New South Wales. Paper presented at the 2nd Annual Student Equity In Higher Education National Conference, Australia.
Bastedo, M. N., & Gumport, P. J. (2003). Access to what? Mission differentiation and academic stratification in U.S. public higher education. Higher Education, 341–359.
Bexley, E. (2008). The first-year experience at La Trobe University. Melbourne, Australia: Centre for the Study of Higher Education.
Google Scholar
Boyadjieva, P., & Ilieva-Trichkova, P. (2018). From conceptualisation to measurement of higher education as a common good: challenges and possibilities. Higher Education, 1–17.
Bradley, D., Noonan, P., Nugent, H., & Scales, B. (2008). Review of Australian higher education: final report. Canberra: Department of Education, Employment and Workplace Relations.
Google Scholar
Britton, J., Dearden, J., Shephard, N., & Vignoles, A. (2016). How English domiciled graduate earnings vary with gender, institution attended, subject and socioeconomic background. Retrieved from http://www.ifs.org.uk/uploads/publications/wps/wp201606.pdf
Cabinet Office Strategy Unit. (2009). Unleashing aspiration: the final report of the panel on fair access to the professions. Retrieved from http://webarchive.nationalarchives.gov.uk/20100407163903/http://www.cabinetoffice.gov.uk/strategy/work_areas/accessprofessions.aspx
Cameron, S., Daga, R., & Outhred, R. (2018). Setting out a conceptual framework for measuring equity in learning. In UNESCO (Ed.), Handbook for meausring equity in education (pp. 16–44). UNESCO Institute for Statistics: Montreal.
Google Scholar
Cherastidtham, I., & Norton, A. (2018). University attrition: what helps and what hinders university completion? Retrieved from https://grattan.edu.au/wp-content/uploads/2018/04/University-attrition-background.pdf
Chien, C.-L., & Huebler, F. (2018). Introduction. In Handbook on measuring equity in education (pp. 11–15). Montreal, Quebec: UNESCO Institute for Statistics.
Google Scholar
CollegeNet. (2017). 2017 Social mobility index. Retrieved from http://www.socialmobilityindex.org/
Collyer, F. (2013). The production of scholarly knowledge in the global market arena: university ranking systems, prestige and power. Critical Studies in Education, 54(3), 245–259. https://doi.org/10.1080/17508487.2013.788049.
Article Google Scholar
Conner, T. W., & Rabovsky, T. M. (2011). Accountability, affordability, access: a review of the recent trends in higher education policy research. Policy Studies Journal, 39, 93–112.
Article Google Scholar
Department of Education. (2014). Completion rates of domestic bachelor students 2005–2012: a cohort analysis. Canberra: Department of Education.
Google Scholar
Department of Education. (2015). Completion rates of domestic bachelor students—a cohort analysis: 2005–2013. Canberra: Department of Education.
Google Scholar
Department of Education and Training. (2018). Selected Higher Education Statistics, 2017. Retrieved from https://www.education.gov.au/selected-higher-education-statistics-2017-student-data
Department of Education Employment and Workplace Relations. (2010). Regional participation: The role of socioeconomic status and access. Retrieved from http://pandora.nla.gov.au/pan/120403/20100705-1509/www.deewr.gov.au/HigherEducation/Programs/Equity/Documents/RegionalParticipation_report.pdf
Department of Employment Education and Training. (1990). A fair chance for all. Canberra: Australian Governmen Printing Service.
Google Scholar
Dill, D. D., & Soo, M. (2005). Academic quality, league tables, and public policy: a cross-national analysis of university ranking systems. Higher Education, 49(4), 495–533. https://doi.org/10.1007/s10734-004-1746-8.
Article Google Scholar
Doran, G. (1981). There's a S.M.A.R.T. way to write management's goals and objectives. Management Review, 70(11), 35–36.
Google Scholar
Dow, K., Adams, B., Dawson, J., & Phillips, D. (2010). Report advising on the development of the Victorian Tertiary Education Plan. Retrieved from http://www.education.vic.gov.au/about/research/Pages/vteplan.aspx
Equality of Opportunity Project. (2017). Data and Replication Code. Retrieved from http://www.equality-of-opportunity.org/data/
European Commission/EACEA/Eurydice. (2015). The European Higher Education Area in 2015: Bologna Process Implementation Report. Retrieved from https://eacea.ec.europa.eu/sites/eacea-site/files/european_higher_education_area_bologna_process_implementation_report.pdf.
Gale, T., Hattam, R., Parker, S., Comber, B., Bills, D., & Tranter, D. (2010). Interventions early in school as a means to improve higher education outcomes for disadvantaged (particularly low SES) students. Retrieved from http://dro.deakin.edu.au/view/DU:30040776
Gale, T., Parker, S., Rodd, P., Stratton, G., Sealey, T., & Moore, T. (2013). Student aspirations for higher education in Central Queensland: a survey of school students’ navigational capacities. Retrieved from https://www.ncsehe.edu.au/publications/student-aspirations-central-queensland/
Gemici, S., Bednarz, A., & Karmel, T. (2014). The factors affecting the educational and occupational aspirations of young Australians. Retrieved from https://www.ncver.edu.au/research-and-statistics/publications/all-publications/the-factors-affecting-the-educational-and-occupational-aspirations-of-young-australians
Gemici, S., Lim, P., & Karmel, T. (2013). The impact of schools on young people's transition to university. Longitudinal Surveys of Australian Youth. Research Report 61. Adelaide: National Centre for Vocational Education Research Ltd.
Australia, G. C. (2014). Graduate salaries 2013: a report on the earnings of new Australian graduates in their first full-time employment. Melbourne: Graduate Careers Australia Ltd..
Google Scholar
Hahn, A., Leavitt, T., & Aaron, P. (1994). Evaluation of the quantum opportunities program (QOP). Did the program work? A report on the post secondary outcomes and cost-effectiveness of the QOP program (1989–1993). Retrieved from http://ies.ed.gov/ncee/wwc/study/29918
Higher Education Standards Panel. (2017). Improving retention, completion and success in higher education. Retrieved from https://docs.education.gov.au/node/50816
Hodges, B., Bedford, T., Hartley, J., Klinger, C., Murray, N., O'Rourke, J., & Schofield, N. (2013). Enabling retention: processes and strategies for improving student retention in university-based enabling programs. Sydney: Office for Learning and Teaching.
Google Scholar
Hsu, A., & Zomer, A. (2016). Environmental performance index’. Wiley StatsRef: Statistics Reference Online. doi:https://doi.org/10.1002/9781118445112.stat03789.pub2.
Johnes, J., & Taylor, J. (1989). Undergraduate non-completion rates: differences between UK universities. Higher Education, 18(2), 209–225. https://doi.org/10.1007/BF00139181.
Article Google Scholar
Koshy, P. (2016). Equity policy in Australian higher education: Past, present and prospective. In Hill, M., Hudson, A., Mckendry, S., Raven, N., Saunders, D., Storan, J. and Ward, T. (Eds.) Closing the Gap: Bridges for Access and Lifelong Learning, Forum for Access and Continuing Education: London: forum for access and continuing education.
Koshy, P., Seymour, R., & Dockery, A. M. (2016). Are there institutional differences in the earnings of Australian higher education graduates? Economic Analysis and Policy, 51, 1–11.
Article Google Scholar
Lewis, D. R., Johnson, D. R., Erikson, R. N., & Bruininks, R. H. (1994). Multiattribute evaluation of program alternatives within special education. Journal of Disability Policy Studies, 5(1), 77–112.
Article Google Scholar
Li, I., & Dockery, A. M. (2015). Does school socio-economic status influence university outcomes? Australian Journal of Labour Economics, 18(1), 75–94.
Google Scholar
Lim, P., Bednarz, A., & Karmel, T. (2014). The impact of school academic quality on low socioeconomic status students. Retrieved from https://www.ncver.edu.au/research-and-statistics/publications/all-publications/the-impact-of-school-academic-quality-on-low-socioeconomic-status-students
Liu, A. (2011). Unraveling the myth of meritocracy within the context of US higher education. Higher Education, 62(4), 383–397.
Article Google Scholar
Lomax-Smith, J., Watson, L., & Webster, B. (2011). Higher education base funding review: final report. Retrieved from https://www.voced.edu.au/content/ngv%3A49506
Luzeckyj, A., King, S., Scutter, S., & Brinkworth, R. (2011). The significance of being first: a consideration of cultural capital in relation to “first in family” student’s choices of university and program. A practice report. The International Journal of the First Year in Higher Education and Culture, 2(2), 91–96. https://doi.org/10.5204/intjfyhe.v2i2.89.
Article Google Scholar
Marginson, S. (2011). Equity, status and freedom: a note on higher education. Cambridge Journal of Education, 41(1), 23–36.
Article Google Scholar
Marginson, S., & Van der Wende, M. (2007). To rank or to be ranked: the impact of global rankings in higher education. Journal of Studies in International Education, 11(3–4), 306–329.
Article Google Scholar
Office for Fair Access. (2014). National strategy for access and student success in higher education: department for Business. Innovation and: Skills.
Google Scholar
Perna, L., & Swail, W. (2001). Pre-college outreach and early intervention. Thought & Action, 17(1), 99–110.
Google Scholar
Phillimore, J., & Koshy, P. (2010). Meeting the Rudd government’s equity targets for universities: Three scenarios. People and Place, 18(1), 1–18.
Google Scholar
Pitman, T. (2014). Unlocking the gates to the peasants: are policies of ‘fairness’ or ‘inclusion’ more important for equity in higher education? Cambridge Journal of Education, 1–13. https://doi.org/10.1080/0305764X.2014.970514.
Pitman, T., Trinidad, S., Devlin, M., Harvey, A., Brett, M., & McKay, J. (2016). Pathways to higher education: the efficacy of enabling and sub-bachelor pathways for disadvantaged students. In National Centre for student equity in higher education (NCSEHE). Perth: Curtin University.
Google Scholar
Pitman, T., Roberts, L., Bennett, D., & Richardson, S. (2017). An Australian study of graduate outcomes for disadvantaged students. Journal of Further and Higher Education, 1–13. https://doi.org/10.1080/0309877X.2017.1349895.
Podinovskii, V. V. (1994). Criteria importance theory. Mathematical Social Sciences, 27(3), 237–252. https://doi.org/10.1016/0165-4896(93)00737-F.
Article Google Scholar
Poister, T. H., Aristigueta, M. P., & Hall, J. L. (2014). Managing and measuring performance in public and nonprofit organizations: an integrated approach (2 ed.). San Francisco: John Wiley.
QS. (2014). QS World University Rankings: Methodology. Retrieved from http://www.topuniversities.com/university-rankings-articles/world-university-rankings/qs-world-university-rankings-methodology
Raphael, D. D. (1946). Equality and equity. Philosophy, 21(79), 118–132. https://doi.org/10.1017/S0031819100005301.
Article Google Scholar
Reay, D., Crozier, G., & Clayton, J. (2009). 'Strangers in paradise'?: working-class students in elite universities. Sociology, 43(6), 1103–1121.
Article Google Scholar
Saaty, T. L. (1990). How to make a decision: the analytic hierarchy process. European Journal of Operational Research, 48(1), 9–26.
Article Google Scholar
Soh, K. (2017). The seven deadly sins of world university ranking: a summary from several papers. Journal of Higher Education Policy and Management, 39(1), 104–115. https://doi.org/10.1080/1360080X.2016.1254431.
Article Google Scholar
Southgate, E., Douglas, H., Scevak, J., M, S., Rubin, M., & Lindell, C. (2014). The academic outcomes of first-in-family in an Australian university: an exploratory study. International Studies in Widening Participation, 1(2), 31–45.
Google Scholar
Stolz, I., Hendel, D. D., & Horn, A. S. (2010). Ranking of rankings: benchmarking twenty-five higher education ranking systems in Europe. Higher Education, 60(5), 507–528. https://doi.org/10.1007/s10734-010-9312-z.
Article Google Scholar
Strayhorn, T. L. (2010). When race and gender collide: social and cultural capital’s influence on the academic achievement of African American and Latino males. The Review of Higher Education, 33(3), 307–332.
Article Google Scholar
The Pell Institute, & PennAhead. (2015). Indicators of higher education equity in the United States. Retrieved from http://www.pellinstitute.org/downloads/publications-Indicators_of_Higher_Education_Equity_in_the_US_45_Year_Trend_Report.pdf
UN Economic and Social Council. (1999). General comment no. 13: The right to education (Art. 13 of the covenant). Retrieved from http://www.refworld.org/docid/4538838c22.html
Washington Monthly. (2016). 2016 College Guide and Rankings. Retrieved from https://washingtonmonthly.com/college_guide
Wellings, P., Black, R., Craven, G., Freshwater, D., & Harding, S. (2019) Performance-based funding for the commonwealth grant scheme, report for the Minister for Education (June) retrieved from.https://docs.education.gov.au/node/52995
Zeleny, M. (1982). Multiple criteria decision making. New York: McGraw-Hill.
Google Scholar

Download references

Acknowledgements

Funding for the research underpinning this paper was provided by the Australian Government Department of Education and Training.

Author information

Affiliations

Curtin University, PO Box U1987, Perth, 6845, Australia
Tim Pitman & Paul Koshy
Australian Council for Educational Research, Camberwell, VIC, 3124, Australia
Daniel Edwards, Liang-Cheng Zhang & Julie McMillan

Authors

Tim Pitman
View author publications
You can also search for this author in PubMed Google Scholar
Daniel Edwards
View author publications
You can also search for this author in PubMed Google Scholar
Liang-Cheng Zhang
View author publications
You can also search for this author in PubMed Google Scholar
Paul Koshy
View author publications
You can also search for this author in PubMed Google Scholar
Julie McMillan
View author publications
You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tim Pitman.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Pitman, T., Edwards, D., Zhang, LC. et al. Constructing a ranking of higher education institutions based on equity: is it possible or desirable?. High Educ 80, 605–624 (2020). https://doi.org/10.1007/s10734-019-00487-0

Download citation

Published: 03 January 2020
Issue Date: October 2020
DOI: https://doi.org/10.1007/s10734-019-00487-0

Keywords

University ranking
Higher education equity
Comparative
Performance measurement