Introduction

For modern countries, quality assessment of the higher education system is important. The system refers to the organisational structure of higher education institutions and the personnel and infrastructure required to educate postsecondary students. It has value as a progression of primary and secondary education, not only as an industry in itself but also as an important source of trained and highly educated citizens in the country. That is why modern countries are eager to improve the quality of the higher education system, which requires a clear and accurate assessment of the system to find a breakthrough in its development, especially after Covid-19 brought many changes to the industry. At the 70th session of the United Nations General Assembly in September 2015, the 2030 Agenda for Sustainable Development also emphasised the importance of lifelong education (UNESCO, 2016). Unfortunately, strong evidence suggests that countries will get more excellent value in return by investing their limited financial resources in primary and secondary education rather than in universities or technical training (McCowan, 2016), so they prefer to invest in the former.

A developed higher education system has value both as an industry and as a source of trained and educated citizens for the nation’s economy. It also plays an essential role in global sustainable development (Franco et al., 2019). Measuring the quality and sustainability of higher education systems is more complex because primary and secondary education can visually reflect the quality of education through the level of students. In contrast, higher education carries the complex parts of research missions, academic integrity, and transnational exchange. Thus, an effective higher education assessment system is difficult yet indispensable. Our job is to find the correct elements to assess the quality and sustainability of higher education in a country or region, project it, and improve the quality and sustainability of higher education by proposing policies.

Several scholars have studied the quality and sustainability of higher education in some countries in recent years. Moreover, campus operations, inter-university collaboration, and partnerships with government and civil societies are essential factors that influence higher education sustainability (Wu and Shen, 2016). It is also influenced by economic and social factors, such as sustainable value-investing donations and community partnership building (Barlett and Chase, 2004). While not all schools engage in all of these activities, the sector identifies core sustainability initiatives in higher education: academic, operational, and administrative (Owens, 2017).

For this study, we selected a dozen factors relating to the quality and sustainability of higher education. This was done through a literature review and an analysis of existing theories. We collected datasets for nine countries with good higher education development by collating data sources, such as the World Bank’s public database and the Organization for Economic Co-operation and Development (OECD) database. We used the entropy valuation method to assign weights to these factors. Based on this, we constructed a quality-sustainability model (QSM) for evaluating higher education in different countries. Using this model and the different factors, with their weight assignments, we derived two indicators that can be used to evaluate the quality and sustainability of higher education in a country. The model was used to assess and rank the quality and sustainability of higher education in the nine countries. To verify the model’s generalisability and value in guiding the quality and sustainability of higher education in a country, we selected some representative countries and assessed their sub-indicators. A regression analysis was conducted using historical data relating to higher education to apply the model’s effects more effectively and to develop goals and policies for higher education development. The study showed that the higher education systems of different countries have varying strengths, which include the developed countries with more advanced higher education systems. The development of higher education needs to be comprehensive and balanced, and if countries share their experiences of development, they may be able to compensate for their own shortcomings and guide their sustainable development more effectively.

This study innovatively explores the results of a quantitative assessment of higher education from the perspectives of economics and sociology. While higher education must be promoted, there is a need to consider supply and demand in the education and employment markets and to avoid social problems, such as an imbalanced talent structure. When evaluating education, the quality of teaching and learning can be considered from several different perspectives. However, constructing a national or regional education system requires a macro perspective on each aspect, and a sudden increase in individual indicators is not necessarily beneficial to the educational system as a whole. The construction of a higher education system requires the support of multidisciplinary theories, and its development needs to encompass many aspects. This study adds the indicator of academic integrity to traditional higher education assessment methods, which is represented by the retraction rate of papers. This study develops popular contemporary higher education assessment measures and provides new perspectives on evaluating education systems.

First, the latter section compares the current development of higher education assessment activities and describes how the 13 indicators needed to construct the QSM have been used to assess higher education throughout history. Second, we use entropic assessment methods to assign weights to these factors to construct the QSM and then use the model and the different factors and weight assignments to derive the two indicators used to assess the quality and sustainability of the higher education system. Third, we compiled data from nine developed countries with mature and reliable data on higher education development based on authoritative data, such as the World Bank and OECD databases, and used the QSM for assessment. Finally, we discuss the model’s results and the findings of its application to provide suggestions for the development of higher education.

Literature review

The activity of higher education assessment has multiplied in the last hundred years and is now very active (Wiethe-Körprich and Bley, 2017). In recent decades, the number of higher education institutions has increased significantly, and the evaluation of higher education has been increasingly studied (Van Mol et al., 2021). The primary purpose of this activity is to improve the quality of the higher education system. Currently, the higher education system in each country is integrated into the national system, generally funded by the state and serving national needs (Reymert et al., 2021). Despite the increasing internationalisation of academic careers, they are still formed in national contexts. Furthermore, national research systems differ in their research priorities and evaluation systems. Universities also have different levels of control over resources (Sivertsen, 2017).

Evaluation activities began many years ago and can be traced (Guba and Lincoln, 1981). In the mid-1960s, assessment began to develop as a scientific field in the United Kingdom and the United States (Worthen and Sanders, 1987). Assessment activities are widely applied and generally defined as recognising, clarifying, and implementing essential criteria to define an object’s value according to these criteria (Fitzpatrick et al., 2004). Using the same set of criteria or models for different regional higher education systems facilitates the identification of differences and the search for relative strengths and weaknesses. Through assessment activities, we can explore the direction of higher education development. This paper compiles the methods and indicators commonly used in existing studies for analysis to determine whether a higher education system is of high quality. As higher education continues to change and assessment activities evolve, a number of methods and important metrics are commonly used in this area, including feedback, formative evaluation, and peer evaluation (Leihy and Salazar, 2017). These methods have good generalisability and can be applied to assess the quality of teaching and learning in higher education in various contexts as well as primary and secondary education. However, to some extent, these three methods ignore the academic, research-based nature of higher education.

The advent of the era of big data has brought more feasibility to assessment activities, and the rational use of data and assessment methods can better ensure the independence of assessment methods, enhance research reliability, and reduce randomness. When assessing higher education, scholars often go back to first hypothesising the impact of certain factors on the quality of higher education or the relationship between certain scholarly output data and higher education. These data are then examined through analytical methods such as Bayesian back-propagation (Gao and Yu, 2021), entropy weight-TOPSIS, and logistic model (Zhang et al., 2021). Other methods are often applied; for example, Yang et al. (2018) constructed a two-stage network model and used Luenberger indicators to analyse the productivity and evolution of research universities in China. Thanassoulis et al. (2017) examined the role of student assessment in higher education evaluation, using a combination of the analytical hierarchy process (AHP) and data envelopment analysis (DEA) to help teachers understand the direction for improving teaching and learning activities. In addition, many scholars have developed studies of higher education systems using structural equation modelling (SEM), which can be used to test alternative models, reliability, validity, theoretical support models, and data screening. Green (2016) examined the existing literature using SEM to study higher education and found that this model is often used to test alternative models, reliability, validity, theoretical support models, and data screening in higher education research models. Big data and data mining techniques have also been applied to higher education evaluations. Data science research methods, including time-varying clustering sampling algorithms, data mining, and big data grey relational decision-making algorithms, have all been applied to data research related to higher education and have been used to find the development of methodologies and the promotion of better-quality teaching and learning (Feng, 2021; Liu and Song, 2021).

After an evaluation, the quality of the research method or model needs to be tested, and this process focuses on the reliability and validity of the evaluation. Especially in recent years, big data technologies have often been applied to higher education’s monitoring and quality analysis. We must use effective data analysis methods to evaluate the relevance of indicators, the applicability of methods, and the representativeness of evaluation subjects in the study to grasp the quality of the evaluation. Xu et al. (2022) evaluated the level of sustainable development of Japanese higher education using factor analysis and principal component analysis. Subsequently, structural validity tests were used to test the rationality of the model, quantitatively assessing the effectiveness of the policy and its impact on reality.

As higher education assessment activities have evolved through the data era, academics have gradually identified many indicators that can assess the quality of higher education (Gupta et al., 2015), such as the employment rate of graduates, the number of papers, and the gender ratio. Therefore, it is important to consider these classical elements, which remain important measures of the quality of higher education, when studying new changes that may impact higher education. Considering the important role of higher education in academic research and the now common incidents of academic misconduct, we have considered indicators such as the number of patents per capita, paper retraction rate, and self-citation rate, which are rarely discussed in historical studies along with those traditional indicators, when organising the assessment of the quality and sustainability potential of higher education. Therefore, based on the indicators for measuring the quality of the higher education system, as confirmed by historical studies, we created the QSM.

Symbol table and assumptions

We propose the following assumptions to better model the problem studied in this paper:

  • The higher education system includes universities, academies and colleges. It represents levels 5–8 of the 2011 International Standard Classification of Education (ISCED) structure. Levels 5–8 refer to tertiary education, from short-cycle tertiary education to doctoral level, and comprise the ISCED definition of higher education, which is the standard used in many data sources.

  • Higher education in each country functions independently of that in other countries, meaning it is not necessary to calculate synergistic cooperation between the higher education systems of each country.

  • There are minor variations in higher education levels within countries or regions, and some average indicators are therefore representative.

  • The assumption is made that indicators are purely positively or negatively correlated and that no critical values change the logic of the indicators.

The abbreviations used in this paper and their corresponding definitions are shown in Table 1. The meanings and definitions of the 13 indicators can be found in Table 2.1 in Appendices B—Tables.

Table 1 Symbol definition.
Full size table

The QSM of higher education

Two models and indicator selection

To assess the quality and sustainability of the higher education system, we built the QSM. It is a multidimensional assessment tool that assesses multiple dimensions of a country’s higher education system, including economic support, innovation capacity, student development, and academic integrity, to produce an overall assessment of the country’s higher education system. The model is divided into two parts: quality (QL) and sustainable development (SD), which are used to assess the higher education system’s current quality vitality and future development potential, respectively. The quality of the higher education system is concerned with the current quality of the system, while sustainability refers to the integration of environmental, social, and economic considerations into the policies, practices, and operations of universities and colleges. It involves a commitment to meeting the present generation’s needs without compromising future generations’ ability to meet their own needs. The model results of each country can be compared with the evaluation results of other countries to discover the shortcomings of the system and the direction of improvement and to provide objective and effective decision support for government decision-makers and members of the system.

As mentioned in the literature review, historical studies have identified some factors that are inevitably taken into account when assessing higher education, and they directly reflect the quality of higher education and are undoubtedly elements of the quality and sustainability of the higher education system in this study, such as the number of papers published and cited, the proportion of international students, and the salary level of graduates. These indicators are also commonly used in educational assessments to assess the quality of higher education institutions. However, considering that we are assessing the level of higher education within a certain region, these data cannot be compared directly but are divided by the corresponding base, such as the total population of the region, economic base indicators, and the total number of higher education institutions within the region.

Innovation capacity and research capacity correspond to the average number of patents and publications per 1,000 citizens. Scientific research is an essential task for the sustainable development of higher education, and the higher the research capability, the more robust the sustainable development capability of higher education. Therefore, many past studies have explored the academic and innovation capacities of some regions around these two indicators. For example, Xu et al. (2013) proposed a multi-attribute comprehensive evaluation method of individual research output (IRO), which can assess the academic ability of scholars by the number and quality of their publications and overcome the one-sidedness of a single indicator to some extent by considering more elements. Chen et al. (2009) analysed Chinese patent filing activity in eight economic regions of China from 1999 to 2004, exploring the relationship among gross domestic product (GDP), research and development, and patent filing in various regions and organisations in China. These studies show that publications and patents can reflect technological development and higher education in a country or region. They embody the academic and creative capacity that is the essential foundation of the higher education system and supports high-quality and sustainable development.

We measure a country’s level of international exchange by the percentage of its students that are international. The cross-border mobility of students can profoundly impact the development of higher education and is a reflection of its good reputation and quality (Abdullah et al., 2017). Therefore, a higher percentage of international students reflects a high level of internationalisation in local higher education and a higher quality level. However, recruiting international students presents several challenges for the higher education system. Sherry et al. (2010) studied the University of Toledo, which has an increasing number of international students, and found that international students face issues such as a new culture, language, and finances. Mature higher education systems often have a reputation for good quality and can defuse their challenges and those of international students, playing an important role in international academic exchange. Thus, the level of internationalisation is an indication of the quality of the higher education system.

The relative position of the salary level of college graduates in society can visualise the degree value of higher education. The pay gap between those with and without higher education reflects the value of higher education in the job market, which helps it be valued and promotes sustainability. Drydakis (2016) compared Bachelor of Science graduates in economics from several U.K. schools in a field study and found that graduates from top-ranked universities can receive more interview invitations and higher entry salaries than other graduates, reflecting the impact of university quality on graduate salaries. High-value degrees can be the foundation of sustainable higher education by attracting capable students and maintaining the vitality of colleges and universities.

The complexity of the higher education system makes it more challenging to focus on assessment activities, especially when we need to explore their quality and sustainability. While we can accurately judge and compare a university by its student performance and research outcomes, when looking at the entire higher education system, we have to focus on academic integrity, financial commitment, and gender equity as elements of sustainability. After compiling historical research, we selected a number of these indicators to measure the quality of the higher education system, including barriers to educational entry, government attention, student development input, gender equity, academic integrity, faculty salaries, and speculation. In addition, because higher education systems often contain many universities, we can combine existing university rankings to determine the number of quality universities within a system and thus estimate the quality of the higher education system in that region. Although applied and validated in different education system assessment scenarios, the selected indicators were once rarely combined with traditional indicators of higher education quality to explore the quality and sustainability of higher education systems within a country or region. These innovations help assess the quality of the higher education system more comprehensively and reveal where the potential for development lies, enhancing the value of the model’s application.

Barriers to educational entry can be measured by the gross enrolment rate of higher education. Reflecting how many people have access to higher education in a country or region, high levels of access reflect that the country’s higher education system is of higher quality and has the potential to grow and be sustainable. Enrolment rates visually represent how many young people of the right age in a country can enter the higher education system. Higher education enrolment rates are generally higher in developed countries than in developing countries. Jiménez et al. (2017) studied the impact of educational enrolment on national entrepreneurship in different countries in Latin America. They found that countries with higher enrolment in higher education have lower entrepreneurship because higher education reduces information asymmetry and perceptions of adverse selection, thus increasing employer trust and promoting business development. Enrolment rates also indicate the supply and demand for higher education. De Campos et al. (2018) analysed data obtained from the Brazilian census and studied indicators related to the quality of higher education in Brazil. They examine the balance between supply and demand in higher education in Brazil. In evaluating the higher education system in the United States, for example, there has been a significant amount of scholarly research on enrolment rates (Fortin, 2006). Luo et al. (2018) and Bozick et al. (2016) examined changes in college enrolment in the United States and the factors that influenced them and explored the impact of enrolment on social development. Based on historical research, we use higher education enrolment rates to measure the quality and sustainability of the system.

Government attention and student development input are measured by the ratio of financial investment in education to GDP, total expenditure per student, and GDP per capita. They are all financial indicators of the level of higher education. The government’s investment in higher education is conducive to quality and higher education development. High government investment in students contributes to building talent within higher education institutions, improving organisational quality, and the sustained output of higher education talent. Tregub and Buffet (2019) studied the impact of French investments in the education system at all levels on French society. It developed recommendations to optimise financial flows in education. Wu and Liu (2009) studied the Chinese government’s investment in higher education from 1985 to 2004, subjected the data to correlation analysis and the Granger causality test with economic growth, tested for series smoothness and cointegration analysis, and found a significant positive correlation between the two. Therefore, it is concluded that investment in higher education improves its quality and promotes socioeconomic development.

Gender equity can be reflected in higher education enrolment rates for male and female students. The more significant the gender gap is in enrolment, the lower the quality of higher education is in the country, as it indicates a gender gap in the higher education system and that this is unhelpful for the sustainability of higher education. In the last few years, David (2011), Devos (2012), Johnson et al. (2015), and many other scholars have discussed gender equality in the higher education system to find a way to develop equity in higher education.

We measure the university quality of a country or region based on the number of its universities that appear in the top 1000 of the QS World University Rankings (QS Rankings) as a percentage of the total number of universities in the region. University quality reflects the level of higher education and international recognition and is an element of the sustainable development of higher education. The QS rankings are credible third-party rankings of world universities based on academic reputation, employer reputation, faculty/student ratio, citations per faculty, and international faculty ratio. The quality of universities is judged based on academic reputation, employer reputation, faculty/student ratio, citations per faculty, international faculty ratio, and international student ratio Craig (2022). Dowsett (2020) has examined the relationship between the international rankings of four Australian universities and their development strategies, showing that specific changes in strategic direction can improve a university’s market position and contribute to significant improvements in its ranking. High-quality universities are numerous and can reflect the overall high quality of a higher education system, so this indicator can be used to measure the sustainability potential of the higher education system.

Academic integrity is reflected in the retraction rate of journal articles. The higher retraction rates of high-level papers reflect higher levels of academic misconduct and lower quality within higher education. Nagella and Madhugiri (2020) studied the retraction rate of journals in the medical field and found that the number of retractions is proportional to the number of papers published by the journal and the citation metrics of the journal. Meanwhile, Yeo-Teh and Tang (2021) and Abritis et al. (2021) both found an “alarming” and “exceptionally high” retraction rate for papers on the coronavirus disease (COVID-19) in recent years, which may be due to errors or fraud in the study. Generally, the lower the retraction rate, the better the academic integrity. A high-quality higher education system should maintain a low rate of retraction of academic output.

Faculty salaries within the higher education system can motivate college faculty and administrators and indirectly influence the quality of a region’s higher education system. High faculty salaries reflect that higher education talent is respected, contributes to higher education’s organisational quality, attracts more talent to higher education, and promotes sustainable development. Cao and Yu (2020), Guo and Wang (2017), and other scholars have studied college teachers’ salaries and incentive systems to analyse how to motivate college teachers. These findings all support the idea that high compensation for higher education faculty can serve as an incentive to improve higher education’s quality and sustainability potential.

The literature’s frequency of citations can reflect the Research Value of the higher education system; accordingly, an excessive self-citation rate is associated with speculative behaviour. The average number of citations of papers within a country indicates the value of higher education research output, which affects higher education sustainability. The high self-citation rate is a well-known phenomenon of academic speculation, illustrating academic dishonesty as detrimental to higher education’s quality and sustainability. Pasterkamp et al. (2007) examined 8864 articles from nine countries and found that self-citations were more common in cardiovascular-related journals and that the United States had the highest research output, with more citations of papers from that country. As a commonly used measure of academic integrity, this indicator will help us analyse the quality and sustainability of the education system.

We have selected 13 quantifiable indicators to help us assess the quality and sustainability of a country’s higher education: barriers to educational entry (access to education), innovation capacity, research capacity, international exchange, government attention, student input, gender equity, academic quality, academic integrity, faculty salaries, degree value, research value, and speculation. In particular, we introduce here three indicators of negative logic: gender injustice (S7), academic misconduct (S9), and speculative behaviour (S13), which are detrimental to the quality and sustainability of higher education when they are higher, as shown in Table 3 in Appendices B—Tables.

We selected indicators from the United States, Australia, Germany, the Netherlands, Japan, Norway, Canada, Sweden and the United Kingdom as measures for the evaluation of higher-level higher education. These countries all have well-developed higher education systems and credible publicly available data. The same measures can be applied across Asia, the Americas, Europe and Australia, which balances the differences caused by geographical location. Using the data from these countries to validate the model will facilitate our ability to improve the model’s credibility and open a discussion on the development of higher education systems in these countries. The specific values of these countries’ indicators are shown in Tables 2.1 and 2.2 in Appendices B—Tables.

Following a model-building process, described in Appendices A—QSM modelling, we developed the following QSM equation (the specific data during the calculation as shown in Appendices C), whereby a higher QSM index represents a state of better quality and sustainability in higher education. The function Si∗ is the standardised index data.

$$QSM = \mathop {\sum }\limits_{i = 1}^n w_iS_i^ \ast$$

Application of QSM: preliminary assessment of nine countries

Using the model to evaluate data from nine countries in 2018

The QSM assesses higher education systems in a country or region. Therefore, the corresponding country or region needs a certain number of higher education institutions to improve the accuracy of the model assessment. At the same time, the higher education system of the country or region needs to be regularly counted and evaluated to produce comparable data as the basis for the model study.

The QSM gives us higher education quality and sustainability indexes for nine countries in the previous constituency. This model’s results will be analysed in detail in the third section of this chapter. Table 2 shows the specific indices and rankings of each country’s indicators.

Table 2 Quality and sustainability index for higher education in selected countries.
Full size table

In the QSM results, Australia was ranked first in the QL rankings, the UK was ranked first in the SD rankings, while US higher education, which is traditionally perceived as large and leading, did not perform as well as expected. Figures 1 and 2 show the performance of these countries.

Fig. 1: QSM–QL of Australia and QSM–SD of the United Kingdom.
figure 1

This figure shows the QL and SD ranking results of the Australia and UK with the 13 indicators.

Full size image
Fig. 2: QSM–QL and QSM–SD of the U.S.
figure 2

This figure shows the QL and SD ranking results of the U.S. with the 13 indicators.

Full size image

Evolution of the countries in 2010–2016

In addition to applying the QSM to the data from nine countries for 2018, we reviewed changes in higher education in the United States, Australia, Japan and Germany between 2010 and 2016. These four countries are in North America, Oceania, Asia and Europe, respectively, and are considered representative of their continents. They are among the most economically and educationally developed countries in their regions. At the same time, due to their educational traditions and geography, there are significant differences between their educational systems, and it is valuable to compare them in a comprehensive study. The results of the QSM are shown in Figs. 3 and 4.

Fig. 3: Data in QSM–QL for selected countries for previous years.
figure 3

This figure shows the QSM–QL results of the U.S., Australia, Germany and Japan from 2010 to 2016.

Full size image
Fig. 4: Data in QSM–SD for selected countries for previous years.
figure 4

This figure shows the QSM–SD results of the U.S., Australia, Germany and Japan from 2010 to 2016.

Full size image

2018 Data discussion

We found something interesting in the results of our QSM assessment. Although the United States has a very high reputation for higher education, its higher education quality and sustainability indicators are not optimal for statistical results. Many of the same factors exist in our QSM–QL and QSM–SD, closely related to a country’s higher education quality and sustainability indices. However, some countries, including the United Kingdom and Japan, still show a large gap between the indicators. The model has different weights in measuring higher education quality and sustainable development on each indicator. We used the model to play a fundamental role in analysing these countries.

Let us dissect why Australia achieved the top ranking among several countries on quality indicators. Although Australia shows low levels of innovation capacity (S2) and gender equity (S7) in higher education, these indicators are not highly weighted in the QSM–QL, as shown in Fig. 1. Conversely, Australia has excellent performance on highly weighted and critical indicators, such as barriers to educational entry (S1) and academic integrity (S9). Australia has made many efforts to achieve a high-quality higher education system, first and foremost in its higher education participation and partnerships program (HEPPP). It has been implemented since 2009, and its primary goal is to create more opportunities for higher education for people of low socioeconomic status, citizens of remote areas, and indigenous people, including financial support for educational programmes and students, increasing access to higher education in remote areas, and promoting female access to education and employment. It guarantees equality of opportunity for Australian citizens to access higher education, makes the threshold for higher education in Australia lower compared with other countries studied, and plays an important role in the quality of Australia’s higher education system. Government support, both policy and financial, has provided the higher education system with a better basis for achieving universal access, which allows Australia to be a place where higher education can be rated highly in terms of quality, as it is a place where many people have access to quality higher education.

Let us dissect why the U.K. has jumped to the top on the sustainability indicators. In the same vein as Australia, the U.K. performs well on the highly weighted indicators, especially student development input (S6), as shown in Fig. 1. Even though the U.K. does not have a high percentage of university students enrolled (S1), the government is very willing to improve the quality of learning for every university student, resulting from their elite higher education that produces high-level higher education talent.

Among the study’s findings, indications that run counter to the general intuition that higher education in the United States is the most advanced are of concern. As shown in Fig. 2, the U.S. loses to “statistics” as they are ranked at the bottom of the nine countries on several highly weighted indicators. The U.S.’s high population averages high results and lowers its research capacity (S3) and international exchange (S4). Although the U.S. has many good universities, it also has many community colleges and specialised graduate schools, which reduces its overall quality (S8). The data show more academic speculation (S13) in the U.S., and although they have a high citation rate for papers, they have a higher self-citation rate. Interestingly, although the U.S. has the highest degree value (S11) among the countries, their male-to-female enrolment ratios vary greatly. This phenomenon is difficult to determine and is a complex pedagogical and sociological issue that we will not discuss further here. What is clear is that the model results show that neither the quality nor the sustainability level of U.S. higher education is as good as expected.

Dynamic data discussion

First, the most notable of our higher education quality indicators is the precipitous drop in the U.S. quality index after 2014, as shown in Fig. 3. Since our indicator compares to the highest level in that year, the worldwide regression around 2010 may not be readily noticeable. Combined with the 2018 data, the U.S. declined to 0.3093, likely due to the country not recovering from the effects of the 2008 financial crisis. The smooth political situation in Germany and Australia also kept their higher education quality at a high and stable level. Therefore, we discuss changes in the level of quality and sustainability of higher education in Japan and the United States.

Japan’s Higher Education Quality Index declined sharply between 2011 and 2013 and quickly returned to higher levels. This unusual period of movement is unparalleled in any other country. However, we quickly discovered the reason for this by examining a series of efforts by Japanese society and the Ministry of Education, Culture, Sports, Science and Technology (MEXT) at the time. In 2011, Japan experienced the Great East Japan Earthquake and the Fukushima nuclear power plant crisis, which put Japanese society under tremendous pressure (Okuyama and Inaba, 2017). As a result, Japan’s economic development, financial situation, and educational stability have been significantly affected during this period, putting greater pressure on the Japanese higher education system, as reflected in the sharp decline in the Higher Education Quality Index for two consecutive years in our model. At the same time, the Japanese higher education system, faced with the challenges of earthquake damage, ageing, and globalisation, is committed to reforming its universities in line with national and social expectations and helping to promote a dynamic and sustainable social structure. Japan, driven by MEXT, enacted the National University Reform Plan in November 2013, which details a blueprint for reforming Japan’s higher education system. Its most significant feature is that the plan encourages each university to fully justify its strengths and characteristics and encourages autonomous improvement and development, thereby enhancing competitiveness and generating new added value. It also planned and financially supported ways to enhance the innovative capacity of Japanese universities and promote the improvement of human resources and international development. Over the next many years, MEXT effectively implemented and tracked this plan, achieving a series of efforts, such as promoting the diversification of higher education and tuition remission, and continued to track and repair the damage caused by the Great East Japan Earthquake for many years, issuing new specific plans every two to three years. The efforts of the Japanese higher education system have led to a rapid recovery in the quality of their higher education, which grew at a high rate in the two years after 2013, returning to pre-earthquake levels.

The U.S. appears to perform flat in higher education’s sustainability indicator. However, in 2018 it dropped to 0.3186, with student development input (S6) falling to almost the nine countries’ lowest level, as shown in Fig. 4. During this time, student loan debt has been increasing within the U.S. higher education system, and more and more graduates face difficulties repaying their student loans. At the same time, due to the recession and tight job market, many graduates find it difficult to find adequate jobs to pay off their debt. This has led to increased student loan defaults and general concerns about overburdening higher education (Barr et al., 2019). These issues show up in the QSM as declining values starting in 2015, given that both the U.S. and its education system’s quality and sustainability were facing challenges at that time.

A discussion of the static and dynamic data raises questions regarding the design of the model: as the QSM takes a holistic view of the many aspects that affect the quality and sustainability of higher education, these factors may influence or even constrain each other. This becomes increasingly evident when the QSM is applied to higher education on a larger scale. For example, the U.S. has many top institutions of higher education, with the result that many people think the region has the highest quality higher education. However, within a large higher education system, many mediocre schools and students can go unnoticed, resulting in a weaker performance when the QSM is used to assess the whole region, as compared to smaller regions with consistently higher quality schools, such as Australia and the UK.

At the same time, from the policy perspective, it is difficult to take into account every aspect of a country’s higher education development promotion, and a focus on increasing the QSM indicators needs to be balanced by taking into account the impact on other indicators and other aspects of society. For example, there is a danger that increasing the value of a degree may result in lower salaries for the uneducated (S11). Therefore, using QSM to assess and explore the development of educational systems contains some interesting dynamics. We will continue to explore this issue in the following discussion.

Multidisciplinary analysis

Policy influences in countries

When we revisit the above policies, many contradictions become apparent. Immediate success is not necessarily desirable in the development of higher education. We cannot offer arbitrary targets that pose significant risks. Economics tells us that if we ask the market to pay higher-education graduates above the market’s equilibrium position, there will be a severe market excess (Heyne et al., 2009), as shown in Fig. 5. Allowing the QSM’s indicators to grow could be detrimental to the quality of society and would potentially create a more significant economic and social crisis.

Fig. 5: The impact of graduates’ salaries on the market equilibrium position.
figure 5

This figure shows the market to pay higher-education graduates above the market’s equilibrium position, there will be a severe market excess. P price, Q quantity, S supply, D demand.

Full size image

There is far more to management practice in higher education than simply using models for assessment. As our research has shown, the quality and sustainability of a higher education system involve many dimensions, and the indicators interact with each other and are even linked to other social issues in the region. For example, increasing the average number of self-citations (S13) can increase the total number of citations (S12). However, the two logics are reversed in the QSM, and increasing the average of self-citations (S13) decreases the country’s QSM evaluation index. Such academic speculation does not reflect well in the QSM. Another example occurs when, due to market excess, educators must reduce the number of higher education graduates to solve the job market problem. Should this happen, financial investment in higher education and the number of international students will decline across the board, and the recognition of higher education in society will also decline rapidly, which is not a desirable outcome.

However, although its practice is full of complex variables, the QSM can be a reliable partner in helping policymakers to understand the current situation, test the current state of development of the education system and suggest areas where improvements can be made. Through the collation of statistics and the monitoring of changes, it can help managers identify problems and generate ideas for development. For example, the QSM does not directly tell us whether a country is over-resourced in higher education, but when we analyse the results for that country, we find that the Higher Degree Value is declining and the QSM–SD is falling. Changes in these indicators can be used alongside other statistics, such as unemployment rates, to help managers understand the current situation and to support future decisions.

Accordingly, the QSM can also be used as a tool for monitoring the effectiveness of policies. Once a national government has enacted a particular higher education-related policy initiative, the QSM can be used to track the impact of the policy continuously throughout the implementation process. Such monitoring and evaluation can take place over several years, as changes in higher education are often slow. Not only does it take time for policies to be implemented, but ultimately, their manifestation in changes in the quality and sustainability of higher education also takes time to appear, especially as the impact between the aspects and indicators of educational management also takes time to manifest. The changes in the QSM indicators in Japan following the enactment of the National University Reform Plan are a good example.

Higher education and society

In addition to developing higher education based on a multi-subject collaborative framework, it is important to be guided by the right mindset. Educators who focus on optimising individual indicators do not necessarily improve the overall quality and development potential of higher education but may instead cause a decline in quality, according to our QSM.

Including higher education within the total national capacity, the development system is a fundamental step in development. The state needs to improve the internal governance capacity of higher education and external recognition holistically and effectively use higher education as a source of human development progress. These initiatives can promote the endogenous growth of higher education’s quality and sustainable development indicators rather than a single policy doing more harm than good.

Different countries have parts of their higher education systems in which they excel, and there is an opportunity for international exchange in education to play a more significant role in the current development of higher education. Educators from different countries can understand and learn from each other through the exchange and use the development experience of others to support the areas where they are at a disadvantage. In modern society, higher education contributes significantly to the development of the economy and society, undertakes a large part of the world’s research, and is an essential pillar of scientific knowledge (Ojeda-Romano et al., 2021). International exchange in higher education improves the quality of education and indirectly promotes social progress.

Research innovation and future development

The QSM provides higher education managers with a tool by which to understand and monitor the quality and sustainability of education. It can provide a quantitative assessment of the current state of the higher education system in a country or region, identifying potential areas for development, supporting ideas for higher education development and helping practitioners monitor policy effectiveness. The QSM inherits the advantages of both PCA and the entropy weighting method (EWM). When weighing the indicators, we utilised the average weight method by combining PCA with EWM. To some extent, this method supplements the indicator’s horizontal comparison with EWM, but it also simplifies the evaluation dimensions, allowing us to point to the shortcomings within national or regional higher education within a manageable range and then progress towards world-class levels. We fully integrated our model with academic misconduct indicators, thereby introducing a new perspective to explaining higher education’s quality and sustainability, as guided by the title.

The collection of certain statistics is difficult, so there is room for improvement in the selection of indicators. Significantly underdeveloped countries were not chosen as examples of the model. In addition to the difficulty of obtaining data, the higher education base in those countries is inferior. The model would have these countries scoring high on the negative logic questions, most likely outperforming some of the developed countries in total ratings. Therefore, using our model to analyse higher education in underdeveloped countries or regions is challenging. We selected only nine countries when constructing our QSM, which may not adequately cover the extreme and minimal values of higher education quality and sustainability indicators. We also selected our data for training the model from 2018, which may have introduced some modelling errors, even though the cumulative variance rate for PCA was reasonable.

Further development

The current research model has innovatively incorporated academic misconduct and speculative behaviour into assessing the quality of a country or region’s higher education system and has validated the model’s applicability in nine developed countries. However, the QSM has not been applied to developing countries or countries or regions with poor statistics on relevant indicators. Therefore, future development of the study could focus on the following areas.

Future research could optimise the model to be better applied to the assessment activities of developing higher education systems. Many countries around the world, such as China, India, and other Asian countries, are currently in a state of rapid growth or constant change. Higher education data in these countries change yearly, making their QSM results not always accurate and difficult to analyse. Enhancing the generalisability of the model worldwide would allow for a better study of the differences and gaps between different education systems and enable the assessment of the current state of development through a quantitative approach. Such a study will facilitate the realisation of comparisons between more countries, discover differences in higher education systems at different development levels, and find breakthroughs in development.

Research can also explore how to make better use of existing indicators. We used 13 indicators to measure the quality and sustainability of higher education, and realistically, administrators will also count more indicators that are necessary for analysis and are readily available. In particular, the research team’s initiative to count and calculate new indicators will make it possible to assess higher education with higher quality. Further adjustments to existing indicators could achieve similar small states. As a result, the model’s reliability will be improved, and we can obtain more accurate assessment results.

We also need to consider how the QSM can be applied to highly underdeveloped countries to guide the building of their education systems. These countries or regions may not have a structured higher education system, so statistics are missing in large numbers, and it is not easy to clarify how to make them develop more efficiently and healthily.

Conclusion

We analysed higher education data from the World Bank, OECD, and other authorities in nine developed countries. First, we used principal component analysis for dimensionality reduction. We obtained two principal model components of our QSM: [cost of higher education, innovation, reputation, access to higher education, and government guidance] and [sustainability of teaching and research, sustainability of schooling, reputation, and sustainability of policy]. We then assigned weights to the 13 sub-indicators using the entropy value method to build a credible model of higher education sustainability.

Data from nine countries were selected for cross-sectional comparisons, and four were selected for longitudinal comparisons from 2010 to 2016. The analysis found that Australian higher education is highly advantageous in quality: it has a very high tertiary education penetration rate and very few academic dishonesty incidents. The U.K. higher education is advantageous in sustainability: its elite education is distinctive.

To improve the quality and sustainability of higher education, each member of the higher education system can start from the perspective of what it can do. It is a complex system, and many indicators can be used to measure its quality. Thus, all 13 indicators in the QSM can significantly impact the overall quality and sustainability of the higher education system. Strengths in individual elements of the dimensions that make up higher education do not lead to an increase in overall levels. For example, the United States produces more patents per capita than the other countries in this case, but higher education’s overall quality and sustainability are assessed at a more mediocre level.

Similarly, weaknesses in individual indicators do not necessarily indicate poor quality in the education system. Using the number of patents as an example, Australia produces the lowest number of patents per capita of the nine countries, but its higher education system is high quality. In reality, higher education in different countries has its strengths, and comparing them with each other and promoting exchange can facilitate the development of systems across the board.

The construction and development of higher education is a continuous and lengthy process, and balanced and stable development can form a high-quality higher education system and further promote social development. The QSM in this study can help decision-makers in each country or region quantitatively analyse the higher education system’s current situation, find progress breakthroughs, and guide actions.