Abstract
The Bologna Process (BP) promoted a series of wide-ranging reforms of higher education (HE) systems in order to improve the quality of teaching activities across Europe. This paper evaluates the effect of these reforms on the teaching efficiency of Italian universities during the period 2000–2010. We employ bootstrapped data envelopment analysis algorithms to assess teaching efficiency. Then, we examine the convergence of the Italian HE system using several panel data estimators. We find clear evidence that Italian universities have become more efficient over time, consistent with the goals of the BP, but that substantial improvement mainly occurs during the initial period of implementation. Our estimates also show a process of convergence in the performance of the Italian HE system, but we find strong evidence of persistent gaps at both university and regional levels. These empirical findings are robust to an alternative estimator, the empirical strategy, and the employed sample.
Similar content being viewed by others
Notes
There is a considerable literature showing a duality in the Italian socio-economic system between the developed North-Centre and the less-developed South, also in human capital endowments (Di Liberto 2008).
At present, the Bologna Process involves 47 countries.
The Italian HE system has experienced a deep and unsystematic process of reform over the last decades. The reform process began in the late 80s with the introduction of the self-regulation principle in 1989 and of financial autonomy of HEIs in 1993, in line with the general trend of the European university sector system in favor of decentralization. More recently, the so-called Gelmini Reform (Law No. 240/2010) further modified the internal organization of universities.
A slow pace in the implementation of the reform was not a specific feature of Italy. See, for instance, Bolli and Farsi (2015) on the reform in Switzerland.
The studies that focus on research activities use output indexes related to HEIs’ prestige (Johnes and Yu 2008), external resources attracted to research activities (Johnes 2008; Agasisti and Pérez-Esparrells 2010), the number of published works and citations (St. Aubyn et al. 2009), and the number of Ph.D. degrees.
From an output-oriented perspective, efficiency is defined as the ratio of a DMU’s observed output to the maximum output that could be achieved given its input level (Farrell 1957).
We assume an output-oriented model to maximize the outputs that could be produced given the inputs. Moreover, we assume a Shephard (1970) output-oriented distance function, and consequently, efficiency scores assume values between zero and one, that is, the reciprocal of the Farrell (1957) distance function.
Notwithstanding their large use, DEA estimators have received criticism, since they rely on extreme points and could be extremely sensitive to data selection, aggregation, model specification, and data errors (Simar and Wilson 2008). Alternative approaches do exist to provide robust measures of efficiency at extreme data points based on partial frontiers and the resulting partial efficiency scores. A detailed survey of these approaches can be found in Simar and Wilson (2008). See also Wilson (2012) for a discussion on these approaches and for a proposed extension of order-m estimator obtained by Cazals et al. (2002).
This function has two principal advantages: first, it is closely related with technical efficiency; and second, it allows the possibility of specifying a multiple-input, multiple-output technology without price information so being suitable for our models, which includes only data on quantities.
We use a system GMM estimator (SYS-GMM) that accounts for possible endogeneity by using a valid instrument and allow jointly estimating the original level and the first-difference regressions (Blundell and Bond 1998). Another possible estimator is the two-step GMM estimator proposed by Arellano and Bond (1991) that uses lagged values of the variables as instruments in the first-differenced regression. However, these were shown to be weak, particularly for the regression in differences (see Blundell and Bond 1998).
Estimation results are available upon request.
However, in the considered period, new HEIs (in particular, online universities) have been established. The results we present in the following sections are also robust with respect to the full unbalanced sample including all universities that completed at least a first round of degree programs.
As previously stated, the choice of both inputs and outputs strictly depends on the availability of data, and we are perfectly aware that the set of variables we include does not allow us to fully capture quality directly. However, we have included ENR_9, GRAD_R, and GRAD_Q in order to reflect the qualitative aspect of teaching efficiency that has been considered explicitly as a key issue in HE reforms in Europe.
We employed the following degree classification grade (in parenthesis the Italian grades): first (summa cum laude); upper second (106–110); lower second (101–105); third (91–100); and finally (66–90). Then following Johnes (2006) the weights used are respectively: first = 30; upper second = 25; lower second = 20; third = 15; and finally = 10.
The output-oriented approach is usually preferable in this setting because inputs such as enrolled students and personnel are assumed to be fixed exogenously, at least in the short run.
See Jondrow et al. (1982) for further details.
Moreover, time-invariant technology is assumed when estimating intertemporal frontiers.
In particular, Simar and Wilson (2008) observed that due the slow convergence rate of DEA estimators, the consistency of DEA estimates strongly depend on the number of observations and variables included in the model (i.e. the dimensionality space). The dimensionality space implies that, for a given sample, a parsimonious model tends to produce better estimates for the efficient frontier than large model. Moreover, for a given model specification, small sample tends to report higher efficiency rate than large sample.
However, one may argue that a better empirical strategy to test HEIs’ efficiency convergence is to employ a contemporaneous frontiers approach as suggested by Casu and Girardone (2010), since the intertemporal frontier approach does not allow for the identification of year-specific effects and, moreover, requires time invariant assumption on the HEIs production process. For these reasons and to provide robustness to our empirical findings, we also run convergence analysis by using contemporaneous frontier estimates. Results are largely comparable with those reported in the next sections and are not reported to economize on space, but are available from the authors upon request.
Furthermore, we observe that our estimates are comparable to previous findings available in the literature (Agasisti and Dal Bianco 2009), considering that we use a common frontier.
Nevertheless, we formally test whether the frontier globally exerts constant (CRS) or variable (VRS) returns to scale with both the procedures developed by Banker (1996) and by Simar and Wilson (2002) for mod 1. The results of Banker (1996)’s test show that we cannot reject the null hypothesis of CRS at the conventional level of significance. Moreover, the procedure proposed by Simar and Wilson (2002) consists in smoothing the probability distribution of the efficiency estimates under the CRS and VRS formulations of the DEA model. A bootstrap re-sampling method (B = 2000) is implemented to develop a robust p value, which enables us to test whether HEIs operate under CRS or VRS. We cannot reject the null hypothesis of global VRS at 5 % confidence level (p values = 0.0927). This implies that, in what follows, we do not account for the size-effects related to Italian HEIs.
However, mod 3 shows that those effects were probably overcome by a reduction of average degree score.
In Sect. 6, we also employ panel fixed-effect estimators to control for such an effect.
In a different sense, we can look at the implementation of the reform as a natural experiment that enables us to compare the average efficiency of HEIs exposed to the reform (i.e., those having the number of graduates higher than the cut-off), to the control group of the remaining HEIs.
We wish to thank an anonymous reviewer for suggesting this point.
We are aware that the proposed method has received criticism (Cordero et al. 2009). Despite criticisms of the one-stage model, it remains popular and is the standard approach for considering such factors, especially in cases where the impact is recognized but not fully understood (Syrjänen 2004; Harrison et al. 2012). Furthermore, the method is more robust in the case of CRS.
A detailed analysis of the pattern of research and teaching in a comparable sample of HEIs can be found in Guccio et al. (2015).
An extensive literature deals with endogeneity problems with respect to convergence equation and growth models. Consistently with the empirical literature (see for instance, Casu and Girardone 2010; Maghyereh and Awartani 2012; Ayadi and Mouley 2013), we use a system-GMM estimator (Arellano and Bover 1995; Blundell and Bond 1998) to deal with this issue. Our results are consistent with those obtained through the GMM estimator (Arellano and Bond 1991) and related tables are available upon request.
Results hold for mod 2 and mod 3, and related tables are available upon request.
Model selection tests have been provided in Table 18.
To provide robustness of our convergence results, we perform several checks (related tables are available upon request). We estimate Eqs. (2), (3) and (5): (1) by using SFA scores; (2) by using bias-corrected efficiency scores (Simar and Wilson 1998); (3) by using scores estimated through contemporaneous frontiers to relax the assumption of time-invariant technology of production; (4) by employing the semiparametric estimator proposed by Simar and Wilson (2007); (5) by estimating models on the subsample of universities that have been established before the year 1997 to assess whether the case selection in terms of HEIs plays a role. Our findings are robust with respect to all the above mentioned checks.
References
Abbot M, Doucoliagos C (2003) The efficiency of Australian universities: a data envelopment analysis. Econ Educ Rev 22:89–97
Agasisti T, Bolli T (2013) The impact of the Bologna Reform on the productivity of Swiss universities. High Educ Q 67:374–397
Agasisti T, Catalano G (2007) Efficienza ed equità nel sistema universitario italiano: gli effetti di quindici anni di riforme. Paper presented at the XIX Conference of the Italian Association of Public Economics (SIEP)
Agasisti T, Dal Bianco A (2006) Data envelopment analysis to the Italian university system: theoretical issues and policy implications. Int J Bus Perform Manag 8:344–367
Agasisti T, Dal Bianco A (2009) Reforming the university sector: effects on teaching efficiency—evidence from Italy. High Educ 57:477–498
Agasisti T, Johnes G (2009) Beyond frontiers: comparing the efficiency of higher education decision-making units across more than one country. Educ Econ 17:59–79
Agasisti T, Pérez-Esparrells C (2010) Comparing efficiency in a cross-country perspective: the case of Italian and Spanish state universities. High Educ 59:85–103
Agasisti T, Pohl C (2012) Comparing German and Italian public universities: convergence or divergence in the higher education landscape? Manag Decis Econ 33:71–85
Aghion P, Dewatripont M, Hoxby C, Mas-Colell A, Sapir A (2010) The governance and performance of universities: evidence from Europe and the US. Econ Policy 25:7–59
Aigner DJ, Lovell CAK, Schmidt P (1977) Formulation and estimation of stochastic frontier production functions. J Econ 6:21–37
Arellano M, Bond S (1991) Some tests of specification for panel data: Monte Carlo evidence and an application to employment equations. Rev Econ Stud 58:277–297
Arellano M, Bover O (1995) Another look at the instrumental variable estimation of error-components models. J Econ 68:29–51
Athanassopoulos A, Shale E (1997) Assessing the comparative efficiency of higher education institutions in the UK by means of data envelopment analysis. Educ Econ 5:117–134
Ayadi R, Mouley S (2013) Monetary policies, banking systems, regulatory convergence, efficiency and growth in the mediterranean [online]. Available at http://www.palgraveconnect.com/pc/doifinder/10.1057/9781137003485.0001. Accessed 12 Oct 2015
Ayadi R, Boussemart J, Leleu H, Saidane D (2013) Mergers and Acquisitions in European banking higher productivity or better synergy among business lines? J Prod Anal 39:165–175
Banker RD (1996) Hypothesis tests using data envelopment analysis. J Prod Anal 7:139–159
Banker RD, Morey RC (1986) Efficiency analysis for exogenously fixed inputs and outputs. Oper Res Int J 34(4):513–521
Banker RD, Natarajan R (2008) Evaluating contextual variables affecting productivity using data envelopment analysis. Oper Res Int J 56(1):48–58
Banker RD, Charnes A, Cooper WW (1984) Some models for estimating technical and scale inefficiencies in data envelopment analysis. Manag Sci 30:1078–1092
Barro RJ, Sala-i-Martin X (1991) Convergence across states and regions. Brook Pap Econ Act 1:107–182
Barro RJ, Sala-i-Martin X (1992) Convergence. J Polit Econ 100:223–251
Beasley J (1990) Comparing university departments. Omega 18:171–183
Beasley J (1995) Determining teaching and research efficiencies. J Oper Res Soc 46:441–452
Bergantino AS, Capozza C, Porcelli F (2013) Measuring the efficiency of the Italian university system: the role of market structure. In: A two step DEA analysis at Faculty Level, paper presented at the XXV Conference of the Italian Association of Public Economics (SIEP)
Blundell R, Bond S (1998) Initial conditions and moment restrictions in dynamic panel data models. J Econ 87:115–143
Bogetoft P, Otto L (2011) Benchmarking with DEA, SFA, and R. Springer, New York
Bolli T, Farsi M (2015) The dynamics of productivity in Swiss universities. J Prod Anal 44(1):21–38
Cappellari L, Lucifora C (2009) The ‘Bologna process’ and college enrolment decisions. Labour Econ 16:638–647
Cardoso AR, Portela M, Sá C, Alexandre F (2008) Demand for higher education programs: the impact of the Bologna process. CESifo Econ Stud 54:229–247
Casu B, Girardone C (2010) Integration and efficiency convergence in EU banking markets. Omega 38:260–267
Cazals C, Florens JP, Simar L (2002) Nonparametric frontier estimation: a robust approach. J Econ 106:1–25
Charnes A, Cooper WW, Rhodes E (1978) Measuring the efficiency of decision making units. Eur J Oper Res 2:429–444
Cooper WW, Seiford LM, Tone K (2007) Data envelopment analysis: a comprehensive text with models, applications, references, and DEA-solver software. Kluwer, Boston
Cordero JM, Pedraja F, Santín D (2009) Alternative approaches to include exogenous variables in DEA measures: a comparison using Monte Carlo. Comput Oper Res 36(10):2699–2706
Crosier D, Horvath A, Kerpanova V, Kocanova D, Parveva T, Dalferth S (2012) The European higher education area in 2012: Bologna process implementation report. In: Education, Audiovisual and Culture Executive Agency. European Commission. Available from EU Bookshop
Debreu G (1951) The coefficient of resource utilization. Econ J Econ Soc 19:273–292
Di Liberto A (2008) Education and Italian regional development. Econ Educ Rev 27:94–107
Di Pietro G (2011) The Bologna Process and widening participation in university education: new evidence from Italy. Empirica 39:357–374
Dundar H, Lewis D (1995) Departmental productivity in American universities: economies of scale and scope. Econ Educ Rev 14:119–144
Farrell M (1957) The measurement of productive efficiency. J R Stat Soc Ser A (Gen) 120:253–290
Flegg AT, Allen DO, Field K, Thurlow TW (2004) Measuring the efficiency of British universities: a multi-period data envelopment analysis. Educ Econ 12:231–249
Fried HO, Knox Lovell CA, Schmidt SS (2008) The measurement of productive efficiency and productivity growth. Oxford University Press, New York
Guccio C, Martorana MF, Mazza I, Monaco L (2015) An assessment of the relationship between research and teaching performances in a static and dynamic perspective. Paper presented at the XXVII annual Conference of the Italian Society of Public Economics (SIEP), Ferrara, 2015. https://editorialexpress.com/cgi-bin/conference/download.cgi?db_name=SIEP2015&paper_id=47. Accessed 12 Oct 2015
Harrison J, Rouse P, Armstrong J (2012) Categorical and continuous non-discretionary variables in data envelopment analysis: a comparison of two single-stage models. J Prod Anal 37(3):261–276
Huisman J, Van der Wende M (2004) The EU and Bologna: are supra-and international initiatives threatening domestic agendas? Eur J Educ 39:349–357
Johnes J (2006) Data envelopment analysis and its applications to the measurement of efficiency in higher education. Econ Educ Rev 25:273–288
Johnes J (2008) Efficiency and productivity change in the english higher education sector from 1996/97 to 2004/5. Manch Sch 76:653–674
Johnes J, Johnes G (1995) Research funding and performance in UK university departments of economics: a frontier analysis. Econ Educ Rev 14:301–314
Johnes J, Yu L (2008) Measuring the research performance of Chinese higher education institutions using data envelopment analysis. China Econ Rev 19:679–696
Jondrow J, Knox Lovell CA, Materov IS, Schmidt P (1982) On the estimation of technical inefficiency in the stochastic frontier production function model. J Econ 19:233–238
Joumady O, Ris C (2005) Performance in European higher education: a non-parametric production frontier approach. Educ Econ 13:189–205
Kantabutra S, Tang J (2010) Efficiency analysis of public universities in Thailand. Tert Educ Manag 16:15–33
Kempkes G, Pohl C (2010) The efficiency of German universities—some evidence from nonparametric and parametric methods. Appl Econ 42:2063–2079
Koopmans T (1951) An analysis of production as an efficient combination of activities. In: Koopmans TC (ed) Activity analysis of production and allocation, Cowles Commission for Research in Economics, Monograph 13. Wiley, New York, pp 33–37
Kuah C, Wong K (2011) Efficiency assessment of universities through data envelopment analysis. Proc Comput Sci 3:499–506
Lambert R, Butler N (2006) The future of European universities: Renaissance or decay?. Centre for European Reform, London
Madden G, Savage S, Kemp S (1997) Measuring public sector efficiency: a study of economics departments at Australian universities. Educ Econ 5:153–168
Maghyereh AI, Awartani B (2012) Financial integration of GCC banking markets: a non-parametric bootstrap DEA estimation approach. Res Int Bus Finance 26(2):181–195
Meeusen W, van den Broeck J (1977) Efficiency estimation from Cobb–Douglas production function with composed error. Int Econ Rev 8:435–444
Monaco L (2012) Measuring Italian university efficiency: a non-parametric approach, MPRA Paper No. 37949
Olivares M, Wetzel H (2011) Competing in the higher education market: empirical evidence for economies of scale and scope in German higher education institutions, University of Lüneburg Working Paper Series in Economics No. 223
Sav GT (2012) Productivity, efficiency, and managerial performance regress and gains in United States universities: a data envelopment analysis. Adv Manag Appl Econ 2:13–32
Sciulli D, Signorelli M (2011) University-to-work transitions: an empirical analysis on perugia graduates. Eur J High Educ 1:39–65
Sheather SJ, Jones MC (1991) A reliable data-based bandwidth selection method for kernel density estimation. J R Stat Soc Ser B (Methodol) 53:683–690
Shephard RW (1970) Theory of cost and production function. Princeton University Press, Princeton, NJ
Simar L, Wilson PW (1998) Sensitivity analysis of efficiency scores: how to bootstrap in nonparametric frontier models. Manag Sci 44:49–61
Simar L, Wilson PW (2000) Statistical inference in nonparametric frontier models: the state of the art. J Prod Anal 13:49–78
Simar L, Wilson PW (2002) Nonparametric tests of returns to scale. Eur J Oper Res 139:115–132
Simar L, Wilson PW (2007) Estimation and inference in two-stage, semi-parametric models of production processes. J Econ 136:31–64
Simar L, Wilson PW (2008) Statistical inference in nonparametric frontier models: recent developments and perspectives. In: Fried HO, Knox Lovell CA, Schmidt SS (eds) The measurement of productive efficiency and productivity growth. Oxford University Press, New York, pp 421–521
St. Aubyn M, Pina A, Garcia F, Pais J (2009) Study on the efficiency and effectiveness of public spending on tertiary education, Directorate General Economic and Monetary Affairs (DG ECFIN), European Commission Economic Papers, No. 390
Sursock A, Smidt H (2010) Trends 2010: a decade of change in European Higher Education. European University Association, Brussels
Syrjänen MJ (2004) Non-discretionary and discretionary factors and scale in data envelopment analysis. Eur J Oper Res 158:20–30
Tomkins C, Green R (1988) An experiment in the use of data envelopment analysis for evaluating the efficiency of UK university departments of accounting. Financ Account Manag 4:147–164
Wand MP, Jones MC (1995) Kernel smoothing. Chapman and Hall Press, London
Weill L (2009) Convergence in banking efficiency across European countries. J Int Financ Mark Inst Money 19:818–833
Wilson PW (2008) FEAR 1.0: a software package for frontier efficiency analysis with R. Socio-Econ Plan Sci 42:247–254
Wilson PW (2012) Asymptotic properties of some non-parametric hyperbolic efficiency estimators. In: Van Keilegom I, Wilson PW (eds) Exploring research frontiers in contemporary statistics and econometrics. Springer, Berlin, pp 115–150
Witte J, Huisman J, Purser L (2009) European higher education reforms in the context of the Bologna process: How did we get here, where are we and where are we going? In: OECD (ed) Higher education to 2030 (globalisation), vol 2. OECD Publishers, Paris, pp 205–229
Wolszczak-Derlacz J, Parteka A (2011) Efficiency of European public higher education institutions: a two-stage multicountry approach. Scientometrics 89:887–917
Zhang T, Matthews K (2012) Efficiency convergence properties of Indonesian banks 1992–2007. Appl Financ Econ 22:1465–1478
Acknowledgments
I would like to thank Prof. Victor Podinovski, two anonymous reviewers, and the associate editor for their insightful and constructive comments. We also thank Tommaso Agasisti, Isidoro Mazza, and the participants of the XXV annual Conference of the Italian Society of Public Economics (SIEP)—Pavia 2013, and of the international Workshop “New Issues of International and Public Economics”—Catania 2014, for their helpful suggestions on earlier versions. Any remaining errors are solely the responsibility of the authors.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Guccio, C., Martorana, M.F. & Monaco, L. Evaluating the impact of the Bologna Process on the efficiency convergence of Italian universities: a non-parametric frontier approach. J Prod Anal 45, 275–298 (2016). https://doi.org/10.1007/s11123-015-0459-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11123-015-0459-6