Skip to main content

Advertisement

SpringerLink
Log in

Measuring the efficiency of European education systems by combining Data Envelopment Analysis and Multiple-Criteria Evaluation

  • Published:
Journal of Productivity Analysis Aims and scope Submit manuscript

Abstract

Education is considered an important factor of economic growth, employment and social inclusion. However, the economic crisis has put the need to achieve educational goals in the most efficient way ever more to the fore. The main objective of this paper is to assess the spending efficiency of European compulsory educational systems, creating a ranking of countries based on the efficiency scores of their systems using a number of standard variables from the literature. To this end, we also present a methodological innovation that combines Data Envelopment Analysis (DEA) with discrete Multiple Criteria Evaluation (MCE), two methods that we consider complementary if used for providing a performance analysis. Moreover, both methods identify a set of common variables which are associated with higher levels of efficiency in educational systems (e.g. some characteristics of teachers, the stock of adults’ human capital and lower expenditures per student). The results show that findings using DEA are largely confirmed by MCE.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. The NAIADE method can be considered particularly useful for efficiency analyses in the field of education for four main reasons:

    1.it has been explicitly designed for public policy applications;

    2.it is flexible, as it can deal with different source of information concerning the criterion scores;

    3.compensability can be fully controlled;

    4.it can also be used for benchmarking exercises.

  2. Whenever possible, preference has been given to Eurostat data because data are usually available for more countries. We have chosen 2012 as our reference year, as most series of data are only available up to 2013. As such, the exercise cannot be performed for recent years. This is because the most relevant output variable, i.e. PISA scores, is only available for 2012 (not 2013). Exceptions include data concerning teachers (age and salaries) which is only available for 2013. Where possible, we calculate an average over the period 2010 to 2012 in order to avoid potential bias in one given year. In the case of expenditure data, we were not able to calculate averages but instead, we chose to only include the year 2012.

  3. In many cases, data are separately available for primary, lower and upper secondary education. For this reason, depending on the individual variable specification, we have summarised the various education levels or averaged them in order to get a measure for both primary and secondary education together. In the latter case, we used weightings in order to assign the appropriate relative share for each education stage (for example, we weighted by the number of pupils at each level) – see Supplementary File 1 for the specific weighting schemes applied in each case.

  4. We have not dealt with the problem of defining and interpreting plausible values for these scores (following the technical instructions provided by OECD (2014b) and commented on by Wu (2005)) as we directly used the country averages as officially calculated and reported by OECD in their institutional reports.

  5. We rely on a simple correlation analysis between efficiency scores and the contextual factors that describe the characteristics of educational systems. We are well aware of the drawbacks of two-stage procedures as pointed out by Simar and Wilson (2011). Indeed, the problem of the two-stage approach is that it relies on a separability condition between the input-output space and the space of the external factors, assuming that these factors have no influence on the attainable set, which may not hold in some situations. For this reason, we limit ourselves to simple descriptive correlations and do not perform any second-stage regression of the kind criticised under the methodological indications recalled here.

  6. Note that we cannot compute efficiency scores for all countries, as we have some missing values in the data on both the input and output side. However, the missing values affect the same countries in each model (for example, both PISA maths and reading scores are not available for Malta), so that each time the same countries are dropped in the calculation of the efficiency scores. In other words, we cannot compute scores for Croatia, Denmark, Greece and Malta in all models.

References

  • Agasisti T (2011) Performances and spending efficiency in higher education: a European comparison through non‐parametric approaches. Educ Econ 19(2):199–224

    Article  Google Scholar 

  • Agasisti T, Hippe R, Munda G (2017) Efficiency of investment in compulsory education: empirical analyses in Europe; EUR 28607 EN. : Publications Office of the European Union, Luxembourg (Luxembourg), p JRC106678. https://doi.org/10.2760/975369

  • Alexander WRJ, Haug AA, Jaforullah M (2010) A two-stage double-bootstrap data envelopment analysis of efficiency differences of New Zealand secondary schools. J Prod Anal 34(2):99–110

    Article  Google Scholar 

  • Antunes CH, Alves MJ, Clímaco J (2016) Multi-objective Linear and Integer Programming. Springer, New York, NY

    Book  Google Scholar 

  • Arrow KJ, Raynaud H (1986) Social choice and multicriterion decision making. M.I.T. Press, Cambridge

    Google Scholar 

  • Barro R. J (2001) Human capital and growth. American Economic Review 91(2):12–17

    Article  Google Scholar 

  • Benhabib J, Spiegel MM (1994) The role of human capital in economic development evidence from aggregate cross-country data. J Monetary Econ 34(2):143–173

    Article  Google Scholar 

  • Bouyssou D (1999) Using DEA as a tool for MCDM: some remarks. J Oper Res Soc 50(9):974–978

    Article  Google Scholar 

  • Cazals C, Florens JP, Simar L (2002) Nonparametric frontier estimation: a robust approach. J Econ 106(1):1–25

    Article  Google Scholar 

  • Charnes A, Cooper WW, Rhodes E (1978) Measuring the efficiency of decision making units. Eur J Oper Res 2(6):429–444

    Article  Google Scholar 

  • Charnes A, Cooper WW, Lewin AY, Seiford LM (eds.) (2013) Data envelopment analysis: theory, methodology, and applications. Springer Science & Business Media, New York

  • Coelli TJ, Rao DSP, O’Donnell, CJ, Battese GE (2005) An introduction to efficiency and productivity analysis. Springer Science & Business Media, New York

  • Coleman JS, Campbell EQ, Hobson CJ, McPartland J, Mood AM, Weinfeld FD, York R (1966) Equality of educational opportunity. U.S. Government Printing Office, Washington, DC, 1066–5684

  • Cooper WW, Seiford LM, Zhu J (Eds.) (2011) Handbook on data envelopment analysis (Vol. 164). Springer Science & Business Media New York

  • Cordero-Ferrera JM, Pedraja-Chaparro F, Salinas-Jiménez J (2008) Measuring efficiency in education: an analysis of different approaches for incorporating non-discretionary inputs. Appl Econ 40(10):1323–1339

    Article  Google Scholar 

  • Cunha M, Rocha V (2012) On the efficiency of Public higher education institutions in Portugal: an exploratory study. FEP Economics and Management, Working Paper

  • Daraio C, Simar L (2007) Conditional nonparametric frontier models for convex and nonconvex technologies: a unifying approach. J Prod Anal 28(1–2):13–32

    Article  Google Scholar 

  • Darling-Hammond L (2000) Teacher quality and student achievement. Education Policy Analysis Archives 8:1

    Article  Google Scholar 

  • De Witte K, López-Torres L (2017) Efficiency in education: a review of literature and a way forward. J Oper Res Soc 68(4):339–363

    Article  Google Scholar 

  • Dill DD, Soo M (2005) Academic quality, league tables, and public policy: a cross-national analysis of university ranking systems. Higher Education 49(4):495–533

    Article  Google Scholar 

  • Doyle J, Green R (1993) Data envelopment analysis and multiple criteria decision making. Omega 21(6):713–715

    Article  Google Scholar 

  • Dronkers J, Robert P (2008) Differences in scholastic achievement of public, private government-dependent, and private independent schools: a cross-national analysis. Educ Policy 22(4):541–577

    Article  Google Scholar 

  • Emrouznejad A, Yang GL (2018) A survey and analysis of the first 40 years of scholarly literature in DEA: 1978–2016. Socio-Economic Planning Sciences 61:4–8

    Article  Google Scholar 

  • Färe R, Karagiannis G (2017) The denominator rule for share-weighting aggregation. Eur J Oper Res 260(3):1175–1180

    Article  Google Scholar 

  • Figueira J, Greco S, Ehrgott M (eds.) (2016) Multiple-criteria decision analysis. State of the art surveys. Springer International Series in Operations Research and Management Science, New York, NY

    Google Scholar 

  • Fried HO, Lovell CK, Schmidt SS (Eds.) (2008) The measurement of productive efficiency and productivity growth. Oxford University Press, Oxford, UK

  • Golany B, Tamir E (1995) Evaluating efficiency-effectiveness-equality trade-offs: a data envelopment analysis approach. Manag Sci 41(7):1172–1184

    Article  Google Scholar 

  • Greenwald R, Hedges LV, Laine RD (1996) The effect of school resources on student achievement. Rev Educ Res 66(3):361–396

    Article  Google Scholar 

  • Grosskopf S, Moutray C (2001) Evaluating performance in Chicago public high schools in the wake of decentralization. Econ Educ Rev 20(1):1–14

    Article  Google Scholar 

  • Grosskopf S, Hayes KJ, Taylor LL (2014) Efficiency in education: research and implications. Appl Econ Persp Policy 36(2):175–210

    Article  Google Scholar 

  • Guskey TR (2007) Multiple sources of evidence: an analysis of stakeholders’ perceptions of various indicators of student learning. Educ Measurement: Issues and Practice 26(1):19–27

    Article  Google Scholar 

  • Haelermans C, De Witte K (2012) The role of innovations in secondary school performance–Evidence from a conditional efficiency model. Eur J Oper Res 223(2):541–549

    Article  Google Scholar 

  • Hanushek EA, Luque JA (2003) Efficiency and equity in schools around the world. Econ Educ Rev 22(5):481–502

    Article  Google Scholar 

  • Hanushek EA, Woessmann L (2008) The role of cognitive skills in economic development. J Econ Lit 46(3):607–668

    Article  Google Scholar 

  • Hanushek EA, Woessmann, L (2010) The economics of international differences in educational achievement (No. w15949). National Bureau of Economic Research

  • Hanushek EA, Woessmann L (2012) Do better schools lead to more growth? Cognitive skills, economic outcomes, and causation. J Econ Growth 17(4):267–321

    Article  Google Scholar 

  • Haveman R, Wolfe B (1995) A review of methods and findings. J Econ Lit 33(4):1829–1878

    Google Scholar 

  • Ho W, Dey PK, Higson HE (2006) Multiple criteria decision-making techniques in higher education. Int J Educ Manag 20(5):319–337

    Article  Google Scholar 

  • Johnes J (2004) Efficiency measurement. International Handbook on the Economics of Education, chapter 16

  • Joro T, Pekka K, Jyrki W (1998) Structural comparison of data envelopment analysis and multiple objective linear programming. Manag Sci 44(7):962–970

    Article  Google Scholar 

  • Keeney R, Raiffa H (1976) Decisions with multiple objectives: preferences and value trade-offs. Wiley, New York, NY

    Google Scholar 

  • Li X-B, Reeves GR (1999) A multiple criteria approach to data envelopment analysis. Eur J Oper Res 115(3):507–517

    Article  Google Scholar 

  • Liu JS, Lu LY, Lu WM, Lin BJ (2013) A survey of DEA applications. Omega 41(5):893–902

    Article  Google Scholar 

  • Madlener R, Antunes CH, Dias LC (2009) Assessing the performance of biogas plants with multi-criteria and data envelopment analysis. Eur J Oper Res 197(3):1084–1094

    Article  Google Scholar 

  • Malen B, Knapp M (1997) Rethinking the multiple perspectives approach to education policy analysis: implications for policy-practice connections. J Educ Policy 12(5):419–445

    Article  Google Scholar 

  • Mancebón MJ, Calero J, Choi Á, Ximénez-de-Embún DP(2012) The efficiency of public and publicly subsidized high schools in Spain: Evidence from PISA-2006 J Opera Res Soc 63(11):1516–1533

    Article  Google Scholar 

  • Moore, M. H. (1995). Creating public value: Strategic management in government. Harvard Business University Press, Cambridge, MA

  • Munda G (1995) Multicriteria evaluation in a fuzzy environment. Physica-Verlag, Contributions to Economics Series, Heidelberg

    Book  Google Scholar 

  • Munda G (2008) Social multi-criteria evaluation for a sustainable economy. Springer, Heidelberg, New York, NY

    Book  Google Scholar 

  • Munda G (2012) Intensity of preference and related uncertainty in non-compensatory aggregation rules. Theory and Decision 73(4):649–669

    Article  Google Scholar 

  • Nikel J, Lowe J (2010) Talking of fabric: A multi‐dimensional model of quality in education. Compare 40(5):589–605

    Article  Google Scholar 

  • Nijkamp P, Rietveld P, Voogd H (1990) Multicriteria Evaluation in Physical Planning. North-Holland, Amsterdam

    Google Scholar 

  • OECD (2014a) PISA 2012 Results in focus. What 15-year-olds know and what they can do with what they know. OECD, Paris

    Google Scholar 

  • OECD (2014b) PISA 2012 Technical report. OECD, Paris

    Google Scholar 

  • Perry L, McConney A (2010) Does the SES of the school matter? An examination of socioeconomic status and student achievement using PISA 2003. The Teachers College Record 112(4):7–8

    Google Scholar 

  • Pritchett L, Filmer D (1999) What education production functions really show: a positive theory of education expenditures. Econ Educ Rev 18(2):223–239

    Article  Google Scholar 

  • Romer PM (1990). Human capital and growth: theory and evidence. In Carnegie-Rochester Conference Series on Public Policy (Vol. 32, pp. 251–286). North-Holland

  • Rossell CH (1993) Using multiple criteria to evaluate public policies: The case of school desegregation. American Politics Quarterly 21(2):155–184

    Article  Google Scholar 

  • Roy B (1996) Multicriteria methodology for decision analysis. Kluwer, Dordrecht

    Book  Google Scholar 

  • Sacerdote B (2011) Peer effects in education: How might they work, how big are they and how much do we know thus far? Handbook of the Economics of Education 3:249–277

    Article  Google Scholar 

  • Sarrico CS, Rosa MJ, Coelho IP (2010) The performance of Portuguese secondary schools: an exploratory study. Quality Assurance Educ 18(4):286–303

    Article  Google Scholar 

  • Simar L, Wilson PW (2000) A general methodology for bootstrapping in non-parametric frontier models. J Appl Stat 27(6):779–802

    Article  Google Scholar 

  • Simar L, Wilson PW (2011) Two-stage DEA: caveat emptor. J Prod Anal 36(2):205

    Article  Google Scholar 

  • Steuer R (1986) Multiple criteria optimization; theory, computation, and application, Wiley Series in Probability and Mathematical Statistics – Applied, Wiley

  • Stufflebeam D (2001) Evaluation models. New Directions for Evaluation Issue 89:7–98

    Article  Google Scholar 

  • Sutherland D, Price R, Gonand F (2010) Improving public spending efficiency in primary and secondary education. OECD J: Econ Stud 2009(1):1–30

    Google Scholar 

  • Tzeng GH, Cheng-Hsin C, Chung-Wei L (2007) Evaluating intertwined effects in e-learning programs: a novel hybrid MCDM model based on factor analysis and DEMATEL. Expert Systems with Applications 32:1028–1044

    Article  Google Scholar 

  • Walker WE (2009) Does the best practice of rational-style model-based policy analysis already include ethical considerations? Omega 37(6):1051–1062

    Article  Google Scholar 

  • Worthington AC (2001) An empirical survey of frontier efficiency measurement techniques in education. Educ Econ 9(3):245–268

    Article  Google Scholar 

  • Wu M (2005) The role of plausible values in large-scale surveys. Studies in Educational. Evaluation 31(2-3):114–128

    Google Scholar 

  • Zhu, J (ed.) (2015) Data envelopment analysis: A handbook of models and methods (Vol. 221). Springer, New York

Download references

Acknowledgements

This article builds on previous work carried out for the JRC’s Centre for Research on Education and Lifelong Learning (CRELL). The views expressed are purely those of the authors and may not in any circumstances be regarded as stating an official position of the European Commission. We are extremely grateful to the Editor and to two anonymous reviewers for their detailed, challenging and precise comments/suggestions which substantially helped us improve a previous draft of the paper.

Author information

Authors and Affiliations

  1. Politecnico di Milano School of Management, Department of Management, Economics and Industrial Engineering, Campus Bovisa - via Lambruschini 4b, Milan, IT, 20156, Italy

    Tommaso Agasisti

  2. European Commission, Joint Research Centre (JRC), Ispra, Seville, Italy

    Giuseppe Munda & Ralph Hippe

Authors
  1. Tommaso Agasisti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giuseppe Munda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ralph Hippe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tommaso Agasisti.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Agasisti, T., Munda, G. & Hippe, R. Measuring the efficiency of European education systems by combining Data Envelopment Analysis and Multiple-Criteria Evaluation. J Prod Anal 51, 105–124 (2019). https://doi.org/10.1007/s11123-019-00549-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11123-019-00549-6

Keywords

JEL Classification

Search

Navigation