Journal of Productivity Analysis

, Volume 47, Issue 1, pp 83–101 | Cite as

An international comparison of educational systems: a temporal analysis in presence of bad outputs

  • Víctor Giménez
  • Claudio Thieme
  • Diego Prior
  • Emili Tortosa-Ausina


This study uses the global non-radial Malmquist index to measure performance change in the educational systems of 29 countries/economies participating in PISA 2003 and 2012 for students at age 15 in the disciplines of mathematics and reading. This methodology is particularly appropriate both for its desirable properties as well as its suitability for the educational context. Results indicate a positive evolution in educational systems’ performance during this period. This improvement is mainly due a positive efficiency change, which offsets the negative technological change observed. Nevertheless, a deeper scrutiny at the country level shows that results varied remarkably among them.


Education Efficiency Global non-radial Malmquist index PISA 

JEL Classifications

C61 H52 I21 



Claudio Thieme and Emili Tortosa-Ausina thank FONDECYT (National Fund of Scientific and Technological Development, grant #1121164 and #1151313) for generous financial support. Víctor Giménez, Diego Prior and Emili Tortosa-Ausina acknowledge the financial support of the Ministerio de Economía y Competitividad (ECO2013-44115-P and ECO2014-55221-P). Emili Tortosa-Ausina also acknowledges the financial support of Generalitat Valenciana (PROMETEOII/2014/046) and Universitat Jaume I (P1.1B2014-17). All four authors are grateful to the Associate Editor and two anonymous referees, whose comments contributed to an overall improvement of the paper. The usual disclaimer applies.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. Afonso A, St. Aubyn M (2005) Non-parametric approaches to education and health effciency in OECD countries. J Appl Econ 8(2):227–246Google Scholar
  2. Afonso A, St. Aubyn M (2006) Cross-country effciency of secondary education provision: a semi-parametric analysis with non-discretionary inputs. Econ Model 23(3):476–491CrossRefGoogle Scholar
  3. Agasisti T (2014) The efficiency of public spending on education: an empirical comparison of EU countries. Eur J Educ 49(4):543–557CrossRefGoogle Scholar
  4. Agasisti T and Zoido P (2015). The efficiency of secondary schools in an international perspective: preliminary results from PISA 2012. OECD Education Working Papers 117, OECD, Paris.Google Scholar
  5. Aparicio J, Crespo-Cebada E, Pedraja-Chaparro F, Santín D (2016a) Comparing school ownership performance using a pseudo-panel database: a Malmquist-type index approach. Eur J Oper Res 256(2):533–542.Google Scholar
  6. Aparicio J, Pastor JT, Vidal F (2016b) The directional distance function and the translation invariance property. Omega 58:1–3CrossRefGoogle Scholar
  7. Aristovnik A, Obadić A (2014) Measuring relative efficiency of secondary education in selected EU and OECD countries: the case of Slovenia and Croatia. TEDE 20(3):419–433CrossRefGoogle Scholar
  8. Badunenko O, Romero-Ávila D (2013) Financial development and the sources of growth and convergence. Int Econ Rev 54(2):629–663CrossRefGoogle Scholar
  9. Balk BM (2001) Scale efficiency and productivity change. J Prod Anal 15(3):159–183CrossRefGoogle Scholar
  10. Brown, G, Micklewright, J, Schnepf, SV, and Waldmann, R (2007). International surveys of educational achievement: how robust are the findings? J R Stat Soc A 170(3):623–646Google Scholar
  11. Carlson D (2001) Focusing state educational accountability systems: four methods for judging school quality and progress. Technical report, Center for Assessment (NCIEA), Dover, NH.Google Scholar
  12. Caves DW, Christensen LR, Diewert WE (1982) The economic theory of index numbers and the measurement of input, output, and productivity. Econometrica 50(6):1393–1414CrossRefGoogle Scholar
  13. Chung YH, Färe R, Grosskopf S (1997) Productivity and undesirable outputs: a directional distance function approach. J Environ Manage 51:229–240CrossRefGoogle Scholar
  14. Clements B (2002) How efficient is education spending in Europe? Eur Rev Econ Finance 1(1):3–26Google Scholar
  15. Cooper, WW, Seiford, LM, and Tone, K (2007). Data envelopment analysis: a comprehensive text with models, applications, references and DEA-solver software. Springer Science & Business Media, New YorkGoogle Scholar
  16. Cordero JM, Santín D, Simancas R (2017) Assessing European primary school performance through a conditional nonparametric model. J Oper Res Soc in press.Google Scholar
  17. Denvir B, Brown M (1986) Understanding of number concepts in low attaining 7–9 year olds: Part I. development of descriptive framework and diagnostic instrument. Educ Stud Math 17(1):15–36CrossRefGoogle Scholar
  18. Deprins D, Simar L, Tulkens H (1984) Measuring labor-efficiency in post offices. In: Marchand M, Pestieau P, Tulkens H (eds) The performance of public enterprises: concepts and measurement. North-Holland, Amsterdam, p 243–267. Chapter 10Google Scholar
  19. Deutsch J, Dumas A, Silber J (2013) Estimating an educational production function for five countries of Latin America on the basis of the PISA data. Econ Educ Rev 36:245–262CrossRefGoogle Scholar
  20. Dyson RG, Allen R, Camanho AS, Podinovski VV, Sarrico CS, Shale EA (2001) Pitfalls and protocols in DEA. Eur J Oper Res 132(2):260–273CrossRefGoogle Scholar
  21. Emrouznejad A, Parker BR, Tavares G (2010) Evaluation of research in efficiency and productivity: a survey and analysis of the first 30 years of scholarly literature in DEA. Socioecon Plann Sci 42(3):151–157CrossRefGoogle Scholar
  22. Ercikan K (2006) Examining guidelines for developing accurate proficiency level scores. Can J Educ 29(3):823–838CrossRefGoogle Scholar
  23. Färe R, Grosskopf S (2004) Modeling undesirable factors in efficiency evaluation: comment. Eur J Oper Res 157(1):242–245CrossRefGoogle Scholar
  24. Färe R, Grosskopf S (2009) A comment on weak disposability in nonparametric production analysis. Am J Agric Econ 91(2):535–538CrossRefGoogle Scholar
  25. Färe R, Grosskopf S, Lovell CAK (1994a) Production Frontiers. Cambridge University Press, CambridgeGoogle Scholar
  26. Färe R, Grosskopf S, Noh D-W, Weber WW (2005) Characteristics of a polluting technology: theory and practice. J Econom 126(2):469–492CrossRefGoogle Scholar
  27. Färe R, Grosskopf S, Norris M, Zhang Z (1994b) Productivity growth, technical progress, and efficiency change in industrialized countries. Am Econ Rev 84(1):66–83Google Scholar
  28. Färe R, Grosskopf S, Pasurka C (1989) Multilateral productivity comparisons when some outputs are undesirable: a nonparametric approach. Rev Econ Stat 71(1):90–98CrossRefGoogle Scholar
  29. Giambona F, Vassallo E, Vassiliadis E (2011) Educational systems efficiency in European Union countries. Stud educ eval 37(2):108–122CrossRefGoogle Scholar
  30. Giménez V, Prior D, Thieme C (2007) Technical efficiency, managerial efficiency and objective-setting in the educational system: an international comparison. J Oper Res Soc 58(8):996–1007CrossRefGoogle Scholar
  31. Golany B, Roll Y (1989) An application procedure for DEA. Omega 17(3):237–250CrossRefGoogle Scholar
  32. Golany B, Thore S (1997) The economic and social performance of nations: efficiency and returns to scale. Socioecon Plann Sci 31(3):191–204CrossRefGoogle Scholar
  33. Grifell-Tatjé E, Kerstens K (2008) Incentive regulation and the role of convexity in benchmarking electricity distribution: economists versus engineers. Ann Public Coop Econ 79(2):227–248CrossRefGoogle Scholar
  34. Grosskopf S, Hayes KJ, Taylor LL (2014) Efficiency in education: research and implications. Appl Econ Perspect Policy 36(2):175–210CrossRefGoogle Scholar
  35. Gupta S, Verhoeven M (2001) The efficiency of government expenditure: experiences from Africa. J Policy Model 23(4):433–467CrossRefGoogle Scholar
  36. Hailu A, Veeman TS (2001) Non-parametric productivity analysis with undesirable outputs: an application to the Canadian pulp and paper industry. Am J Agric Econ 83(3):605–616CrossRefGoogle Scholar
  37. Henderson DJ, Parmeter CF (2015) Applied Nonparametric Econometrics. Cambridge University Press, Cambridge, MACrossRefGoogle Scholar
  38. Henderson DJ, Russell RR (2005) Human capital and macroeconomic convergence: a production-frontier approach. Int Econ Rev 46(4):1167–1205CrossRefGoogle Scholar
  39. Hollingsworth B, Smith P (2003) Use of ratios in data envelopment analysis. Appl Econ Lett 10(11):733–735CrossRefGoogle Scholar
  40. Jacob WJ, Holsinger DB (2008) Inequality in education: a critical analysis. In: Jacob WJ, Holsinger DB (eds) Inequality in education. Springer, Hong Kong, p 1–33CrossRefGoogle Scholar
  41. Johnes J (2015) Operational research in education. Eur J Oper Res 243(3):683–696CrossRefGoogle Scholar
  42. Johnes J (2004) Efficiency measurement. In: Johnes G, Johnes J (eds) The international handbook on the economics of education. Edward Elgar, Cheltenham, UKGoogle Scholar
  43. De Jorge J, Santín D (2010) Determinantes de la eficiencia educativa en la Unión Europea. Hacienda Publica Española/Revista de Economía Pública 193(2):131–155Google Scholar
  44. Kumar S (2006) Environmentally sensitive productivity growth: a global analysis using malmquist–luenberger index. Ecol Econ 56(2):280–293CrossRefGoogle Scholar
  45. Kumar S, Russell RR (2002) Technological change, technological catch-up, and capital deepening: relative contributions to growth and convergence. Am Econ Rev 92(3):527–548CrossRefGoogle Scholar
  46. Kuosmanen T (2005) Weak disposability in nonparametric production analysis with undesirable outputs. Am J Agric Econ 87:1077–1082CrossRefGoogle Scholar
  47. Kuosmanen T, Podinovski VV (2009) Weak disposability in nonparametric production analysis: reply to Färe and Grosskopf. Am J Agric Econ 91:539–545CrossRefGoogle Scholar
  48. Li Q, Racine JS (2007) Nonparametric econometrics: theory and practice. Princeton University Press, Princeton and OxfordGoogle Scholar
  49. Loader CR (1996) Local likelihood density estimation. Ann Stat 24(4):1602–1618CrossRefGoogle Scholar
  50. Luenberger D (1992) New optimality principles for economic efficiency and equilibrium. J Optim Theory Appl 75(2):221–264CrossRefGoogle Scholar
  51. Nakano M, Managi S (2008) Regulatory reforms and productivity: an empirical analysis of the Japanese electricity industry. Energy Policy 36(1):201–209CrossRefGoogle Scholar
  52. OECD (2014) PISA 2012 technical report. Technical report, OECD Publishing, Paris.Google Scholar
  53. Oh D (2010) A global Malmquist-Luenberger productivity index. J Prod Anal 34(3):183–197CrossRefGoogle Scholar
  54. Olesen OB, Petersen NC, Podinovski VV (2015) Efficiency analysis with ratio measures. Eur J Oper Res 245(2):446–462CrossRefGoogle Scholar
  55. Pastor JT, Lovell CAK (2005) A global Malmquist productivity index. Econ Lett 88(2):266–271CrossRefGoogle Scholar
  56. Picazo-Tadeo AJ, Prior D (2009) Environmental externalities and efficiency measurement. J Environ Manage 90(11):3332–3339CrossRefGoogle Scholar
  57. Quah DT (1993a) Empirical cross-section dynamics in economic growth. Eur Econ Rev 37:426–434CrossRefGoogle Scholar
  58. Quah DT (1993b) Galton’s fallacy and tests of the convergence hypothesis. Scand J Econ 95(4):427–443CrossRefGoogle Scholar
  59. Reinhard S, Lovell CAK, Thijssen GJ (2002) Analysis of environmental efficiency variation. Am J Agric Econ 84:1054–1065CrossRefGoogle Scholar
  60. Scarf HE (1994) The allocation of resources in the presence of indivisibilities. J Econ Perspect 8(4):111–128CrossRefGoogle Scholar
  61. Scott DW (1992) Multivariate density estimation: theory, practice, and visualization. Wiley, New York, NYCrossRefGoogle Scholar
  62. Seiford LM, Zhu J (2002) Modeling undesirable factors in efficiency evaluation. Eur J Oper Res 142(1):16–20CrossRefGoogle Scholar
  63. Sheather SJ, Jones MC (1991) A reliable data-based bandwidth selection method for kernel density estimation. J R Stat Soc B 53(3):683–690Google Scholar
  64. Silverman BW (1986) Density estimation for statistics and data analysis. Chapman and Hall, LondonCrossRefGoogle Scholar
  65. Sueyoshi T, Goto M (2011) Measurement of returns to scale and damages to scale for dea-based operational and environmental assessment: how to manage desirable (good) and undesirable (bad) outputs? Eur J Oper Res 211(1):76–89CrossRefGoogle Scholar
  66. Sutherland D, Price R, Gonand F (2009) Improving public spending efficiency in primary and secondary education. OECD Econ Stud 2009(4):1–30Google Scholar
  67. Teddlie C, Reynolds D (2000) The international handbook of school effectiveness research. Routledge, LondonGoogle Scholar
  68. Thieme C, Giménez V, Prior D (2012) A comparative analysis of the efficiency of national educational systems. APER 13:1–15Google Scholar
  69. Tone K, Sahoo BK (2003) Scale, indivisibilities and production function in data envelopment analysis. Int J Prod Econ 84(2):165–192CrossRefGoogle Scholar
  70. Watanabe M, Tanaka K (2007) Efficiency analysis of Chinese industry: a directional distance function approach. Energy Policy 35(12):6323–6331CrossRefGoogle Scholar
  71. Weber WL, Domazlicky B (2001) Productivity growth and pollution in state manufacturing. Rev Econ Stat 83(1):195–199CrossRefGoogle Scholar
  72. Wenglinsky H (1998) Finance equalization and within-school equity: the relationship between education spending and the social distribution of achievement. Educ Eval Policy Anal 20(4):269–283CrossRefGoogle Scholar
  73. De Witte K, López-Torres L (2017) Efficiency in education: a review of literature and a way forward. J Oper Res Soc in press.Google Scholar
  74. Worthington AC (2001) An empirical survey of frontier efficiency measurement techniques in education. Educ Econ 9:245–268CrossRefGoogle Scholar
  75. Xue M, Harker PT (2002) Note: ranking DMUs with infeasible super-efficiency DEA models. Manage Sci 48(5):705–710CrossRefGoogle Scholar
  76. Yörük BK, Zaim O (2005) Productivity growth in OECD countries: a comparison with Malmquist indices. J Comp Econ 33(2):401–420CrossRefGoogle Scholar
  77. Zhang N, Choi Y (2013) Total-factor carbon emission performance of fossil fuel power plants in China: a metafrontier non-radial Malmquist index analysis. Energ Econ 40:549–559CrossRefGoogle Scholar
  78. Zhang N, Zhou P, Choi Y (2013) Energy efficiency, CO2 emission performance and technology gaps in fossil fuel electricity generation in Korea : a meta-frontier non-radial directional distance function analysis. Energy Policy 56:653–662CrossRefGoogle Scholar
  79. Zhou P, Ang BW, Wang H (2012) Energy and CO2 emission performance in electricity generation: a non-radial directional distance function approach. Eur J Oper Res 221(3):625–635CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Universitat Autònoma de BarcelonaBellaterra (Barcelona)Spain
  2. 2.Universidad Diego PortalesSantiagoChile
  3. 3.Universitat Jaume ICastellónSpain

Personalised recommendations