Skip to main content

Advertisement

SpringerLink
Log in

An international comparison of educational systems: a temporal analysis in presence of bad outputs

  • Published:
Journal of Productivity Analysis Aims and scope Submit manuscript

Abstract

This study uses the global non-radial Malmquist index to measure performance change in the educational systems of 29 countries/economies participating in PISA 2003 and 2012 for students at age 15 in the disciplines of mathematics and reading. This methodology is particularly appropriate both for its desirable properties as well as its suitability for the educational context. Results indicate a positive evolution in educational systems’ performance during this period. This improvement is mainly due a positive efficiency change, which offsets the negative technological change observed. Nevertheless, a deeper scrutiny at the country level shows that results varied remarkably among them.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. Around 510,000 students in 65 economies took part in Pisa 2012 assessment of reading, mathematics and science representing about 28 million 15-year-olds globally. Complete information about PISA and databases can be found at https://www.oecd.org/pisa/.

  2. Recent literature reviews on efficiency in education include De Witte and López-Torres (2017), Johnes (2015), Grosskopf et al. (2014), Emrouznejad et al. (2010) and, a bit more distant in time, Johnes (2004) and Worthington (2001). In several of these studies, among other issues, the authors review thoroughly the studies that have dealt with the issue of efficiency in education, listing the inputs, outputs and environmental/contextual variables, considering the different levels of analysis (university, school/high school, district/county/city, or country), as well as the different methodological approaches. In addition, some authors (De Witte and López-Torres 2017) have an explicit attempt to link the standard economics of education literature and the (nonparametric) efficiency literature.

  3. See also the recent contribution by Aparicio et al. (2016a), in which the Malmquist index is applied to different samples of PISA data (2006, 2009 and 2012).

  4. Some excellent monographs on this issue are those by Silverman (1986), Scott (1992), Li and Racine (2007) and, more recently, Henderson and Parmeter (2015).

  5. The international contractor in each country randomly selects schools for participation in PISA. At these schools, the test is given to students between the ages of 15 years 3 months and 16 years 2 months at the time of the test, rather than to students in a specific year of school; this age represents the end of compulsory education in most participating countries. In general, each version of PISA considers a minimum of 150 schools per participant country/economy (or all the schools if there are fewer than 150 schools in that country/economy). Within each participating school, a sample of students, usually numbering 35, is selected with equal probability (all students take that test if there are fewer than 35 in the school and with a minimum of 20 students so as to guarantee the validity of the test within and among schools). In total, in each country a minimum size of 4500 students are tested.

  6. The original values can be obtained by subtracting 10 from the values shown in Table 3.

  7. We will refer to the concepts of productivity and performance interchangeably.

References

  • Afonso A, St. Aubyn M (2005) Non-parametric approaches to education and health effciency in OECD countries. J Appl Econ 8(2):227–246

    Google Scholar 

  • Afonso A, St. Aubyn M (2006) Cross-country effciency of secondary education provision: a semi-parametric analysis with non-discretionary inputs. Econ Model 23(3):476–491

    Article  Google Scholar 

  • Agasisti T (2014) The efficiency of public spending on education: an empirical comparison of EU countries. Eur J Educ 49(4):543–557

    Article  Google Scholar 

  • Agasisti T and Zoido P (2015). The efficiency of secondary schools in an international perspective: preliminary results from PISA 2012. OECD Education Working Papers 117, OECD, Paris.

  • Aparicio J, Crespo-Cebada E, Pedraja-Chaparro F, Santín D (2016a) Comparing school ownership performance using a pseudo-panel database: a Malmquist-type index approach. Eur J Oper Res 256(2):533–542.

  • Aparicio J, Pastor JT, Vidal F (2016b) The directional distance function and the translation invariance property. Omega 58:1–3

    Article  Google Scholar 

  • Aristovnik A, Obadić A (2014) Measuring relative efficiency of secondary education in selected EU and OECD countries: the case of Slovenia and Croatia. TEDE 20(3):419–433

    Article  Google Scholar 

  • Badunenko O, Romero-Ávila D (2013) Financial development and the sources of growth and convergence. Int Econ Rev 54(2):629–663

    Article  Google Scholar 

  • Balk BM (2001) Scale efficiency and productivity change. J Prod Anal 15(3):159–183

    Article  Google Scholar 

  • Brown, G, Micklewright, J, Schnepf, SV, and Waldmann, R (2007). International surveys of educational achievement: how robust are the findings? J R Stat Soc A 170(3):623–646

  • Carlson D (2001) Focusing state educational accountability systems: four methods for judging school quality and progress. Technical report, Center for Assessment (NCIEA), Dover, NH.

  • Caves DW, Christensen LR, Diewert WE (1982) The economic theory of index numbers and the measurement of input, output, and productivity. Econometrica 50(6):1393–1414

    Article  Google Scholar 

  • Chung YH, Färe R, Grosskopf S (1997) Productivity and undesirable outputs: a directional distance function approach. J Environ Manage 51:229–240

    Article  Google Scholar 

  • Clements B (2002) How efficient is education spending in Europe? Eur Rev Econ Finance 1(1):3–26

    Google Scholar 

  • Cooper, WW, Seiford, LM, and Tone, K (2007). Data envelopment analysis: a comprehensive text with models, applications, references and DEA-solver software. Springer Science & Business Media, New York

  • Cordero JM, Santín D, Simancas R (2017) Assessing European primary school performance through a conditional nonparametric model. J Oper Res Soc in press.

  • Denvir B, Brown M (1986) Understanding of number concepts in low attaining 7–9 year olds: Part I. development of descriptive framework and diagnostic instrument. Educ Stud Math 17(1):15–36

    Article  Google Scholar 

  • Deprins D, Simar L, Tulkens H (1984) Measuring labor-efficiency in post offices. In: Marchand M, Pestieau P, Tulkens H (eds) The performance of public enterprises: concepts and measurement. North-Holland, Amsterdam, p 243–267. Chapter 10

    Google Scholar 

  • Deutsch J, Dumas A, Silber J (2013) Estimating an educational production function for five countries of Latin America on the basis of the PISA data. Econ Educ Rev 36:245–262

    Article  Google Scholar 

  • Dyson RG, Allen R, Camanho AS, Podinovski VV, Sarrico CS, Shale EA (2001) Pitfalls and protocols in DEA. Eur J Oper Res 132(2):260–273

    Article  Google Scholar 

  • Emrouznejad A, Parker BR, Tavares G (2010) Evaluation of research in efficiency and productivity: a survey and analysis of the first 30 years of scholarly literature in DEA. Socioecon Plann Sci 42(3):151–157

    Article  Google Scholar 

  • Ercikan K (2006) Examining guidelines for developing accurate proficiency level scores. Can J Educ 29(3):823–838

    Article  Google Scholar 

  • Färe R, Grosskopf S (2004) Modeling undesirable factors in efficiency evaluation: comment. Eur J Oper Res 157(1):242–245

    Article  Google Scholar 

  • Färe R, Grosskopf S (2009) A comment on weak disposability in nonparametric production analysis. Am J Agric Econ 91(2):535–538

    Article  Google Scholar 

  • Färe R, Grosskopf S, Lovell CAK (1994a) Production Frontiers. Cambridge University Press, Cambridge

    Google Scholar 

  • Färe R, Grosskopf S, Noh D-W, Weber WW (2005) Characteristics of a polluting technology: theory and practice. J Econom 126(2):469–492

    Article  Google Scholar 

  • Färe R, Grosskopf S, Norris M, Zhang Z (1994b) Productivity growth, technical progress, and efficiency change in industrialized countries. Am Econ Rev 84(1):66–83

    Google Scholar 

  • Färe R, Grosskopf S, Pasurka C (1989) Multilateral productivity comparisons when some outputs are undesirable: a nonparametric approach. Rev Econ Stat 71(1):90–98

    Article  Google Scholar 

  • Giambona F, Vassallo E, Vassiliadis E (2011) Educational systems efficiency in European Union countries. Stud educ eval 37(2):108–122

    Article  Google Scholar 

  • Giménez V, Prior D, Thieme C (2007) Technical efficiency, managerial efficiency and objective-setting in the educational system: an international comparison. J Oper Res Soc 58(8):996–1007

    Article  Google Scholar 

  • Golany B, Roll Y (1989) An application procedure for DEA. Omega 17(3):237–250

    Article  Google Scholar 

  • Golany B, Thore S (1997) The economic and social performance of nations: efficiency and returns to scale. Socioecon Plann Sci 31(3):191–204

    Article  Google Scholar 

  • Grifell-Tatjé E, Kerstens K (2008) Incentive regulation and the role of convexity in benchmarking electricity distribution: economists versus engineers. Ann Public Coop Econ 79(2):227–248

    Article  Google Scholar 

  • Grosskopf S, Hayes KJ, Taylor LL (2014) Efficiency in education: research and implications. Appl Econ Perspect Policy 36(2):175–210

    Article  Google Scholar 

  • Gupta S, Verhoeven M (2001) The efficiency of government expenditure: experiences from Africa. J Policy Model 23(4):433–467

    Article  Google Scholar 

  • Hailu A, Veeman TS (2001) Non-parametric productivity analysis with undesirable outputs: an application to the Canadian pulp and paper industry. Am J Agric Econ 83(3):605–616

    Article  Google Scholar 

  • Henderson DJ, Parmeter CF (2015) Applied Nonparametric Econometrics. Cambridge University Press, Cambridge, MA

    Book  Google Scholar 

  • Henderson DJ, Russell RR (2005) Human capital and macroeconomic convergence: a production-frontier approach. Int Econ Rev 46(4):1167–1205

    Article  Google Scholar 

  • Hollingsworth B, Smith P (2003) Use of ratios in data envelopment analysis. Appl Econ Lett 10(11):733–735

    Article  Google Scholar 

  • Jacob WJ, Holsinger DB (2008) Inequality in education: a critical analysis. In: Jacob WJ, Holsinger DB (eds) Inequality in education. Springer, Hong Kong, p 1–33

    Chapter  Google Scholar 

  • Johnes J (2015) Operational research in education. Eur J Oper Res 243(3):683–696

    Article  Google Scholar 

  • Johnes J (2004) Efficiency measurement. In: Johnes G, Johnes J (eds) The international handbook on the economics of education. Edward Elgar, Cheltenham, UK

  • De Jorge J, Santín D (2010) Determinantes de la eficiencia educativa en la Unión Europea. Hacienda Publica Española/Revista de Economía Pública 193(2):131–155

    Google Scholar 

  • Kumar S (2006) Environmentally sensitive productivity growth: a global analysis using malmquist–luenberger index. Ecol Econ 56(2):280–293

    Article  Google Scholar 

  • Kumar S, Russell RR (2002) Technological change, technological catch-up, and capital deepening: relative contributions to growth and convergence. Am Econ Rev 92(3):527–548

    Article  Google Scholar 

  • Kuosmanen T (2005) Weak disposability in nonparametric production analysis with undesirable outputs. Am J Agric Econ 87:1077–1082

    Article  Google Scholar 

  • Kuosmanen T, Podinovski VV (2009) Weak disposability in nonparametric production analysis: reply to Färe and Grosskopf. Am J Agric Econ 91:539–545

    Article  Google Scholar 

  • Li Q, Racine JS (2007) Nonparametric econometrics: theory and practice. Princeton University Press, Princeton and Oxford

    Google Scholar 

  • Loader CR (1996) Local likelihood density estimation. Ann Stat 24(4):1602–1618

    Article  Google Scholar 

  • Luenberger D (1992) New optimality principles for economic efficiency and equilibrium. J Optim Theory Appl 75(2):221–264

    Article  Google Scholar 

  • Nakano M, Managi S (2008) Regulatory reforms and productivity: an empirical analysis of the Japanese electricity industry. Energy Policy 36(1):201–209

    Article  Google Scholar 

  • OECD (2014) PISA 2012 technical report. Technical report, OECD Publishing, Paris.

  • Oh D (2010) A global Malmquist-Luenberger productivity index. J Prod Anal 34(3):183–197

    Article  Google Scholar 

  • Olesen OB, Petersen NC, Podinovski VV (2015) Efficiency analysis with ratio measures. Eur J Oper Res 245(2):446–462

    Article  Google Scholar 

  • Pastor JT, Lovell CAK (2005) A global Malmquist productivity index. Econ Lett 88(2):266–271

    Article  Google Scholar 

  • Picazo-Tadeo AJ, Prior D (2009) Environmental externalities and efficiency measurement. J Environ Manage 90(11):3332–3339

    Article  Google Scholar 

  • Quah DT (1993a) Empirical cross-section dynamics in economic growth. Eur Econ Rev 37:426–434

    Article  Google Scholar 

  • Quah DT (1993b) Galton’s fallacy and tests of the convergence hypothesis. Scand J Econ 95(4):427–443

    Article  Google Scholar 

  • Reinhard S, Lovell CAK, Thijssen GJ (2002) Analysis of environmental efficiency variation. Am J Agric Econ 84:1054–1065

    Article  Google Scholar 

  • Scarf HE (1994) The allocation of resources in the presence of indivisibilities. J Econ Perspect 8(4):111–128

    Article  Google Scholar 

  • Scott DW (1992) Multivariate density estimation: theory, practice, and visualization. Wiley, New York, NY

    Book  Google Scholar 

  • Seiford LM, Zhu J (2002) Modeling undesirable factors in efficiency evaluation. Eur J Oper Res 142(1):16–20

    Article  Google Scholar 

  • Sheather SJ, Jones MC (1991) A reliable data-based bandwidth selection method for kernel density estimation. J R Stat Soc B 53(3):683–690

    Google Scholar 

  • Silverman BW (1986) Density estimation for statistics and data analysis. Chapman and Hall, London

    Book  Google Scholar 

  • Sueyoshi T, Goto M (2011) Measurement of returns to scale and damages to scale for dea-based operational and environmental assessment: how to manage desirable (good) and undesirable (bad) outputs? Eur J Oper Res 211(1):76–89

    Article  Google Scholar 

  • Sutherland D, Price R, Gonand F (2009) Improving public spending efficiency in primary and secondary education. OECD Econ Stud 2009(4):1–30

    Google Scholar 

  • Teddlie C, Reynolds D (2000) The international handbook of school effectiveness research. Routledge, London

    Google Scholar 

  • Thieme C, Giménez V, Prior D (2012) A comparative analysis of the efficiency of national educational systems. APER 13:1–15

    Google Scholar 

  • Tone K, Sahoo BK (2003) Scale, indivisibilities and production function in data envelopment analysis. Int J Prod Econ 84(2):165–192

    Article  Google Scholar 

  • Watanabe M, Tanaka K (2007) Efficiency analysis of Chinese industry: a directional distance function approach. Energy Policy 35(12):6323–6331

    Article  Google Scholar 

  • Weber WL, Domazlicky B (2001) Productivity growth and pollution in state manufacturing. Rev Econ Stat 83(1):195–199

    Article  Google Scholar 

  • Wenglinsky H (1998) Finance equalization and within-school equity: the relationship between education spending and the social distribution of achievement. Educ Eval Policy Anal 20(4):269–283

    Article  Google Scholar 

  • De Witte K, López-Torres L (2017) Efficiency in education: a review of literature and a way forward. J Oper Res Soc in press.

  • Worthington AC (2001) An empirical survey of frontier efficiency measurement techniques in education. Educ Econ 9:245–268

    Article  Google Scholar 

  • Xue M, Harker PT (2002) Note: ranking DMUs with infeasible super-efficiency DEA models. Manage Sci 48(5):705–710

    Article  Google Scholar 

  • Yörük BK, Zaim O (2005) Productivity growth in OECD countries: a comparison with Malmquist indices. J Comp Econ 33(2):401–420

    Article  Google Scholar 

  • Zhang N, Choi Y (2013) Total-factor carbon emission performance of fossil fuel power plants in China: a metafrontier non-radial Malmquist index analysis. Energ Econ 40:549–559

    Article  Google Scholar 

  • Zhang N, Zhou P, Choi Y (2013) Energy efficiency, CO2 emission performance and technology gaps in fossil fuel electricity generation in Korea : a meta-frontier non-radial directional distance function analysis. Energy Policy 56:653–662

    Article  Google Scholar 

  • Zhou P, Ang BW, Wang H (2012) Energy and CO2 emission performance in electricity generation: a non-radial directional distance function approach. Eur J Oper Res 221(3):625–635

    Article  Google Scholar 

Download references

Acknowledgements

Claudio Thieme and Emili Tortosa-Ausina thank FONDECYT (National Fund of Scientific and Technological Development, grant #1121164 and #1151313) for generous financial support. Víctor Giménez, Diego Prior and Emili Tortosa-Ausina acknowledge the financial support of the Ministerio de Economía y Competitividad (ECO2013-44115-P and ECO2014-55221-P). Emili Tortosa-Ausina also acknowledges the financial support of Generalitat Valenciana (PROMETEOII/2014/046) and Universitat Jaume I (P1.1B2014-17). All four authors are grateful to the Associate Editor and two anonymous referees, whose comments contributed to an overall improvement of the paper. The usual disclaimer applies.

Author information

Authors and Affiliations

  1. Universitat Autònoma de Barcelona, Bellaterra (Barcelona), 08193, Spain

    Víctor Giménez & Diego Prior

  2. Universidad Diego Portales, Avenida Santa Clara 797, Huechuraba, Santiago, Chile

    Claudio Thieme

  3. Universitat Jaume I, Avenida de Vicent Sos Baynat, s/n, Castellón, 12071, Castelló, Spain

    Emili Tortosa-Ausina

Authors
  1. Víctor Giménez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Claudio Thieme
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Diego Prior
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Emili Tortosa-Ausina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Emili Tortosa-Ausina.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Giménez, V., Thieme, C., Prior, D. et al. An international comparison of educational systems: a temporal analysis in presence of bad outputs. J Prod Anal 47, 83–101 (2017). https://doi.org/10.1007/s11123-017-0491-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11123-017-0491-9

Keywords

JEL Classifications

Search

Navigation