Social Indicators Research

, Volume 82, Issue 1, pp 111–145 | Cite as

An Introduction to ‘Benefit of the Doubt’ Composite Indicators

  • Laurens Cherchye
  • Willem Moesen
  • Nicky Rogge
  • Tom Van Puyenbroeck
Article

Abstract

Despite their increasing use, composite indicators remain controversial. The undesirable dependence of countries’ rankings on the preliminary normalization stage, and the disagreement among experts/stakeholders on the specific weighting scheme used to aggregate sub-indicators, are often invoked to undermine the credibility of composite indicators. Data envelopment analysis may be instrumental in overcoming these limitations. One part of its appeal in the composite indicator context stems from its invariance to measurement units, which entails that a normalization stage can be skipped. Secondly, it fills the informational gap in the ‘right’ set of weights by generating flexible ‘benefit of the doubt’-weights for each evaluated country. The ease of interpretation is a third advantage of the specific model that is the main focus of this paper. In sum, the method may help to neutralize some recurring sources of criticism on composite indicators, allowing one to shift the focus to other, and perhaps more essential stages of their construction.

Key words

composite indicators data envelopment analysis performance benchmarking technology 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Babbie E. (1995) The Practice of Social Research. Wadsworth Publishing Company, BelmontGoogle Scholar
  2. Booysen F. (2002) An Overview and Evaluation of Composite Indices of Development. Social Indicators Research 59: 115–151CrossRefGoogle Scholar
  3. Brandolini A. (2002) Education and Employment Indicators for the EU Social Agenda. Politica Economica 18: 55–62Google Scholar
  4. Charnes A., Cooper W. W., Rhodes E. (1978) Measuring the Efficiency of Decision Making Units. European Journal of Operational Research 2: 429–444CrossRefGoogle Scholar
  5. Cherchye L., C. A. K. Lovell, W. Moesen and T. Van Puyenbroeck: 2005, ‘One Market, One Number? A Composite Indicator Assessment of EU Internal Market Dynamics’, forthcoming in European Economic Review Google Scholar
  6. Cherchye L., Moesen W., Van Puyenbroeck T. (2004) Legitimately Diverse, Yet Comparable: On Synthesizing Social Inclusion Performance in the EU. Journal of Common Market Studies 42:919–955CrossRefGoogle Scholar
  7. Cherchye, L. and T. Kuosmanen: 2006, ‘Benchmarking Sustainable Development: A Synthetic Meta-Index Approach’ Chapter 7. in M. McGillivray and M. Clarke (eds), Perspectives on Human Development (United Nations University Press), to appearGoogle Scholar
  8. Cook, W. D.: 2004, ‘Qualitative Data in DEA’, in W. W. Cooper, L. Seiford and J. Zhu (eds), Handbook on Data Envelopment Analysis (Kluwer Academic Publishers, Dordrecht), pp.␣75–97Google Scholar
  9. Cook W. D., Kress M. (1991) A Multiple Criteria Decision Model with Ordinal Preference Data. European Journal of Operations Research 54: 191–193CrossRefGoogle Scholar
  10. Cook W. D., Kress M. (1994) A Multiple Criteria Composite Index Model for Quantitative and Qualitative Data. European Journal of Operations Research 78: 367–379CrossRefGoogle Scholar
  11. Cooper, W. W., L. M. Seiford and K. Tone: 2000, Data Envelopment Analysis: A Comprehensive Text with Models, Applications, References and DEA-Solver Software (Kluwer Academic Publishers, Dordrecht)Google Scholar
  12. Cooper, W. W., L. M. Seiford and J. Zhu: 2004, Handbook on Data Envelopment Analysis (Kluwer Academic Publishers, Dordrecht)Google Scholar
  13. Desai, M., S. Fukuda-Parr, C. Johansson and F. Sagasti: 2002, ‘Measuring the Technology Achievement of Nations and the Capacity to Participate in the Networking Age’. Journal of Human Development 3(1), pp. 95–122Google Scholar
  14. Despotis, D. K.: 2005, ‘A Reassessment of the Human Development Index via Data Envelopment Analysis’, Journal of the Operational Research Society 56(8), pp. 969–980Google Scholar
  15. Ebert U., Welsch H. (2004) Meaningful Environmental Indices: A Social Choice Approach. Journal of Environmental Economics and Management 47: 270–283CrossRefGoogle Scholar
  16. European Commission: 2004, The EU Economy Review 2004, European Economy, Nr. 6 (Office for Official Publications of the EC, Luxembourg)Google Scholar
  17. Foster, J. and A. Sen: 1997, On Economic Inequality, 2nd expanded edn (Clarendon Press, Oxford)Google Scholar
  18. Freudenberg, M.: 2003, ‘Composite Indicators of Country Performance: A Critical Assessment’, OECD Science, Technology and Industry Working Papers 2003/16Google Scholar
  19. Golany, B.: 1988, ‘A Note on Including Ordinal Relations among Multipliers in Data Envelopment Analysis’, Management Science 34(8), pp. 1029–1033Google Scholar
  20. Halme M., Joro T., Koivu M. (2002) Dealing with Interval Scale Data in Data Envelopment Analysis. European Journal of Operational Research 137: 22–27CrossRefGoogle Scholar
  21. Hopkins M. (1991) Human Development Revisited: A New UNDP Report. World Development 19: 1469–1473CrossRefGoogle Scholar
  22. Kao C., Hung H. T. (2005) Data Envelopment Analysis with Common Weights: The Compromise Solution Approach. Journal of the Operational Research Society 56: 1196–1203CrossRefGoogle Scholar
  23. Kumar S., Russel R. R. (2002) Technical Change, Technological Catch-Up, and Capital Deepening: Relative Contributions to Growth and Convergence. American Economic Review 92: 527–548CrossRefGoogle Scholar
  24. Lovell C. A. K, Pastor J. T., Turner J. A. (1995) Measuring Macroeconomic Performance in the OECD: A Comparison of European and Non-European Countries. European Journal of Operational Research 87: 507–518CrossRefGoogle Scholar
  25. Mahlberg, B. and M. Obersteiner: 2001, ‘Remeasuring the HDI by Data Envelopment Analysis’, International Institute for Applied Systems Analysis Interim Report 01–069Google Scholar
  26. Melyn, W. and W. Moesen: 1991, ‘Towards a Synthetic Indicator of Macroeconomic Performance: Unequal Weighting when Limited Information is Available’, Public Economics Research Paper 17, CES, KU LeuvenGoogle Scholar
  27. Micklewright, J.: 2001, ‘Should the UK Government Measure Poverty and Social Exclusion with a Composite Index?’, in CASE, Indicators of progress: A Discussion of Approaches to Monitor the Government’s Strategy to Tackle Poverty and Social Exclusion, CASE Report 13, London School of EconomicsGoogle Scholar
  28. Munda, G. and M. Nardo: 2003, ‘On the Methodological Foundations of Composite Indicators Used for Ranking Countries’, mimeo, Universitat Autonoma de BarcelonaGoogle Scholar
  29. Nardo, M., M. Saisana, A. Saltelli and S. Tarantola (EC/JRC) and A. Hoffman and E. Giovannini (OECD): 2005, Handbook on Constructing Composite Indicators: Methodology and User Guide, OECD Statistics Working PaperGoogle Scholar
  30. Pedraja-Chap arro F., Salinas-Jimenez J., Smith P. (1997) On the Role of Weight Restrictions in Data Envelopment Analysis. Journal of Productivity Analysis 8: 215–230CrossRefGoogle Scholar
  31. Saisana M., Saltelli A., Tarantola S. (2005) Uncertainty and Sensitivity Analysis as tools for the quality assessment of composite indicators. Journal of the Royal Statistical Society Series A 168: 1–17Google Scholar
  32. Semple J. (1996) Constrained Games for Evaluating Organizational Performance. European Journal of Operational Research 96: 103–112CrossRefGoogle Scholar
  33. Storrie, D. and H. Bjurek: 2000, ‘Benchmarking European Labour Market Performance with Efficiency Frontier Techniques’, CELMS Discussion Paper, Göteborg UniversityGoogle Scholar
  34. Thanassoulis, E., M. C. Portela and R. Allen: 2004, ‘Incorporating Value Judgements in DEA’, in W. W. Cooper, L. Seiford and J. Zhu (eds), Handbook on Data Envelopment Analysis (Kluwer Academic Publishers, Dordrecht), pp. 99–138Google Scholar
  35. Wong Y. H. B., Beasly J. E. (1990) Restricting Weight Flexibility in Data Envelopment Analysis. Journal of the Operational Research Society 41: 829–835CrossRefGoogle Scholar
  36. Zaim O., Färe R., Grosskopf S. (2001) An Economic Approach to Achievement and Improvement Indexes. Social Indicators Research 56: 91–118CrossRefGoogle Scholar
  37. Zhu, J.: 2003, Quantitative Models for Performance Evaluation and Benchmarking, International Series in Operations Research and Management Science (Kluwer Academic Publishers, Dordrecht)Google Scholar

Copyright information

© Springer Science+Business Media, Inc. 2006

Authors and Affiliations

  • Laurens Cherchye
    • 1
  • Willem Moesen
    • 1
  • Nicky Rogge
    • 1
    • 2
  • Tom Van Puyenbroeck
    • 1
    • 2
  1. 1.Centre for Economic StudiesCatholic University of LeuvenLeuvenBelgium
  2. 2.European University CollegeBrusselsBelgium

Personalised recommendations