Advertisement

Hitting a Moving Target: Evaluating the ‘Exit’ from State-building Missions

  • Georg Frerks
Chapter
Part of the NL ARMS book series (NLARMS)

Abstract

Why is it so difficult to say something meaningful about the exit from state building missions? Or to put it otherwise: why is the evaluation thereof so problematic? In this article I reflect on the nature of exit from state-building missions and the challenge of evaluating it. I do this by first discussing the growing importance of evaluation and the conditions required for a proper evaluation as exposed in the professional evaluation literature. I refer here to the notion of evaluability. A key problem is the very basic question of establishing proper evaluation criteria and procedures. It is difficult to determine against what criteria state-building missions need to be evaluated. This is complicated by the dichotomy observed in reality between exit when particular end states have been achieved or exit at a particular end date. Comparing the praxis around the exit from state-building missions with the conditions of evaluability reveals several complications either in relation to the subject of evaluation, the research process needed to collect and analyse the required data, or the context of violent conflict. Improvements of evaluation practice in state-building operations can be achieved by working on a number of the preconditions outlined, by improving the formulation of the objectives and intervention logic of state-building operations. Investing more in narrative types of research endeavor, qualitative case studies, ethnographic approaches and the use of everyday, bottom-up indicators may help understanding the exit from state-building better, in a way that is grounded, and contingent on the local situation, while taking into account the multiple perspectives of the different stakeholders involved.

Keywords

evaluation evaluability exit intervention logic state-building 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. ALNAP (2001) Humanitarian Action: Learning from Evaluation, ALNAP Annual Review 2001, ALNAP, LondonGoogle Scholar
  2. American Evaluation Association, Guiding principles for evaluators. http://www.eval.org/p/cm/ld/fid=51. Accessed 30 March 2015
  3. Ayoob M (2007) State Making, State Breaking, and State Failure. In: Crocker CA et al. (eds), Leashing the Dogs of War, United States Institute of Peace, Washington, pp 95-114Google Scholar
  4. Beuren J van, Gewald J-B (2004) From Output to Outcome, 25 years of IOB evaluations, Aksant Academic Publishers, AmsterdamGoogle Scholar
  5. Bratton C, Chang ECC (2006) State Building and Democratization in Sub-Saharan Africa. Forwards, Backwards, or Together?, Comparative Political Studies, 39(9):1059-1083Google Scholar
  6. Bush K (2004) Commodification, Compartmentalization, and Militarization of Peacebuilding. In: Keating T, Knight WA (eds) Building Sustainable Peace, University of Alberta Press, Edmonton, pp 23-45Google Scholar
  7. Call CT (2008) Knowing Peace When you See It: Setting Standards for Peacebuilding Success, Civil Wars, 10(2):173-194Google Scholar
  8. Caplan R (2012a) Exit Strategies and Statebuilding. In: Caplan R (ed) Exit Strategies and State Building, Oxford University Press, New York, pp 3-18Google Scholar
  9. Caplan R (2012b) Policy Implications. In: Caplan R (ed) Exit Strategies and State Building, Oxford University Press, New York, pp 311-319Google Scholar
  10. Chelimski E (1995) Preamble: New Dimensions in Evaluation. In: Piciotto R, Rist RC (eds) Evaluating Country Development Policies and Programs: New Approaches for a New Agenda, New Directions for Evaluation, 67:3-11Google Scholar
  11. Chesterman S, Ignatieff M, Th akur R (eds) (2005) Making states work: State failure and the crisis of governance, United Nations University Press, Tokyo, New York, ParisGoogle Scholar
  12. Commissie Hoofdstructuur Rijksdienst (1980) Rapport no. 3. Elk kent de laan die derwaarts gaat, Staatsuitgeverij, Den HaagGoogle Scholar
  13. Davies R, Payne L (2015) Evaluability Assessments: reflection on a review of the literature, Evaluation, 21, 2: 216-231Google Scholar
  14. Debiel T, Lambach D (2009) How State-Building Strategies Miss Local Realities, Conflict, Security & Development, 21(1):22-28Google Scholar
  15. Denskus T (2010) Challenging the international peacebuilding evaluation discourse with qualitative methodologies, Evaluation and Program Planning, 35:148–153Google Scholar
  16. Diehl PF, Druckman D (2010) Evaluating Peace Operations. Lynne Rienner Publishers, BoulderGoogle Scholar
  17. Diehl PF, Druckman D (2012) Peace Operation Success: The Evaluation Framework, Journal of International Peacekeeping, 16:209-225Google Scholar
  18. Downes-Martin S (2011) Operations assessment in Afghanistan is broken. What is to be done? Naval War College Review, 64(4):103-125Google Scholar
  19. Druckman D, Stern PC (1997) Evaluating Peacekeeping Missions, Mershon International Studies Review 41:151-165Google Scholar
  20. Eriksen SS (2009) The Liberal Peace Is Neither: Peacebuilding, State building and the Reproduction of Conflict in the Democratic Republic of Congo, International Peacekeeping, 16(5): 652–666Google Scholar
  21. Fortna VP, Huang R (2012) Democratization after Civil War: A Brush-Clearing Exercise, International Studies Quarterly, 56:801-808Google Scholar
  22. Grandia Mantas M (2015) Deadly Embrace? The Decision Paths to Uruzgan and Helmand. PhD-thesis, Leiden University, LeidenGoogle Scholar
  23. Greene JC (1994) Qualitative program evaluation, practice and promise. In: Denzin NK, Lincoln YS (eds) Handbook of Qualitative Research. Sage Publications, Thousand Oaks, pp 530-544Google Scholar
  24. Greene JC (1997), Evaluation as Advocacy, Evaluation Practice, 18(1): 25-35Google Scholar
  25. Guba EG, Lincoln YS (1989) Fourth Generation Evaluation. Sage, LondonGoogle Scholar
  26. Horst, P, Nay JN, Scanlon JW, Wholey JS (1974) Program management and the federal evaluator, Public Administration Review, 34:300-308Google Scholar
  27. Jones DD (2006) Understanding Measures of Effectiveness in Counterinsurgency Operations. School of Advanced Military Studies, United States Army Command and General Staff College, KansasGoogle Scholar
  28. Kennedy-Chouane MG (2011) Improving conflict prevention and peacebuilding assistance through evaluation, OECD Journal: General Papers, Vol. 2010/1 http://dx.doi.org/10.1787/gen_papers-2010-5kgc6cl2vlvd. Accessed 11 April 2015
  29. Lekanne dit Deprez B (1995) Evaluation as Contested Knowledge. In: Frerks GE, den Ouden JHB (eds) In Search of the Middle ground. Essays on the Sociology of Planned development. Liber amicorum Dirk van Dusseldorp. Wageningen Agricultural University, Wageningen, pp 114-128Google Scholar
  30. Long N (2001) Development sociology, actor perspectives. Routledge, London and New YorkGoogle Scholar
  31. Mac Ginty R (2013) Indicators+: A proposal for everyday peace indicators, Evaluation and Program Planning, 36: 56-63Google Scholar
  32. Marsden D, Oakley P, Future issues and perspectives in the evaluation of social development, Community Development Journal, 26(4): 315-328Google Scholar
  33. Marten K (2007) Statebuilding and Force: The Proper Role of Foreign Militaries, Journal of Intervention and Statebuilding, 1(2): 231-247Google Scholar
  34. Ministry of Foreign Affairs (1994) Humanitarian Aid to Somalia, evaluation report. Ministry of Foreign Affairs, The HagueGoogle Scholar
  35. Ministry of Foreign Affairs (2009) Evaluation policy and guidelines for evaluation. Ministry of Foreign Affairs, The HagueGoogle Scholar
  36. NATO (2001) Peace Support Operations, AJP-3.4.1, NATO, BrusselsGoogle Scholar
  37. NATO (2011) Allied Joint Doctrine for the Conduct of Operations, AJP-3(B), NATO, BrusselsGoogle Scholar
  38. OECD (1991; reprint 2008) Principles for Evaluation of Development Assistance, Development Assistance Committee, OECD, ParisGoogle Scholar
  39. OECD (1999) Guidance for Evaluating Humanitarian Assistance in Complex Emergencies, OECD, ParisGoogle Scholar
  40. OECD (2010a) Evaluating Development Co-Operation. Summary of Key Norms and Standards (2nd ed.), OECD, ParisGoogle Scholar
  41. OECD (2010b) Glossary of key terms in evaluation and results based management. OECD Publications, ParisGoogle Scholar
  42. OECD (2010c) Quality Standards for Development Evaluation, OECD, ParisGoogle Scholar
  43. O’Hanlon M (2010) Staying Power. The U.S. Mission in Afghanistan Beyond 2011, Foreign Affairs, 89(5): 63-79Google Scholar
  44. Paris R, Sisk T (2009) Conclusion: confronting the contradictions. In: Paris R and Sisk T (eds) The Dilemmas of Statebuilding, Routledge, London and New York, pp 304-315Google Scholar
  45. Patton MQ (2001) Evaluation, Knowledge Management, Best Practices, and High Quality Lessons Learned, American Journal of Evaluation, 22:329Google Scholar
  46. Pauly E (2005) The Role of Evaluation in the 21st Century Foundation, Bertelsmann Foundation, GüterslohGoogle Scholar
  47. Piciotto R (2015) Democratic evaluation for the 21st century. Evaluation, 21(2): 150-166Google Scholar
  48. Rebien CC (1994) Evaluating Development Assistance. The Potentials of Participatory Evaluation. Forum for Development Studies 21(1-2): 295-307Google Scholar
  49. Richards P (2005) New War: An Ethnographic Approach. In: Richards P (ed) No Peace, No War. James Currey Publishers, Oxford, pp 1-21Google Scholar
  50. Richmond OP (2014) New Approaches to Peacebuilding, International Peacekeeping, 21(5): 696-700Google Scholar
  51. Rossi PH (1991) Comprehensive, Tailored, Theory-Driven Evaluation – A Smorgasbrod of Options. In: Shadish RS, Cook TD, Leviton LC (eds) FoundationsGoogle Scholar
  52. of Program Evaluations, Theories of Practice. Sage, Newbury Park, London and New Delhi, pp 377-437Google Scholar
  53. Rutman L (1977) Planning an Evaluation Study. In: Rutman L (ed) Evaluation Research Methods: A Basic Guide, Sage Publications, Beverly Hills/London, pp 13-38Google Scholar
  54. Schroden J (2011) Why Operations Assessments Fail. It’s Not Just the Metrics, Naval War College Review, 64(4): 89-102Google Scholar
  55. Scharbatke-Church C (2011) Evaluating Peacebuilding. Not yet all it could be.Google Scholar
  56. Berghof Handbook. www.berghof-handbook.net. Accessed 13 April 2015
  57. Scriven M (2000) Evaluation and the Philanthropic Fallacy, Grantmakers’ Evaluation Network, Vol. 7, No. 1Google Scholar
  58. Shadish RS, Cook TD, Leviton LC (eds) Foundations of Program Evaluations, Theories of Practice. Sage, Newbury Park, London and New DelhiGoogle Scholar
  59. Serwer D, Thomson P (2007) A framework for Success: International Intervention in Societies Emerging from Conflict. In: Crocker CA et al. (eds.) Leashing theGoogle Scholar
  60. Dogs of War, United States Institute of Peace, Washington, pp 553-581Google Scholar
  61. Simons H (2015) Interpret in context: Generalizing from the single case in evaluation, Evaluation, 21(2): 173-188Google Scholar
  62. Stedman SJ (2001) Spoiler Problems in Peace Processes. In: Brown ME et al. (eds) Nationalism and Ethnic Conflict, The MIT Press, Cambridge, Mass. and London, pp 366-414Google Scholar
  63. Suhrke A (2013) Statebuilding in Afghanistan: a contradictory engagement, Central Asian Survey, 32(3): 271-286Google Scholar
  64. Th akur R (2012) Using a Framework to Evaluate Peace Operations: The Role of the United Nations, Journal of International Peacekeeping, 16: 353–37Google Scholar
  65. UK Joint Doctrine Publication 3-40 (2009) Security and stabilisation: the military contribution, Her Majesty’s Stationary Office, LondonGoogle Scholar
  66. US Counterinsurgency Manual 3-24 (2006) ChicagoGoogle Scholar
  67. Weiss CH (1988) Evaluation for Decisions: Is anybody there? Does anybody care?. Evaluation Practice, 9(1): 5-19Google Scholar
  68. Weiss CH (1991) Linking Evaluation to Policy Research. In: Shadish RS, Cook TD, Leviton LC (eds) Foundations of Program Evaluations, Theories of Practice. Sage, Newbury Park, London and New Delhi, pp 179-224Google Scholar
  69. Wholey JS (1977) Evaluability Assessment. In: Rutman L (ed) Evaluation Research Methods: A Basic Guide, Sage Publications, Beverly Hills/London, pp 39-57Google Scholar
  70. Williams P (2001) Fighting for Freetown: British military interventions in Sierra Leone, Contemporary Security Policy, 22(3):140-168Google Scholar

Copyright information

© T.M.C. Asser Press and the authors 2016

Authors and Affiliations

  1. 1.Netherlands Defence AcademyBredaThe Netherlands

Personalised recommendations