Abstract
How has the Fund institutionalized independent evaluation as a means of assessing its performance? This paper process-traces the contentious creation of the Fund’s Independent Evaluation Office (IEO). I use primary interviews conducted at the Fund headquarters in 2008–2010 and Fund archive documents dating back to the beginning of the debate over independent evaluation in 1992 to analyze the interaction of internal and external actors and interests that led finally to the creation of the IEO in 2001. I then comment on the ‘performance of the performance evaluator.’ I draw from a recent external evaluation of the IEO (Lissakers et al. 2006), as well as interviews and secondary sources, to identify enduring contestation over the IEO’s function and scope of authority and to discern how this has affected the ability of the IEO to inform and shape the Fund’s process and outcome performance. To this end, I discuss four issues currently facing the IEO: the need to establish both actual and perceived independence, the problems of ambiguous or non-existent metrics for assessing Fund performance, difficulties in balancing candor of evaluation reports with credibility in the eyes of multiple constituencies, and the challenges of fostering a culture of learning in the Fund.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
By comparison, the World Bank established its Operations Evaluation Department (now the Independent Evaluation Group) in 1973 and the Asian Development Bank OED was created in 1978.
It is also likely that the Fund was strongly influenced by peer institutions, especially the World Bank Operations Evaluation Department (just down the street from the Fund) and the Evaluation Cooperation Groups of the Multilateral Development Banks), created in 1996.
For a full list of IEO completed reports, see http://www.ieo-imf.org/eval/eval_complete.html.
See statements by Executive Director Ian Clark (Canada), IMF Archives (EBM/96/16: 57).
Specifically, officials were concerned that evaluation staff wishing to return to the Fund might rationally be deterred from pushing reticent managers for sensitive information or reporting critical findings that may embarrass future bosses or client governments. See Lissakers et al. (2006: 13) and Easterly (2006).
In a January 1993 Executive Board meeting, Executive Director Prader argued that any rules assigned to staff selection in an EVO must avoid the “drawback of creating the same kind of dual staff structure in the Fund, in which all the ambitious economists would try to avoid secondment to the evaluation office” (IMF Archives EBM/93/9: 26).
This strongly reflected the experience of the World Bank. The World Bank created its Operational Evaluation Department (now called the Independent Evaluation Group) in 1973—one of the first to be established in a large intergovernmental organization. The OED’s history reveals a long struggle with the recruitment of staff in light of the widely held perception within the Bank that a move to the OED was “career-ending.” The World Bank ultimately abandoned its “one-way rule.” See Picciotto (2003).
Interviews with IEO and Fund staff. Ahluwalia stepped down as IEO Director in 2004 to return to India as Deputy Chairman of the Planning Commission. He was succeeded by David Goldsbrough as Acting Director, and then by Tom Bernes in April 2005.
However, dismissal of the IEO director requires a 70% majority vote on the Board.
Increasingly sophisticated quantitative analyses offer an improved understanding of IMF effectiveness but also contain many contradictory results, as one survey concludes (Steinward and Stone 2008).
There have been only two country-specific reports to date from the IEO. The first was on Jordan, which had ended its relationship with the Fund prior to the evaluation and was thus considered a “safe” topic. The other was Argentina. This report was fiercely debated when it was presented to the Board. In the case of the IEO’s report on capital account crises, the discussion and publication of the report was delayed at the request of the Indonesian government and opposed by Fund management due to fears that the release of the report would adversely affect the Fund’s credibility in the East Asian countries (Lissakers et al. 2006: 16). See also IEO (2003a, 2004b, 2005a, b).
Interview with John Hicklin, former Deputy Director of the IEO. Another example concerns the IEO’s report on Fiscal Adjustment in IMF-Support Programs (2003b). The IEO found little evidence to support the commonly held notion among Fund critics that Fund-supported programs adopt a “one-size-fits-all” approach, are inflexible, cause a decline in social spending, and are associated with lower growth. IEO staff expressed concern that such “positive findings” might be seen by NGOs as evidence of bias on part of the IEO, thus making its conclusions and recommendations suspect.
Noticeably, the requirement is for the management and staff to report on the implementation of the Board-endorsed recommendations of the IEO. The Board has not always agreed with the IEO’s recommendations for policy change in the Fund. For example, the Board disagreed with the IEO’s recommendation to place a cap on conditions (to four or five per year, which is half of the current average for performance criteria and prior actions in Fund loans). On the other hand, the IEO’s report on the Fund’s exchange rate policy advice was widely supported by the Board despite a very critical reaction by the management and staff, which may promise more results in terms of change in organizational policy and practice IEO (2008: 3, 5).
The first PMR, published in December 2007, was a very substantial document, which attempted to respond to all of the recommendations stemming from the first ten IEO reports. Interviews with IEO and PDR staff indicate that there was some resistance to this exercise.
The Fund’s surveillance activities generally fall into three categories: bilateral surveillance of individual countries through Article IV consultations, multilateral surveillance through the World Economic Outlook Reports and Global Financial Stability Reports, and the financial sector assessment papers (FSAP) (Bernes 2008: 7).
Bernes (2008: 4). Bernes specifically notes three IEO evaluations that followed up on the Crow Report: the 2003 evaluation of the Fund’s response to capital account crisis, the 2006 report on the Financial Sector Assessment Program (FSAP), and the 2006 report on multilateral surveillance (IEO 2003b, 2006a, b).
Bernes (2008). One of the bigger points of frustration, of course, was the failure of the Fund to conduct a Financial Sector Assessment for the United States, which evaded the process because of a loophole in Fund policies which require states to “volunteer” for FSAPs. Bernes’ main point of contention here is that the Fund’s staff, management and Board had not responded to the IEO’s recommendation in its evaluation of the FSAP to use “naming and shaming” as a way to get “systemically important countries” (read: the political and economic powerhouses, such as the United States) to be part of the FSAP process, thereby undermining the very purpose of FSAP (Bernes 2008: 8). He goes on to say that a key focus on future IEO reports should be to “examine the institution’s ability to ‘speak to power’, and highlight the risks of not doing so, when the members that pose the greatest systemic risk are also the largest shareholders” (Bernes 2008: 9).
References
Avant, D. D., Finnemore, M., & Sell, S. S. (Eds.). (2010). Who governs the globe? New York: Cambridge University Press.
Barnett, M., & Finnemore, M. (2004). Rules for the world: International organizations in world politics. Ithaca: Cornell University Press.
Bernes, T. (2007). The independent evaluation office: Lessons learned, emerging themes, and priorities for future work. Presentation to the Conference on Evaluating the IMF’s Exchange Rate Advice, Singapore, 4 December 2007.
Bernes, T. (2008). Evaluation, learning, and accountability: The IMF in a time of global crisis. Paper presented at the Fourth Evaluation Conference, Polish Agency for Enterprise Development, Warsaw, Poland, 18 October 2008.
Boorman, J. (2008). An agenda for reform of the International Monetary Fund (IMF). Dialogue on Globalization Occasional Paper No.38, January 2008. Available at http://www.nuso.org/upload/fes_pub/boorman.pdf.
Boughton, J., & Lombardi, D. (2009). Finance, development, and the IMF. Oxford: Oxford University Press.
Bretton Woods Project. (1998). Policing the policemen. Bretton Woods Project Update, 15 April 1998.
Buchanan, A., & Keohane, R. O. (2006). The legitimacy of global governance institutions. Ethics & International Affairs, 20(4), 405–437.
Cotarelli, C. (2005). Efficiency and legitimacy: Trade-offs in IMF governance. Working Paper 05/107. Washington, D.C.: IMF. Available at http://www.imf.org/external/pubs/ft/wp/2005/wp05107.pdf. Accessed 26 May 2010.
Easterly, W. (2006). Testimony before the U.S. Senate Foreign Relations Committee, Hearing on Multilateral Development Banks: Promoting effectiveness and fighting corruption, 28 March. Transcript available at http://www.senate.gov/~foreign/hearings/2006/hrg060328a.html. Accessed 28 June 2006.
Global Transparency Initiative (GTI). (2008). Right to information at the IMF: How to improve the fund’s transparency policy. Available at http://www.ifitransparency.org. Accessed 30 May 2010.
Grant, R., & Keohane, R. O. (2005). Accountability and abuses of power in world politics. American Political Science Review, 99(1), 29–43.
Griesgraber, J. M. (2008). Influencing the IMF. Available at http://www.new-rules.org/docs/influencing_the_imf.pdf. Accessed 5 February 2009.
Griesgraber, J. M. (2009). Reforms for major new roles of the International Monetary Fund? The IMF post-G20 summit. Global Governance, 15, 179–185.
Grigorescu, A. (2008). Horizontal Accountability in Intergovernmental Organizations. Ethics & International Affairs, 22(3), 285–308.
Gutner, T., & Thompson, A. (2010). The politics of IO performance: a framework. Review of International Organizations, 5(3). doi:10.1007/s11558-010-9096-z.
Hale, T. (2008). Transparency, accountability and global governance. Global Governance, 14, 73–94.
Hawkins, D., Lake, D., Nielson, D., & Tierney, M. (2006). Delegation and agency in international relations. New York: Cambridge University Press.
Helleiner, E., & Momani, B. (2008). Slipping into obscurity: crisis and institutional reform at the IMF. In A. Alexadroff (Ed.), Can the world be governed? Waterloo: WLU Press / CIGI.
Hurd, I. (2007). After anarchy: legitimacy and power in the United Nations Security Council. Princeton: Princeton University Press.
IEO. (2002). Evaluation of the prolonged use of IMF resources. Washington, D.C.: International Monetary Fund. Available at http://www.ieo-imf.org/pub/evalreports.html.
IEO. (2003a). IMF and recent capital account crises: Indonesia, Korea, Brazil. Washington, D.C.: International Monetary Fund. Available at http://www.ieo-imf.org/pub/evalreports.html.
IEO. (2003b). Evaluation of fiscal adjustment in IMF-supported programs. Washington, D.C.: International Monetary Fund. Available at http://www.ieo-imf.org/pub/evalreports.html.
IEO. (2004a). Report on the evaluation of poverty reduction strategy papers (PRSPs) and the poverty reduction growth facility (PRGF). Washington, D.C.: International Monetary Fund. Available at http://www.ieo-imf.org/pub/evalreports.html.
IEO. (2004b). Report on the evaluation of the role of the IMF in Argentina, 1991-2001. Washington, D.C.: International Monetary Fund. Available at http://www.ieo-imf.org/pub/evalreports.html.
IEO. (2005a). Report on the evaluation of the IMF's approach to capital account liberalization. Washington, D.C.: International Monetary Fund. Available at http://www.ieo-imf.org/pub/evalreports.html.
IEO. (2005b). Evaluation report: IMF support to Jordan, 1989-2004. Washington, D.C.: International Monetary Fund. Available at http://www.ieo-imf.org/pub/evalreports.html.
IEO. (2005c). Report on IMF technical assistance. Washington, D.C.: International Monetary Fund. Available at http://www.ieo-imf.org/pub/evalreports.
IEO. (2006). An evaluation of the IMF’s multilateral surveillance. Washington, D.C.: International Monetary Fund. Available at http://www.ieo-imf.org/pub/evalreports.html.
IEO. (2007). IEO evaluation of IMF exchange rate policy advice, 1999–2005. Washington, D.C.: International Monetary Fund. Available at http://www.ieo-imf.org/pub/evalreports.html.
IEO. (2008). Governance of the IMF: An evaluation. Washington, D.C.: International Monetary Fund. Available at http://www.ieo-imf.org/pub/evalreports.html.
IEO. (2010). IMF interactions with member countries. Washington, D.C.: International Monetary Fund. Available at http://www.ieo-imf.org/pub/evalreports.html.
IEO. Annual Reports 2007–09. Washington: International Monetary Fund. Available at http://www.ieo-imf.org.
IMF. (2007). Periodic monitoring report on the status of board-endorsed IEO recommendations and management implementation plans. Prepared by the Policy Development and Review Department, 3 December 2007.
IMF. (2008). Periodic monitoring report on the status of board-endorsed IEO recommendations and management implementation plans. Prepared by the Policy Development and Review Department, December 2008.
IMF Archives. BUFF/94/10. The chairman’s remarks at the conclusion of the discussion on the work program, Executive Board Meeting 94/97. 11/7/1994.
IMF Archives. BUFF/95/125. Statement by the managing director on the development of the evaluation functions in the fund executive board meeting. 1/10/1996.
IMF Archives. BUFF/00/131. Concluding remarks by the chairman: Making the fund’s independent evaluation office operational. Executive Board Meeting 00/81, 8/3/2000.
IMF Archives. EBAP/92/166. Establishing an evaluation office in the fund. 12/17/1992.
IMF Archives. EBAP/98/64. External evaluation of fund surveillance: Report by the evaluation group of executive directors. 6/24/1998.
IMF Archives. EBAP/00/20. Review of experience with evaluation in the fund. Background paper prepared for the IMF Executive Board, 2000.
IMF Archives. EBAP/00/29 Supplement 1. Independent evaluation in the fund and in other institutions. Background paper prepared for the IMF Executive Board by the Office of Internal Audit and Inspection, 3/14/2000.
IMF Archives. EBAP/00/84. Making the fund’s independent evaluation office operational. Paper prepared by the Evaluation Group of Executive Directors, 7/19/2000.
IMF Archives. EBAP/00/105. Executive board report to the IMFC on the establishment of the independent evaluation office (EVO) and its terms of reference. 9/12/2000.
IMF Archives. EBM/93/9. Minutes of the executive board meeting. 1/22/1993.
IMF Archives. EBM/96/16. Minutes of the executive board meeting, 2/26/1996.
IMF Archives. PR/06/67. First deputy managing director Anne Krueger’s statement to the IMF executive board on Montek Ahluwalia. 6/25/2004.
IMF Archives. SUR/95/35. Mexico-report on fund surveillance, 1993–94. 4/7/1995. [Chaired by Sir Alan Whittome, also known as the Whittome Report].
Kapur, D., & Webb, R. (2006). Beyond the IMF. G-24 Policy Brief No.1. Available at http://www.g24.org/briefs.htm. Accessed 2 July 2010.
Kenen, P. B. (2004). Appraising the IMF’s performance: A review of the first three studies by the new independent evaluation office. Finance & Development, 41(1), 41–45, March.
Lissakers, K. (2004). Blunt approach does the trick. Finance & Development, 41(1), 46–47, March.
Lissakers, K., Husain, I., & Woods, N. (2006). Report of the external evaluation of the independent evaluation office. 29 March 2006. Washington, D.C.: International Monetary Fund.
Milleron, J.-C. (2004). Enhancing the learning culture. Finance & Development, 41(1), 48–49, March.
Momani, B. (2010). IMF rhetoric on reducing poverty and inequality. In J. Clapp & R. Wilkinson (Eds.), Global governance, poverty and inequality (pp. 71–89). London: Routledge.
Mussa, M. (2007). IMF surveillance over China’s exchange rate policy. Paper presented at the Conference on China’s Exchange Rate Policy, Peter Institute, October 19, 2007.
Park, S., & Vetterlein, A. (Eds.). (2010). Owning development. Cambridge: Cambridge University Press.
Picciotto, R. (2003). The logic of renewal: Evaluation at the World Bank, 1992–2002. In P. Grasso et al. (Eds.), World Bank operations evaluation department: The first 30 years (pp. 61–71). Washington: World Bank.
Polak, J. (1998). IMF study group report: Transparency and evaluation; Report and Recommendations by a Special Study Group convened by the Center of Concern, April 1998.
Saxton, J. (Chairman). (1998). IMF financing: A review of the issues. Joint Economic Committee Study of the United States House of Representatives, March 1998. Available at http://www.house.gov/jec/imf/imf/htm.
Steinward, M. C., & Stone, R. W. (2008). The International Monetary Fund: a review of the recent evidence. Review of International Organizations, 3(2), 123–49.
Vreeland, J. (2003). The IMF and economic development. New York: Cambridge University Press.
Weaver, C. (2008). Hypocrisy trap: The World Bank and the poverty of reform. Princeton: Princeton University Press.
Welch, C. (2004). Credible start, untested impact. Finance & Development, 41(1), 50–51, March.
Weiss, M. A. (2009). The global financial crisis: the role of the International Monetary Fund (IMF). Congressional Research Service Report for Congress, RS22976. Available at http://www.crs.gov. Accessed 1 May 2010.
Wood, A., & Welch, C. (1998). Policing the policemen—the case for an independent evaluation mechanism for the IMF. Produced by the Bretton Woods Project and Friends of the Earth US, April 1998.
Acknowledgments
For advice on earlier drafts of this paper, I would like to thank Tamar Gutner, Robert Keohane, Bessma Momani, Jon Pevehouse, Len Seabrooke, Randy Stone, Alex Thompson, Antje Vetterlein and Rawi Abdelal. I would especially like to thank John Hicklin and Tom Bernes at the Fund IEO, Premala Isaac at the Fund archives, and the many other Fund staff who graciously agreed to interviews. Any shortcomings and errors are entirely my own.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Weaver, C. The politics of performance evaluation: Independent evaluation at the International Monetary Fund. Rev Int Organ 5, 365–385 (2010). https://doi.org/10.1007/s11558-010-9094-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11558-010-9094-1
Keywords
- International Monetary Fund
- International organizations
- Performance evaluation
- Accountability
- Transparency
- Organizational learning