Evaluating Complex Development Programs

Abstract

Evaluators express preferences for certain methods over others. This chapter highlights the debate and assumptions underlying these preferences. There are debates on which methods represent the “gold standard” for evaluation. The key point discussed in this chapter is that the “gold standard” for evaluation methodology is Appropriateness to evaluation questions and the contextual realities of complex programs.

In God We Trust—All Others Must Have Credible Evidence

Donaldson et al. 2009, p. 2

Neither the quantitative hook set of for the big fish nor the qualitative net scaled for the little fish adequately captures life in the most seas. We need a paradigm to help us become scuba divers

Dattu 1994, pp. 61–70

It has been said that economics is a box of tools. But we shall have to resist the temptation of the law of the hammer, according to which a boy, given a hammer, finds everything worth pounding, not only nails but also Ming vases. We shall have to look, in the well known metaphor, where the [lost]key was dropped rather than where the light happens to be. We shall have to learn not only how to spell ‘banana’ but also when to stop. The professionals, whom a friend of mine calls ‘quantoids’ and who are enamored of their techniques, sometimes forget that if something is not worth doing, it is not worth doing well.

Streeten 2002, p. 110 quoted in Debrah Eade 2003, p. ix

With in any given approach, how do we choose the methods we use? And what factors influence how we apply them? These choices are not always clear or conscious. If we are really honest about it, many of us probably often choose methods because they are familiar and draw on skills that we feel confident about; or because we perceive that the funders require them……realistically, most ‘practitioners’ operate within constraints of time and resources that affect the choices we make; we also operate within political contexts that shape our choices

Rowlands 2003, p. 3.