Replication of Quantitative Work in Development Studies: Experiences and Suggestions

Abstract

Replication¹ is seen as a key characteristic of natural science (Collins, 1985; Jasny et al., 2011); observations, especially those employing complex instruments, and experiments need to be repeated, and statistical analyses be scrutinized, before results gain credibility. This is not the case in social sciences; social science data are seldom re-produced,² or re-analysed to check the original calculations, or analysed using alternative perspectives or frameworks.³ Hence, it is not clear that quantitative social science can advance in the same way as natural science. Nevertheless, there have often been calls among quantitative social scientists for replication (Frisch, 1933; Dewald et al., 1986; King, 1995; Gleditsch and Metelits, 2003; McCullough and Vinod, 2003; Pesaran, 2003; Bernanke, 2004; Freeze, 2007; McCullough and McKitrick, 2009; Burman et al., 2010), especially of computational⁴ studies (Peng, 2011). The proposed benefits include: full understanding of the computations⁵ and estimations, including for pedagogy; credibility; a basis for further work either assessing the robustness of the study to alternative variable constructions, model specifications, estimation methods or software, or extending or building on it; and an audit function — to identify and deter fraud and/or over-interpretation of the data (McCullough et al., 2006).