Abstract
Articulating evaluation questions is a deceptively challenging aspect of designing evaluation plans in media development. This chapter examines the creation of the “menu of common questions” tool from the IDEAS Guide, with a focus on dialogue and two-way communication aspects of governance and accountability programs. The chapter then discusses how advances in complexity thinking and evaluation can be applied to questions of impact and causal analysis in media assistance. While there is a growing body of evaluations using randomized control trials and quasi-experimental designs in media development, alternative options for systematic causal and contribution analysis are under-used options. While there is great potential for improving impact evaluation practices using these techniques, it is important not to lose sight of the importance of learning through evaluation, which is ultimately the way in which programs benefit from evaluation and become more effective.
This is a preview of subscription content, log in via an institution to check access.
Notes
- 1.
This chapter benefits greatly from my experience of working in close collaboration with Professor Patricia Rogers, director of the Better Evaluation website. Her influence extends beyond books and articles cited here. BetterEvaluation.org is a useful reference for following up the conceptual discussions outlined in this chapter with more practical advice and options.
References
Abraham Dowsing, K., & Hoijtink, L. (2014). Proving the impact of media on behavior change. http://www.bbc.co.uk/blogs/bbcmediaaction/entries/69e77b7e-105e-3702-8189-c8cb03f4fd5f. Accessed 23 March 2017.
Alcorn, J., Chen, A., Gardner, E., & Matsumoto, H. (2011). Mapping donor decision making on media development. An overview of current monitoring and evaluation practice. http://www.mediamapresource.org/wp-content/uploads/2011/04/DonorDecionmaking.MediaMap.pdf. Accessed 22 Jan 2013.
Alsop, R., & Heinsohn, N. (2005). Measuring empowerment in practice: Structuring analysis and framing indicators. World Bank Policy Research Working Paper Series. http://elibrary.worldbank.org/doi/pdf/10.1596/1813-9450-3510. Accessed 14 March 2014.
Banda, F., Berger, G., Panneerselvan, A. S., Nair, L., & Whitehouse, M. (2009). How to assess your media landscape: A toolkit approach. http://gfmd.info/images/uploads/toolkit.doc. Accessed 17 Aug 2012.
BBC Media Action. (2012). Conceptualising accountability: An approach to measurement. Briding Theory and Practice: Research Dissemination Series. http://downloads.bbc.co.uk/rmhttp/mediaaction/pdf/Conceptualising_accountability.pdf. Accessed 15 Sept 2013.
BBC Media Action Policy and Learning. (2016). Inside a randomized control trial: Insights from Bangladesh. http://www.bbc.co.uk/blogs/mediaactioninsight/entries/29ecc68c-9778-493e-8ee8-f0713e2ad16d. Accessed 23 March 2017.
Byrne, A., & Vincent, R. (2012). Innovations in the evaluation of social change communication for HIV and AIDS. In R. Obregon & S. Waisbord (Eds.), (pp. 288–308). Oxford: Wiley-Blackwell.
Cracknell, B. E. (2000). Evaluating Development Aid: Issues, Problems and Solutions. New Delhi, Thousand Oaks and London: Sage Publications.
Estrella, M. (2000). Learning from change. In M. Estrella, J. Blauert, D. Campilan, J. Gaventa, J. Gonsalves, I. Guijt, D. Johnson & R. Ricafort (Eds.), Learning from change: Issues and experiences in participatory monitoring and evaluation (pp. 1–15). London and Ottawa: Intermediate Technology Publications Ltd, and the International Development Research Centre.
Fetterman, D. M. (2001). Empowerment evaluation: The pursuit of quality. Advances in Program Evaluation, 7, 73–106. doi:10.1016/S1474-7863(01)80067-3.
Funnell, S. C., & Rogers, P. J. (2011). Purposeful program theory: Effective use of theories of change and logic models. San Francisco: Wiley.
Glouberman, S., & Zimmerman, B. (2002). Complicated and complex systems: What would successful reform of medicare look like? http://c.ymcdn.com/sites/www.plexusinstitute.org/resource/collection/6528ED29-9907-4BC7-8D00-8DC907679FED/ComplicatedAndComplexSystems-ZimmermanReport_Medicare_reform.pdf. Accessed 25 Feb 2014.
Goroka workshop notes. (2014).
International Initiative for Impact Evaluation. (2010). Impact evaluation glossary. Journal of Development Effectiveness, 2(1), 187–194. doi:10.1080/19439340903580737.
Jacobson, T. L., Pan, L., & Joon Jun, S. (2011). Indicators of citizen voice for assessing media development: A communicative action approach. In M. E. Price, S. Abbott & L. Morgan (Eds.), Measures of press freedom and media contributions to development: Evaluating the evaluators (pp. 281–304). New York: Peter Lang Publishing.
Johnson, H., & Wilson, G. (2009). Learning for development. London and New York: Zed Books.
Lennie, J., & Tacchi, J. (2013). Evaluating communication for development: A framework for social change. Oxford: Earthscan, Routledge.
Manyozo, L. (2012). Media, communication and development: Three approaches. New Delhi, Thousand Oaks, London and Singapore: SAGE.
Mayne, J. (2012). Contribution analysis: Coming of age? Evaluation, 18(3), 270–280. doi:10.1177/1356389012451663.
Moehler, D. C. (2013). Democracy, governance and randomised media assistance. Bridging theory and practice. http://downloads.bbc.co.uk/rmhttp/mediaaction/pdf/Democracy_gov_working_paper_4.pdf. Accessed 22 Sept 2013.
Myers, M., Woods, N., & Odugbemi, S. (2005). Monitoring and Evaluating Information and Communication for Development (ICD) programs: Guidelines. http://web.idrc.ca/uploads/user-S/11592105581icd-guidelines.pdf. Accessed 28 Sept 2012.
Noske-Turner, J. (2015). 10 years of evaluation practice in media assistance: Who, when, why and how? Nordicom Review, 36, 41–56.
Noske-Turner, J., Horst, H., & Tacchi, J. (2016). IDEAS guide: A guide for media and communication projects. http://www.betterevaluation.org/sites/default/files/IDEAS%20Guide%20for%20web.pdf. Accessed 13 March 2017.
OECD-DAC. (2002). Glossary of key terms in evaluation and results based management. http://www.oecd.org/dataoecd/29/21/2754804.pdf. Accessed 16 July 2012.
Patton, M. Q. (2008). Utilization-focused evaluation. Thousand Oaks, London, New Delhi, Singapore: Sage Publications.
Patton, M. Q. (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York: The Guilford Press.
Patton, M. Q. (2012). A utilization-focused approach to contribution analysis. Evaluation, 18(3), 364–377. doi:10.1177/1356389012449523.
Ramalingam, B. (2013). Aid on the edge of chaos: Rethinking international cooperation in a complex world. Oxford: Oxford University Press.
Raman, V. V., & Bhanot, A. (2008). Political crisis, mediated deliberation and citizen engagement: A case study of Bangladesh and Nirbachoni Sanglap. http://downloads.bbc.co.uk/worldservice/pdf/wstrust/Bangladesh_Sanglap_Governance.pdf. Accessed 13 Sept 2012.
Roberts, M. (2011). The media map project: Cambodia: Case study on donor support to independent media 1990–2010. Media map, country case studies. http://www.mediamapresource.org/wp-content/uploads/2011/04/Cambodia_web.pdf. Accessed 5 December 2012.
Schedler, A., Diamond, L. J., & Plattner, M. F. (1999). The self restraining state: Power and accountability in new democracies. Boulder, London: Lynne Rienner Publishers.
Sekei, L. H., Lugoe, N. B., & Thulstrup, K. M. (2016). Take the pill, discuss the issues and act: Using RCTs, PAR and FGDs to evaluate a media entrepreneurship programme in Tanzania. In N. Wildermuth & T. Ngomba (Eds.), Methodological reflections on researching communication and social change (pp. 183–206). Cham: Springer.
Tacchi, J. (2015). The role of “stillness” and “nostalgia” in sustainable development: Asking different questions in communication for development. In A. H. Hansen, O. Hemer & T. Tufte (Eds.), Memory on trial: Media, citizenship and social justice (pp. 25–34). Berlin and London: Lit Verlag.
USAID (2011). Evaluation learning from experience: USAID Evaluation policy. http://www.usaid.gov/sites/default/files/documents/1868/USAIDEvaluationPolicy.pdf. Accessed 13 Feb 2014.
Warnock, K. (2011). Driving change through rural radio debate in Uganda. http://www.panosea.org/docs/pub/RRD%20Evaluation%20Report.pdf. Accessed 13 Sept 2012.
Wong, G. (2008). Creating democratic value: Evaluating efforts to promote democracy abroad: Approaches to the evaluation of international democracy assistance. Centre for the study of democracy: Queens University. http://www.queensu.ca/csd/publications/Eval_of_Intl_Democracy.pdf. Accessed 13 Feb 2012.
Woolcock, M. (2009). Toward a plurality of methods in project evaluation: A contextualised approach to understanding impact trajectories and efficacy. Journal of Development Effectiveness, 1(1), 1–14. doi:10.1080/19439340902727719.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2017 The Author(s)
About this chapter
Cite this chapter
Noske-Turner, J. (2017). Questions and Answers, and that Vexed Question of Impact. In: Rethinking Media Development through Evaluation. Palgrave Studies in Communication for Social Change. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-319-58568-0_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-58568-0_5
Published:
Publisher Name: Palgrave Macmillan, Cham
Print ISBN: 978-3-319-58567-3
Online ISBN: 978-3-319-58568-0
eBook Packages: Literature, Cultural and Media StudiesLiterature, Cultural and Media Studies (R0)