Introduction

Are Parliamentarians the new avant-garde in the global fight against malnutrition? In November 2018, the UN Food and Agricultural Organisation hosted the first Global Parliamentary Summit against Hunger and Malnutrition, convening over 200 parliamentary representatives. This was further followed by efforts to set up a Parliamentary Alliance for Food and Nutrition in East Africa in April 2019. UNICEF has hosted regional parliamentary conferences on nutrition in Viet Nam, Namibia and Burkina Faso since 2014. Additionally, the global Scaling Up Nutrition movement reported that in 2017, 36 SUN countries engaged MPs in order to “cement nutrition as a national priority”, and has sought to mobilise speakers of parliaments through the Inter-Parliamentary Union (Scaling Up Nutrition n.d.).

The mobilisation of parliamentariansFootnote 1 is part of a larger shift in the framing of progress on nutrition as an outcome of political processes, emerging from the 2007/2008 global food crisis. Advocacy with parliamentarians and policymakers is seen to play a critical role in bringing nutrition higher up on policy agendas (Pelletier et al. 2013; Shiffman 2016, 2010, 2007; Walt and Gilson 2014), strengthening budgetary allocations and oversight, and generating and sustaining high levels of political commitment to deliver on these agendas (FAO et al. 2013; Foresight 2011; Gillespie et al. 2013). When armed with strong evidence, policy advocates can speak truth to power. Accordingly, international donors, philanthropical foundations, (inter)national NGOs and civil society groups have made significant efforts to align nutrition goals, approaches and interventions. They have also raised awareness about the underlying and immediate, direct and indirect, human, economic and other consequences of malnutrition with key policy stakeholders, and advocated for particular context specific policy solutions (Pelletier et al. 2013).

As the hopes invested in parliamentary and policy advocacy grow, little is known, as yet, about its impact. However, donors who are under increasing domestic pressure to showcase the developmental benefits of taxpayer funded aid are increasingly demanding that advocacy organisations demonstrate impact. Demonstrating impact may, however, be more easily demanded than delivered on. While a handful of studies note the important role of policy advocacy in informing nutrition policy and programme and reforming institutional architectures, such as in Uganda, Bangladesh, Viet Nam (Pelletier et al. 2013) and Peru (Mejia Acosta and Haddad 2014), there are significant methodological and practical barriers to causally establishing the association between advocacy and intended change in policy, law, public spending or implementation (Nathan et al. 2002; Naeve et al. 2017).

The relation between advocacy efforts and policy outcome is usually uncertain and non-linear. Observed impacts may not be easily attributed to advocacy initiatives, as political landscapes are fluid, complex, and shaped by various actors with diverse interests, advocacy strategies and interventions. In the context of working with political elites, such as MPs, this complexity is typically magnified by the social distance between advocate and target, as well as the lack of transparency of the process by which political elites make decisions. Advocacy also rarely entails one-off effort, but rather involves iterative, ongoing actions at multiple levels and sites, to constitute a complex intervention.

The last few decades have witnessed significant debates about rigour in evaluations of complex interventions (e.g. Schmitt and Beach 2015; Byrne 2013; Yin 2013; Vellema et al. 2013). Methodological innovations have focused on theory based evaluation, for instance in contribution analysis (Mayne 2008) and realist evaluation (Pawson and Tilley 1997), and on the application of case studies in impact evaluation (Stern et al. 2013; Yin 2013). While these innovations are increasingly adopted in international development studies, application to policy advocacy remains scarce despite theoretical promise (see: Naeve et al. 2017; Kane et al. 2017).

Accordingly, this paper aims to make an original contribution to these literatures, by innovatively applying a combined Contribution Analysis and Process Tracing (CAPT) procedure (Befani and Mayne 2014) to evaluate the impact of a specific case of parliamentary advocacy for nutrition in Tanzania. We argue that the CAPT procedure can be valuable in determining impact of advocacy, by considering the configurations of factors that have causal force, and by interrogating the strength of evidence used in evaluations. However, we also note some challenges in applying CAPT. Furthermore, we seek to inform current global efforts that are engaging parliamentarians as nutrition champions. To this end, we identify four interrelated factors of consideration to parties seeking to fund, embark on or evaluate parliamentary advocacy.

Following this introduction, the remainder of the article is structured as follows: section two discusses current thinking on systematic qualitative evaluation and its applicability to assess the impact of advocacy. This is followed by an introduction of the case study and an outline of materials and methods, followed by a results section which presents a step-by-step CAPT analysis. Finally, we discuss findings and conclusions.

Theory: systematic qualitative evaluation for advocacy impact

Although there is a well-established body of literature theorising, investigating and making causal claims about policy advocacy, evaluating such claims is fraught with numerous methodological challenges. First, policy advocacy lacks a common definition and shared understandings of its forms, elements, dynamics and markers are limited (Gen and Wright 2013; Mellinger 2014).Footnote 2 Lack of agreement concerning objectives, concepts and methods impedes the rigorous assessment of policy advocacy impacts (Naeve et al. 2017; Nathan, Rotem, and Ritchie 2002). Moreover, advocacy evaluators are often oriented towards aiding and enhancing the advocacy initiative, and less towards establishing causality and strength of effect (Naeve et al. 2017, p. 9).

While political scientists and development economists have long relied on econometric models to rigorously test causal claims (Voors 2018, 80), popular experimental and quasi-experimental approaches are typically unsuited for assessing policy advocacy. Because they depend on the construction of counterfactuals and controlled implementation environments these methods cannot be applied to the fluid political contexts within which policy advocacy takes place (Naeve et al. 2017). Indeed, the relation between advocacy efforts and policy outcome is usually uncertain and non-linear, and observed impacts may not be easily attributed to particular advocacy initiatives. This is because political landscapes are typically populated by a large number of actors with diverse interests, advocacy strategies and interventions. Moreover, timeframes between advocacy interventions and policy impact can be long and indeterminate.

Evaluation methodologies are becoming more sensitive to this complexity. Hence, a now commonly held view considers the world as comprised of complex systems that mediate causal effects of interventions in non-linear ways, with multiple causal paths that can lead to the same outcome (Mahoney and Goertz 2006; Byrne 2013). Here, causality is not located in a single determinant, but rather viewed as operating through configurations of causal factors. Consequently, explorations of the impact of a complex intervention such as advocacy requires understanding interactions between intervention, context, and the people involved in these (cf. Byrne 2013; Yin 2013; Michaud-Létourneau, Gayard, and Pelletier 2019).

Theory based evaluations seek to analyse a theorised causal mechanism that is explicitly or implicitly used in a programme of activities and interventions. They usually compare a causal chain from inputs to outcomes and impact against a previously developed programme theory (Schmitt and Beach 2015). Contribution Analysis (CA) is considered to be amongst the most promising theory based evaluation methods (Naeve et al. 2017). CA has been widely applied, for instance to evaluate European Union development aid, agriculture, employment and governance programmes (Delahais and Toulemonde 2012), to explore the impact of research on Canadian tobacco control policy processes (Riley et al. 2018), and to assess policy advocacy for nutrition in seven countries (Michaud-Létourneau et al. 2019).

In contribution analysis, causality is explored through a theory of change, and by illustrating how activities and interventions may constitute a chain of results which ultimately leads to the observed outcome (Mayne 2008). CA actively takes account of the interaction between intervention and context, and how various stakeholders mediate the relation between intervention and observed outcome (Vogel 2013). However, while CA theorises development programme causal mechanisms in order to make stronger causal inference, actual causal linkages between parts of these mechanisms are often left unstudied (‘black-boxed’), and the strength of the evidence brought to bear on the case is typically not evaluated (Schmitt and Beach 2015). This is, however, a strong suit of the Process Tracing (PT) method.

PT is designed to be applied in complex contexts where competing causal explanations may be found for observed outcomes (Beach 2013; Collier 2011). Within the remit of International Development, PT has, for instance, been used to analyse advocacy programmes, including an evaluation of Oxfam GB’s campaign for Universal Free Health Care in Ghana (Bryce-Stedman 2013; Oxfam GB 2013), for understanding the impact of complex aid instruments such as budget support programmes (Schmitt and Beach 2015), and for assessing the policy impact of political commitment metrics for nutrition (te Lintelo et al. 2019).

PT involves devising a causal mechanism involving a series of interlocking parts that specify all the links between causes and outcomes. Such mechanisms are increasingly conceptualised as a system that transfers causal forces (e.g. Beach 2013). Hence it is the specific configuration of parts, within a given context, that together produce the outcome or impact. A clearly articulated causal mechanism can be empirically assessed using a series of tests that critically interrogate the inferential strength of confirmatory and dis-confirmatory evidence. Such tests seek to maximise uniqueness; they aim to find evidence that has a very low probability of existing in the absence of the mechanism and that would discount other potential explanations for how the observed outcome might have come about. Other tests also seek to optimise certainty; they aim to assess confirmatory evidence that needs to be minimally in place to have confidence that the proposed causal mechanism functions as expected, and dis-confirmatory evidence that would destroy having such confidence (Schmitt and Beach 2015).

Summing up, both CA and PT seek to evaluate programme outcomes in a context of multiple competing, and sometimes interrelated causal factors, where many activities, actions or events may explain observed outcomes. Both also adopt an inferential causal logic. But while Schmitt and Beach (2015) propose the superiority of Process Tracing to Contribution Analysis by noting its ability to unpack causality and provide better tests for assessing the strength of evidence, Befani and Mayne (2014) argue that Process Tracing, unlike CA, may underplay the roles of context and alternative explanations (rival hypotheses). Befani and Mayne (2014) hence suggest combining the best of both worlds, to propose a synthesis Contribution Analysis and Process Tracing (CAPT) procedure. Accordingly, in the absence of further practical applications this paper originally employs a CAPT analysis to address the following research question: What is the impact of policy advocacy with and towards Parliamentarians on the incorporation of nutrition into political party manifestos for the 2015 general election in Tanzania?

The Case

Tanzania faces, despite recent improvements, continued high prevalence of malnutrition. Malnutrition constitutes a significant health and development challenge in Tanzania. Maternal anaemia rates remain close to 40%. At the same time, rates of overweight and obesity are quickly increasing, affecting 15% of men and 24% of adult women. Stunting rates, as a measure of chronic food insecurity have seen significant improvements over the last decade yet stunting still affects every third child under 5 years of age (Government of Tanzania 2016).

The Partnership for Nutrition in Tanzania (PANITA) convenes over 300 civil society organisations in the country and leads the civil society network for the global Scaling Up Nutrition movement in Tanzania.

Its advocacy seeks to place and retain nutrition high on policymakers’ agendas, to foster enabling environments within which nutrition outcomes can improve. In 2012, PANITA started hosting a secretariat for the Parliamentary Group on Child Rights, Food Security and Nutrition (PG), an informal cross-party grouping of about 40 MPs championing nutrition. It jointly devised a strategic advocacy plan the following year. These efforts capitalised on political reforms adopted in 2007 which had encouraged greater parliamentary autonomy, debate and strengthened its law-making, budget scrutiny and supervisory and investigatory roles (Prabhu 2014). In 2012, PANITA also commenced a research and advocacy collaboration with the UK-based Institute of Development Studies (IDS) as part of the Hunger and Nutrition Commitment Index (www.hancindex.org) project, aiming to advance accountability for nutrition.Footnote 3 Tanzania-based food and nutrition security experts noted that political party manifestoes did not give adequate attention to nutrition (te Lintelo and Lakshman 2015). This was significant, because manifestos enduringly provide Tanzanian governments with strategic policy direction (Selbervik 2006).Footnote 4

Accordingly, the 2015 general elections were soon considered as a strategic opportunity for nutrition advocacy. Consequently, working with the Parliamentary Group, and with support from IDS and others, PANITA commenced a parliamentary advocacy campaign aiming to substantively enhance the presence of nutrition within political party manifestos for the two largest parties.Footnote 5 This advocacy process is the key focus of our impact evaluation.

Materials and Methods

In this study, researchers took a ‘developmental evaluator’ role (Coffman 2009; Patton 2011). Researchers had a long-term, partnering relationship with Tanzanian civil society partners engaged in nutrition policy advocacy, supporting the parliamentary advocacy process as it unfolded. This enabled collection of secondary and primary data over a long time period and from different sources, as well as an increased understanding of the nuances of working in this context.

We analysed the 2010 and 2015 election manifestos for Chama Cha Mapinduzi (CCM), the political party that has governed the country since the introduction of multi-party democracy in 1990. We also analysed the manifesto for the Umoja wa Katiba ya Wananchi (UKAWA) coalition competing in the 2015 general elections.Footnote 6 This was compared the Chama cha Demokrasia na Maendeleo (CHADEMA) 2010 manifesto, as this party led the UKAWA coalition.Footnote 7 The 2015 documents were analysed for the presence of 114 nutrition related search-terms developed by a leading Tanzanian nutrition expert, fluent in kiSwahili (the languages of the reviewed documents) and English, using simple frequency counts. The 2010 documents were analysed adopting the same search terms. Moreover, we conducted stakeholder mappings of Tanzania’s nutrition policy environment to identify key advocacy actors and their targets, through desk-based reviews supplemented with key informant interviews. This was complemented by key informant interviews with four MPs, to take stock of their efforts to promote the nutrition recommendations, and with 12 senior government officials and 11 other stakeholders from civil society and UN agencies based in Tanzania, for a deeper understanding of the nutrition advocacy environments. We further reviewed published and unpublished HANCI project documents, such as terms of references of scheduled activities, detailed participant observation records and PANITA reports about advocacy engagements.

Next, we applied the CAPT analytical framework. In Contribution Analysis, causality is inferred by assessing four conditions (Befani and Mayne 2014; Mayne 2008) to ascertain that: the intervention programme is based on a viable and well-reasoned theory of change; intervention activities were implemented as planned; the theory of change can be verified by the evidence of expected outcomes; and other assumptions on factors and risks influencing the programme were assessed for their ability to make a significant contribution and included in the theory of change were this was found to be the case.

PT can fruitfully complement Contribution Analysis by weighing the inferential strength of each piece of evidence brought to bear upon potential causal factors in the causal mechanism, using a series of tests (Befani and Mayne 2014; Beach and Pedersen 2013). The objective of applying these tests is to empirically rule out certain causal factors (shown as a risk or assumption in a CA theory of change) that may have logical appeal, while also enhancing confidence in the plausibility that other causal factors have contributed to the observed outcome. Process Tracing thus involves the development and application of ‘hoop tests’ to ascertain the validity of consecutive components that comprise a causal mechanism. This encompasses, firstly, listing evidence that would be expected to be observed (i.e. necessary) for a component to be plausibly deemed in place. Subsequently, confirming and disconfirming evidence is gathered and evaluated using the hoop tests. Hoop tests aim to enhance certainty about the validity of a mechanism and can also be used to discount some potential alternative explanations. Next, a further set of tests are applied. ‘Smoking gun’ tests seek to increase confidence in the causal mechanism, by, where possible, identifying highly unique evidence. Such evidence is very unlikely to occur if the proposed mechanism is not in place and cannot be attributed to potential rival causal explanations.

Accordingly, we start by conducting the following analytical steps: (step 1) Set out the cause–effect issue to be addressed; (step 2) Develop the postulated ToC and risks to it, including other influencing factors; (step 3) Gather the existing evidence on the ToC; (step 4) Assemble and assess the contribution claim; (step 5) Gather new evidence from the implementation of the intervention and finally, (step 6) Revise and strengthen the contribution story.

Results

This section presents a step-by-step CAPT analysis.

Step 1: Set Out the Cause–Effect Issue to be Addressed

The key question we seek to examine is whether, and in what ways, we can attribute any inclusion of nutrition in political party manifestos of CCM and UKAWA in 2015 to the advocacy of PANITA, working in coalition with the Parliamentary Group on Child Rights, Food Security and Nutrition Security, UN-REACH, IDS and occasionally supported by other actors. Accordingly, the paper seeks to analyse, firstly, whether there is evidence that political party manifestos for the 2015 general elections incorporated nutrition as a topic of concern. Secondly, it evaluates if the PANITA led advocacy contributed to this outcome.

Step 2: Develop the Postulated ToC and Risks to It, Including Other Influencing Factors

The PANITA led group espoused an implicit theory of change that can be framed as:

Civil society advocacy messages on the significance of nutrition for development in Tanzania, when carefully undergirded with empirical evidence, and presented in suitable formats by reputable actors and through appropriate channels to policy elites involved in the shaping and drafting of the political party manifestos, enhance the chance that nutrition is incorporated in political party manifestos. Having nutrition included in party manifestos will lead government to advance nutrition policy change, enhance allocation of financial resources, and strengthen the implementation of nutrition interventions as per the manifesto. This, in turn, will help drive reducing malnutrition in Tanzania.

Figure 1 shows the theory of change and its proposed connections between activities, outputs, immediate outcomes, intermediation outcomes and ultimate impacts. The intervention programme envisaged three kinds of activities, oriented towards: (1) presenting evidence based advocacy messages to policy elites (2) developing a set of Nutrition Recommendations (NR); (3) advocacy activities promoting their uptake amongst MPs and party elites; and (4) training journalists to present nutrition as an issue of political commitment in media reports.

Fig. 1
figure 1

Theory of change

Full size image

Key risks and assumptions can be identified for each part of the chain between interventions and ultimate impacts. Figure 1 offers illustrative examples for assumptions relevant to each part of the theory of change. However, we focus on evaluating parliamentary advocacy, and thus emphasise assumptions between immediate and intermediate outcomes, as the later part of the theory of change is beyond the scope of the project. Accordingly, we specifically ask, How could policy advocates leverage a set of newly crafted nutrition recommendations to influence the content of political party manifestos? Our analysis identified three critical risks and assumptions in this respect:

  1. 1.

    Targeting: The right MPs and political party members are targeted. Advocacy activities present the nutrition recommendations to MPs and party members who are either directly responsible for the manifesto drafting, or who know those who are on the drafting committee and are willing to share the nutrition recommendations with them. This in turn, assumes that members of the committees drafting manifestos are known to advocacy actors. Tanzanian civil society groups are independently able to identify, and are adequately resourced to engage targeted MPs, and broader party leadership involved in manifesto drafting from leading parties.

  2. 2.

    Circulation: MPs and political party members targeted by advocacy receive the nutrition recommendations and will be persuaded and motivated to circulate these to peers. This would depend on the nutrition recommendations being appropriate, communicated in the right manner, and trusted.

  3. 3.

    Timing: Advocacy engagement (under 2 above) would need to happen at an appropriate time to allow NR to be included in the manifestos (not after the text had been drafted or before drafting had started).

We interrogate these assumptions and risks later in the paper (Step 5).

Step 3: Gather the Existing Evidence on the ToC

The next step in our analysis entails assessing the evidence regarding outputs, immediate and intermediate outcomes. We offer a quick sketch, and then deepen the discussion in following sections.

The PANITA-IDS collaboration commenced by joint review of potential uses of HANCI knowledge products for advancing accountability for nutrition in Tanzania. Workshops in July and November 2013 reviewed index rankings, scorecards and primary research findings, to translate and draft these into four context specific policy advocacy messages about political commitment for nutrition. One read,’Experts surveyed by HANCI note that there is no strong commitment to hunger and nutrition as key development issues in the current manifestoes (sic) of political parties (2010–2015)’. Next, evidence and advocacy messages were presented to the Parliamentary Group and Tanzanian journalists. In ensuing discussions, MPs declared that incorporating nutrition into party manifestos for the 2015 general elections would be an effective way of promoting the government’s longer-term political commitment to nutrition. Moreover, they pledged their personal support promoting this goal. For PG members, having nutrition substantially mentioned in their party manifesto not only provided political sanction for an issue they wish to champion, but also fostered accountability towards the electorate. Thus, the PG chairperson noted that in grassroots electoral campaigning ‘Each member carries the election manifesto from last election, to show people what we promised, and what we have done’ (pers.comm, Hon. L. Mng’ong’o, MP, October 2014).

PG members followed up on their pledges by presenting the four advocacy messages to fellow MPs in the budget session of Parliament in April-June 2014 (pers. comms, October 2014). By then, PANITA had hired local consultants to develop a report that would present the case for nutrition to be given greater attention in electoral manifestos. Their analysis was based on reviews of literature, interviews with key Tanzanian stakeholders and supported by reviews by nutrition specialists at Save the ChildrenFootnote 8 and UN-REACH. Consensus on this document was subsequently generated internally amongst PG members, who also consulted with 42 MPs in Dodoma. Next, the PG chairperson proposed condensing the lengthy report into a booklet containing the key advocacy points, and PANITA and UN-REACH brought in four nutrition specialists to craft this. The pocket-size Nutrition Recommendations (NR) document was produced in English and kiSwahili. The booklet (Fig. 2) was designed to offer MPs and other political leaders an easy overview of key nutrition issues and advocacy messages, to facilitate their circulation and referencing in political debate (immediate outcome).

Fig. 2
figure 2

Output: The Nutrition Recommendations booklet

Full size image

Step 4: Assemble and Assess the Contribution Claim, and Challenges to It

In the previous section, we provided evidence regarding the envisaged output (the booklet), the immediate outcome (MPs have easy access to nutrition recommendations) and also the intermediate outcome: i.e. the presence of nutrition in political party manifestos. In this section, we will deepen the analysis to assess the contribution story. We thus explore the “blackbox” between our outputs and immediate outcomes on the one hand and between the latter and the intermediate outcome on the other hand. This impels us to revisit evidence for the activities and assumptions identified in step 2 regarding circulation, targeting, timing, presence and exclusivity. We adopt process tracing steps to interrogate the quality and the inferential strength of such evidence through hoop and smoking gun tests.

Once the Nutrition Recommendations (NR) booklet was produced, PANITA and allies worked together to promote a range of consultative, advocacy and awareness raising activities with MPs, while also supporting local journalists to report on nutrition in mainstream media. Table 1 chronologically sets out activities listed in step 2 as well as subsequent advocacy action targeting MPs and party members in the run up to the elections.

Table 1 Timeline of key activities towards influencing party manifestos
Full size table

By the end of 2014, Tanzanian politics was dominated by a constitutional review process, subsequently aborted. PG members claimed to have been actively involved in promoting PANITA’s advocacy messages at this time. They proposed that parliamentary records would offer written records of their efforts. A visit to the parliamentary records office however resulted in the director asserting that’Hansards have never been given out’ to visitors (pers. comm, March 2015), and subsequent written requests to obtain access were met with a stony silence.

Preoccupation with the constitutional review process meant that PG members initially did not share the Nutrition Recommendations with colleagues (pers. comm., PG chair, October 2014). However, by February 2015, the PG chairperson launched the NR in the well of Parliament, to substantial applause, witnessed by this paper’s authors.Footnote 9 A side event later in the day chaired by the deputy minister of Community Development, Gender and Children debated the recommendations with 43 MPs. The events sought to increase MP ownership and invite pledges to promote the NR within their parties’ manifesto drafting processes.

None of the 14 attendant PG members would become part of manifesto drafting or executive committees. Yet, they included senior politicians: two MPs were sitting Ministers of State, for Constitutional Affairs and Justice, and for Gender and Social Development. Moreover, in subsequent years, three members would gain ministerial posts in key areas related to nutrition, in Agriculture, Health, and the President’s Office for Regional Affairs and Local Government. The seniority of members makes it more likely they had close direct or indirect connections to the manifesto drafting process and were able to influence it. At the end of May, PANITA invited Parliamentary Group leaders, senior figures and MPs from a range of political parties to attend the Regional Launch of the first Global Nutrition Report 2015 in Dar es Salaam. The PG chair, as guest of honour, presented the booklet. Members of ten other parties were invited, and representatives of seven parties attended. Several days later, PANITA and the PG hosted another meeting in Dodoma, where NRs were shared with Chairpersons of several of the smaller political parties, including the Tanzania Labour Party (Hon. Augustino Mrema MP), the United Democratic Party (Hon. John Momose Cheyo MP), and the NCCR-MAGEUZI (Hon. Joseph Mbatia MP), who later joined the UKAWA coalition. At this time, the PG leadership also shared and discussed the nutrition recommendations with the Secretary General of CCM, as well as with six members of the CCM Manifesto drafting committee (Hon Steven Wasira (MP), Hon. Maua Daftari (MP), Hon. Adam Malima (MP), Hon. Samia Suluhu Hassan (MP), Hon. Mary Nagu (MP) and Hon. Willium Lukuvi (MP). Hon. Hassan would become the future Vice-President. Furthermore, during this visit, the PANITA leader conducted a number of bilateral meetings including with Hon. Tundu Lisu MP, the Chief Whip of CHADEMA. Subsequently, the chairperson and three other PG members with PANITA initiated a launch event in Iringa Region, debating the nutrition recommendations with senior bureaucrats such as the Regional Administrative Secretary, District Executive Directors, and with 55 district councillors.Footnote 10

Several leaders of small political parties, as well as MPs from CCM and CHADEMA, offered testimonies at the June 2015 event in Dodoma. Not all were members of the PG, hus, Hon. Augustino Mrema (MP & Chair of Tanzania Labour Party), as well as veteran MP, Hon. John Momose Cheyo, Chair of UDP party acknowledged having received the copy of the Nutrition recommendations from the PG chair. Similarly, Hon. Mahamoud Mgimwa, a member of the influential Parliamentary Budget Committee, promised to support inclusion of the nutrition recommendations in the CCM manifesto. PG member Hon Said S. Said MP from the Civic United Front noted ‘I have given the recommendations to our chairperson’, while a CCM MP who declared herself to be a member of the CCM manifesto drafting committee, also promised to personally promote the NR (anonymous, pers.comm, June 2015). Hon. Esther Matiko MP (CHADEMA) testified ‘I have talked to Secretary General of my party about the recommendations and I will also make sure that we incorporate nutrition in our manifesto because we have not yet finalized the manifesto’. Hon. Rebecca Mgondo (CHADEMA) also declared ‘I have already talked to Deputy Secretary General Hon John Mnyika on the recommendations and also I promised to give him a copy of the recommendations’. Furthermore, at the launch of the Global Nutrition Report, Hon Lediana Mng’ong’o MP and Hon. Mahamoud Mgimwa MP publicly declared that they had shared the nutrition manifesto recommendations with the members of the CCM manifesto drafting committee.

Additional evidence emerged that the booklets were circulated by policymakers at the heart of nutrition policy processes, beyond the plans devised by PANITA. As such, the acting managing director of the Tanzania Food and Nutrition Council recalled that while the TFNC did not try to influence the manifesto drafting, they keenly used the booklet in advocacy with other national level policymakers. Having Parliamentarians recognising the importance of nutrition ‘was one of the things that made us [TFNC] proud’ (pers. comm, J. Kaganda, July 2018).

Furthermore, MPs highlighted that they operated at both national and at local constituency level. For instance, Hon. Rev. Luckson Mwanjele (CCM) noted (pers. comm, June 2015): ‘After the [February 2015] launch I got several chances to talk about nutrition in my region in the Regional Consultative Council meetings and also to the members of the council committees’.

Parliamentarians also underlined the potential of mobilising grass-roots party activists to amplify the nutrition recommendations within party hierarchies and decision-making structures. ‘The PG has two channels of influencing, one through Parliament, and because we also operate in our respective regions and districts, we can influence the representatives who become part of the National Executive Committee (NEC). These members also give reports to the CCM Secretariat, who will then report to the NEC.’ (pers. comm., Hon Zarina Shamte Madabida, MP, October 2014). Similarly, the PG chair noted (pers. comm, Hon. L. Mng’ongo’, MP, October 2014), ‘Now it’s the right time to start advocating for our nutrition manifesto…there is a drafting committee per party, CCM elects this committee for a 5 year period. Those drafting committees are given guidelines. We want to engage Central Committee Members [in] advocacy because they develop the guidelines.’ Moreover, ‘We need to get the National Executive Committee’. Furthermore, she noted how PG members were mobilising women’s networks within CCM and CHADEMA, ‘we have a well organised system, from village level and ward level, at district level. If we want to meet women, they come into thousands. Here we can and are already raising awareness on nutrition. So, we’ll use women as our entry point in pushing hunger and nutrition [onto our parties’, ed.] agenda’.

Yet, the necessity to operate at multiple levels also strained MPs’ capacity to effectively circulate the nutrition recommendations. The chairperson of the Parliamentary Group quite reasonably argued that operating at multiple levels (local, regional to national) meant that ‘we need funds to start advocating for inclusion of our agenda into manifestos’ (pers.comm, Hon. L. Mng’ong’o, MP, October 2014). However, the extent of such support was contested. MPs’ demands for financial support challenged PANITA’s limited budget and raised legitimate questions about when it would be valid to reimburse MP expense claims. In one instance, MPs had suddenly demanded elevated sitting fees to engage in a nutrition event. PANITA’s zonal coordinators also noted that their lack of funds to pay for such fees made it futile for them to advocate on the nutrition recommendations with their local MPs. On another occasion, a stunned UN staffer informed that MPs had outlined the need for a multimillion fund for supporting its nutrition advocacy activities (pers. comm., anonymous, October 2014). These reflections suggest a relevant precondition for circulation: the ability of policy advocates to navigate the tension between MP demands for and donor provision of scarce funding resources shapes their potential to attain and retain parliamentarians’ support advancing key advocacy messages.

Finally, we obtained one testimony from a senior civil servant who was part of the CCM manifesto drafting committee, affirming that they had received the nutrition recommendations directly from PG members (anonymous, pers. comm, July 2018). While this testimony constitutes very strong evidence that the circulation and targeting assumptions hold, we were unable to triangulate it, as it was not possible to identify and interview other CCM drafting committee members.

In summary, the presented testimonial, circumstantial and sequence evidence suggests that basic hoop tests are passed for the assumptions regarding circulation, timing and targeting, to increase confidence that the posited link between immediate and intermediate outcomes functioned as expected.

Step 5: Gather New Evidence from the Implementation of the Intervention

The process of going through step 4 identified prospective alternative explanations that could challenge the contribution story. In this section, we hence explore two key rival hypotheses and put these to the test using process tracing techniques, to deepen the analysis.

  1. 1.

    Nutrition was already significantly included in the manifestos of the key parties prior to elections in 2015.

  2. 2.

    The inclusion of nutrition in the 2015 manifestos was mainly the consequence of advocacy carried out by other actors than PANITA.

If these rival hypotheses pass hoop tests, we will need to reject our contribution story.

To assess rival hypothesis 1, we analysed the text of the 2010 and 2015 manifestos (Table 2) to establish whether and in what ways nutrition was incorporated, and the envisaged intermediate outcome was achieved.

Table 2 Intermediate outcome: Nutrition keyword search results for political party manifestos 2010–2015
Full size table

We found that the 180-page CCM manifesto focused on tackling youth unemployment, alleviating rural poverty, combating corruption, and maintaining peace and stability. UKAWA’s 90-page document focused on strengthening Tanzania’s economy, the rule of law and human rights, improving social services, and eradicating poverty (Africa Research Institute 2015). Unsurprisingly, nutrition was not a major issue in the manifestos, as unlike hunger, it rarely is a topic of popular electoral concern (Heaver 2005). We did not find specific phrases, sentences or passages from the NR that had been included, verbatim, in the manifestos. Regardless, Table 2 shows that three important terms were found in both the booklet and in the 2015 party manifestos. These included lishe (nutrition), which was found 12 times in the CCM manifesto but only twice in the UKAWA manifesto. The search terms utapiamlo and the utapiamlo mkali (malnutrition and severe acute malnutrition) are found once, but only in the CCM manifesto.

We also noted that the CCM manifesto gave greater attention to nutrition in 2015 (15 mentions) than in 2010 (one mention). This is unlikely to have happened randomly. Political power is highly centralised in Tanzania, in the office of President. Presidential candidate John Magufuli ‘evidently contributed’ to the manifesto (Africa Research Institute 2015), however his record in office as President till date suggests he has less interest in nutrition than his predecessor. Accordingly, it would not have been unreasonable for nutrition to have received reduced traction in the CCM manifesto, were it not for the PANITA led advocacy effort. Rival hypothesis 1 hence does not pass the hoop test and needs to be rejected in case of CCM. However, the number of mentions of nutrition in UKAWA’s manifesto actually decreased, when comparing 2015 (one mention) to 2010 (five mentions). As the 2010 manifesto contained five mentions of nutrition, we conservatively conclude that rival hypothesis 1 cannot be disconfirmed with confidence for CHADEMA/UKAWA.

The decrease of nutrition mentions in UKAWA’s manifesto (2015 as compared to 2010) may have occurred for several reasons. Despite the PG being a cross-party body, the mediation by a passionate but dominant PG chairperson affiliated with CCM may have affected the ability to reach out to members of the UKAWA coalition. Consultation and launch events occasionally witnessed a disproportional presence of CCM MPs (even considering that in 2010, 186 seats where held by CCM and 53 by all other parties, of which 23 were held by CHADEMA). For instance, the Dodoma launch event was only attended by two MPs who were not representing CCM. Similarly, the May 2015 launch of the Global Nutrition Report was attended by seven political parties, yet CHADEMA did not attend. Accordingly, as the advocacy had a stronger overall thrust towards CCM, observed differences in party manifesto outcomes were always likely.

In order to ascertain the validity of rival hypothesis 2, we explored the nutrition advocacy context in Tanzania, to determine potential roles and contributions of other than PANITA coalition actors towards the observed intermediate outcome (CCM manifesto).

Our stakeholder mapping identified donor, UN, civil society, academic and government actors working on nutrition advocacy. Indeed, given the multi-faceted and complex nature of the malnutrition challenge, many organisations were active in the nutrition advocacy space in Tanzania in the years leading up to and following the 2015 election. Tanzania’s government budgets are significantly dependent on aid monies accounting for 33 per cent of spending in 2010–2011 (Development Partners Group Tanzania n.d.), and the biggest donors, such as the UK Department for International Development (DFID) and USAID have strongly promoted nutrition. USAID, for instance, through its Feed the Future initiative, strongly supported private sector involvement in agricultural value chains (the Tuboreshe Chakula programme), and behaviour change communication (Mwanza Bora programme). From 2015 onwards, the World Bank commenced the large scale Power of Nutrition programme, which aims to save at least 67,000 lives and prevent at least 166,000 cases of stunting by 2020 (The Power of Nutrition, 2015). UNICEF in particular has for a long time actively advocated on nutrition policy and closely supported coordinating agencies within the government of Tanzania (Leach and Kilama 2009).

Other organisations also sought to strengthen the institutional capacities of government bodies. In particular, the global Scaling Up Nutrition (SUN) Movement has been influential. US Secretary of State Hillary Clinton’s visit to Tanzania in 2010 to launch the SUN movement attracted high level attention, and shortly afterwards, President Kikwete (2005–2015) declared himself a ‘nutrition champion’. Kikwete announced Tanzania as one of the ‘early riser countries’ and became member of the SUN Lead Group that provides strategic direction to the movement. Subsequently, Kikwete issued a Presidential ‘Call to Action’ on nutrition (United Republic of Tanzania 2013), accompanied by a raft of policy reforms, towards strengthening multi-sectoral coordination within government, improved human resourcing and increasing public spending on nutrition.

The SUN movement is led by a government focal point who sits in the Prime Minister’s Office and coordinates four national level networks, one for civil society members (led by PANITA), another for donors, a business network and a UN network, each convening organisations involved in nutrition. The SUN business network is coordinated by the Global Alliance for Improved Nutrition (GAIN). The SUN donor network was convened by USAID and Irish Aid, while the UN network was coordinated by UN-REACH (Renewed Efforts Against Child Hunger and undernutrition), convening UNICEF, WFP, WHO and FAO (UN-REACH 2012).

UN-REACH played an important role supporting the development of the Nutrition Recommendations. A review of PANITA’s 2012 and the PG 2013 advocacy strategy documents, as well as conversations with the executive director, indicated that neither PANITA nor the PG had plans influencing manifestos before late 2013. USAID hosted the donor group on nutrition and supported the FANTA project from 2010–2017. This project conducted significant advocacy and capacity building efforts with the Government of Tanzania, however, it did not engage parliamentarians or political parties (FHI 360/FANTA 2017). While other donors active in the nutrition space such as Irish Aid and DFID funded the HANCI project, they too were not involved in direct advocacy towards parliamentarians or political parties. Other major donors, such as the European Union, did not fund nutrition activities in Tanzania. Testimonial evidence from leading PG members noted little engagements with civil society organisations, besides with PANITA/Save the Children (pers. comm, PG members, October 2014). PANITA zonal coordinators reinforced that few regional or local NGOs engage with parliamentary advocacy, citing resource constraints as a key challenge. Indeed, the majority of international and national NGOs and CSOs working on nutrition in Tanzania focus on service provision. However, a few do engage in policy advocacy. For instance, Hellen Keller International was pivotal in advocating for new food fortification legislation, but the person leading these efforts informed that HKI did not lobby parliamentarians or party manifestos (pers. comm, I. Kitururu, July 2018). Accordingly, our evidence clearly shows that besides PANITA, no other groups conducted parliamentary advocacy on nutrition or aimed to influence the manifesto processes.

To conclude, in case of the CCM, the evidence presented is sufficiently strong to lead us to reject rival hypotheses 1 and 2.

Step 6: Revise and Strengthen the Contribution Story

In this paper, we have presented strong affirmative evidence and found no disconfirming evidence, to give us confidence that there was a clear causal relation between PANITA led advocacy activities, outputs (nutrition recommendations produced), immediate outcomes (NR available to MPs) and intermediate outcomes (nutrition substantively featuring in the CCM manifesto). We also identified three key assumptions underlying the relation between immediate and intermediate outcomes, two of which posited rival hypotheses explaining observed outcomes. Here too, evidence passed critical hoop tests. As we did not find disconfirming evidence, we conclude that the contribution story presented in Fig. 1 holds for the CCM.

In addition, our Tanzania case study points to five new assumptions (shown with * in Fig. 2) to further specify the contribution story. These assumptions relate to one another, and focus on the relation between outputs, immediate outcomes and intermediate outcomes in the contribution story.

Intelligibility

First, we assume that the parliamentary advocacy process is sufficiently knowable, while recognising that (development) evaluators’ ability to evaluate parliamentary advocacy is subject to significant constraints. This assumption affects the entire causal chain (Fig. 3). Understanding the impact of parliamentary advocacy requires investigating efforts that advocates (civil society or other) make in connecting with parliamentarians, but also assessing the latter subsequent actions. Even though Tanzanian MPs generously gave their time in interviews and during advocacy events, scrutinising their (sometimes lofty) claims, activities and possible impacts proved challenging. In the absence of access to official parliamentary data for analysis, the study relied on personal or second-hand observations during PANITA hosted events. Advocacy activities carried out by MPs outside these events, though very likely to have occurred and critical in terms of achieving advocacy objectives, were not traceable or difficult to triangulate with other sources (e.g. parliamentary records). Indeed, advocacy often depends on personal relations, with meetings in the corridors of power designed to be private, outside of public view. Any reports on these activities hence essentially depended on potentially biased or self-aggrandising and hard-to-verify testimonies. Likewise, the manifesto drafting processes of CCM and UKAWA were shrouded in considerable secrecy. It was difficult identifying and persuading committee members to be interviewed, even well after the elections. Documentary evidence of the manifesto drafting process (such as minutes of meetings), though likely existent, is treated as confidential and inaccessible. Accordingly, there are significant obstacles validating and triangulating evidence to strengthen confidence in causal inference claims about the effects of parliamentary advocacy. Yet, the sample of evidence that was gathered for evaluation in this paper is implicitly assumed to be reasonably representative of the broader potential pool of evidence on the parliamentary advocacy process.

Fig. 3
figure 3

Revised contribution story

Full size image

Power

A second assumption is that unequal power relationships between civil society advocates and MPs, and between MPs and manifesto drafting committee members are not impeding coalition formation, dialogue, critical exchange of ideas, and influencing opportunities. We have seen in our case study that financial and political resources, including trust, play a significant role in the power dynamics that shape negotiated relationships between nutrition advocates and MPs. Fruitful collaboration with MPs had been facilitated through PANITA’s hosting the PG secretariat, and through regular activities convening and training the MPs on nutrition issues. Mutual trust had clearly built up over time, and this fundamentally enabled a joint approach towards influencing manifestos. PG members thus provided important support to civil society advocates in figuring out the nature and timing of key steps in the manifesto drafting process and identifying key actors within this. Nevertheless, dependence on mediation by parliamentarians entailed risks for civil society groups engaging in parliamentary advocacy. In our case, the chairperson exercised an iron grip, on occasion personalising PG decision-making processes. Moreover, relations within the PG and between PG leadership and PANITA leadership were quite hierarchical. Having such a dominant gatekeeper made it harder for PANITA and PG members to develop unmediated, independent bilateral connections with, for instance, the secretary generals of leading political parties, known to be influential in manifesto drafting processes.

Elites

A third assumption that we propose based on the case study is that MPs are neither the only, nor the most influential political elites engaged in manifesto drafting processes. Parliamentary advocates thus had to, from early on, engage unfamiliar and opaque party structures, functionaries and hierarchies shaping manifestos. Navigating these was clearly more challenging than advocating within ordinary policymaking processes.

Political Space

Fourth, we assume that there is sufficient political space for advocacy towards and by parliamentarians to be undertaken in order to influence party manifestos, and that nutrition advocates can effectively navigate this terrain. In Tanzania, the final years of President Kikwete’s rule witnessed a gradual circumscription of political and media space for criticism, and this has quickened since. Tanzania is not unique in this respect; political space for policy advocacy is narrowing globally (Carothers and Brechenmacher 2014). For PANITA, advocacy was fundamentally grounded in an ‘insider’ approach aimed at maintaining cordial relations with political and policy elites, as this enabled participation in advisory fora to the government, for instance in the High Level Steering Committee on Nutrition. Nutrition advocates thus require significant political guile to carefully navigate potentially treacherous political terrain. For development partners seeking to work with advocates, it is hence important to realise that a more activist, antagonistic approach to parliamentary advocacy, while more visible and more easily verifiable than a cautious advocacy approach, may also be unsustainable and counterproductive in contexts of closing political space.

Resources

Fifth, we assume that adequate financial resources are available to nutrition champions to cultivate a productive advocacy relation with Parliamentarians. In our case, financial resources occasionally strained the relationship between nutrition advocates and parliamentarians. PG members had a strategic vision of running simultaneous advocacy efforts at national and local levels. This prospectively allowed for targeting relevant political party structures and members involved in the manifesto drafting process and allow MPs to be seen to be personally campaigning for nutrition in their constituencies, as they were seeking re-election. Civil society advocates can carry significant legitimacy and superior knowledge of local political economies. However, financial resources to embark on parliamentary advocacy are typically limited, to constrain their ability to give effective financial support to MPs and to hold them account for their spending. Furthermore, a combination of necessary prudence and lack of financial muscle may lead nutrition advocates, often located in capital cities, to espouse a vision that prioritises national level advocacy, overlooking relevant subnational decision-making structures and manifesto drafting processes. This in turn may lead to a prioritisation of provision of information to parliamentarians and providing financial support for dissemination events, potentially doing insufficient justice to the requirement of parliamentarians to operate at multiple levels, particularly when they are campaigning for re-election.

Discussion and Conclusion

Parliamentary advocacy for nutrition is currently receiving significant global attention. However, questions remain about suitable methods to evaluate such efforts and empirical cases are scarce. This paper presented an impact evaluation of parliamentary advocacy in Tanzania. It showed how to assess causal inference in a complex case, through a rare application of the Contribution Analysis and Process Tracing procedure, as proposed by Befani and Mayne (2014).

CAPT offers a systematic qualitative evaluation approach that was found well-suited to assessing the impact of parliamentary advocacy. Through its step-by-step application, we have shown that distinct configurations of activities, actors and outputs can be plausibly understood to have had a contributory role in achieving increased attention to nutrition in the manifesto of the winner of Tanzania’s 2015 general election. We have noted how parliamentary advocacy and development evaluators’ ability to evaluate its outcomes is shaped within the Tanzanian context by power dynamics within advocacy relations, shrinking political space, resourcing, and constraints on validating evidence. Moreover, we have posited and evidenced the relevance of assumptions concerning targeting, circulation and timing as mediating the causal relation between MPs having access to nutrition advocacy messages, and their actual incorporation into a political party manifesto. These assumptions may well be of relevance for assessing parliamentary advocacy in other countries with high burdens of malnutrition.

Methodologically, CAPT involves the grafting onto one another of two relevant but somewhat distinct approaches within the larger family of systematic qualitative evaluation. This has distinct advantages, but also raises some difficulties. Our analysis affirms (a) that the addition of the process tracing steps allows for an unpacking of causal forces within the black-box of a CA theory of change, (b) process tracing tests regarding the quality of evidence used for analysing causal chains usefully complements the CA approach to strengthen the theory of change regarding parliamentary advocacy, and (c) that CA puts greater emphasis than PT on understanding the environment within which a causal process takes place, to more actively pursue the identification of alternative causal explanations. In all, CAPT can usefully support analytical generalisation; the ‘preferred manner of generalizing from case studies and case study evaluations’ (Yin 2013, p. 327).

The thoroughness of conducting both CA and PT in one exercise however comes at a cost of being time-consuming, and of risking the repetition of analytical steps. Moreover, as CA and PT use related, yet distinct concepts and terminologies, there is risk of conceptual confusion. For instance, the CA’s theory of change and PT’s causal mechanism both contain steps or components that make up a causal chain. However, these are pitched at different levels of abstraction, with the causal mechanism the most simplified form in PT, as a series of interlocking motions and effects, which explain how a specific input led to observed outcomes, while CA posits a theory of change with a more abstract and generalised causal process.

Finally, while evaluation methodologies typically posit that the researcher or evaluator should be objective and distanced from the subject being evaluated, researchers in this study took a ‘developmental evaluator’ role. They had been involved right from the start of identifying the opportunity to influence manifestos, and throughout the advocacy process, building on an existing partnering relationship with the leading advocacy organisation. Rather than being external and independent, evaluators collaborated to ‘conceptualize, design and test new approaches in a long-term, ongoing process of continuous improvement, adaptation, and intentional change’ (Coffman 2009, p. 7). While this approach potentially risked introducing confirmation bias, the CAPT procedure safeguarded against this by offering a detailed elaboration and careful, explicit weighing of evidence. Iterative rounds of engagement facilitated deeper insight in, and the essential ability to respond to emerging advocacy dynamics, as well as access to unpublished materials not ordinarily accessible to an external evaluator. This enabled triangulation and enhanced the internal validity of study findings. Accordingly, international promotors of parliamentary advocacy should consider early involvement of ‘developmental evaluators’ to assess impact, to gain in-depth insight and to support theorisation of and practical learning about effective ways of undertaking parliamentary advocacy interventions within context.