Using Participatory Approaches in Measuring Resilience and Development in Isiolo County, Kenya

Abstract

This article highlights the process of using participatory approaches in measuring resilience using the Tracking Adaptation and Measuring Development (TAMD) Framework. The utilization of participatory approaches in Isiolo County using the TAMD framework is aligned to the recent thinking of measuring ‘subjective resilience’ using people’s perceptions to quantify household resilience. This article outlines the process of developing subjective indicators with communities, collection of baseline, monitoring and early outcome data by communities who were assisted in the development of their own adaptation theories of change. It also highlights the lessons and implications for policy if the approach is to be replicated at sub-national and community levels.

1 Introduction

There is no commonly accepted definition of resilience across all disciplines.¹ However the International Panel for Climate Change (IPCC) in its Annual Report 5, builds on the definition used by the Arctic Council in 2013 and defines resilience as the capacity of social, economic, and environmental systems to cope with a hazardous event or trend or disturbance, responding or reorganizing in ways that maintain their essential function, identity, and structure, while also maintaining the capacity for adaptation , learning, and transformation.² With this in mind, there is a need to measure the impact effectiveness of adaptation actions and how they contribute to a population’s resilience. Measuring resilience also contributes to measuring people’s ability to respond to and accommodate adverse events.³

Isiolo County is located in upper eastern Kenya covering an area of 25,336.1 km². Most of the county is a flat low lying plain. Isiolo is regarded as one of the arid counties and is hot and dry for most of the year with two rainy seasons; short rains (October and November) and long rains (March–May) with average rainfall of 580 mm. The main ethnic groups found in the county are Borana, Turkana, Samburu, Somali and Meru. The main economic activities practiced in the county include pastoralism, subsistence agriculture, small-scale trade, and limited harvesting of Gum Arabica resin. Over the years, its communities have continued to feel the increasing impacts of climate variability due to the increasing frequency of drought episodes and their negative impacts.⁴ These impacts include: longer trekking distances for women and girls, over dependence on humanitarian aid, infrastructure destruction due to flash flooding, changing livelihoods as communities are unable to recover from the increasing frequency of drought episodes amongst others.

The county was chosen for the TAMD feasibility testing, as it was the first county to receive climate financing from the Department for International Development (DFID) for the establishment of a County Adaptation Fund (CAF).⁵ The objective of the CAF is to finance public good investments for improved resilience to climate change through the County government and six ward adaptation planning committees (CAPCs and WAPCs respectively) through the Adaptation Consortium.⁶ The six wards are Kinna, Garbatulla, Sericho, Oldonyiro, Merti and Chari.

Resilience in Isiolo, according to the resident communities is equated to long term development outcomes such as sustainable livelihoods due to better livestock production which leads to increased incomes, improved human health, access to natural resources/pasture, food security and access to education.

Thus the main question was whether investing climate finance in public investment goods was going to elicit resilience measures as described by the communities. For this the TAMD framework developed by Brooks and others⁷ was chosen as the tool that would be used to test whether resilience measures defined by the communities themselves were possible at the sub-national (county) and ward (community) levels.

2 Approach

The TAMD framework is for use in many contexts and at many scales to assess and compare the effectiveness of interventions that directly or indirectly assist populations in adapting to climate change . It also provides an explicit framework for two tracks; Track 1 entails assessing the capacity of institutions to undertake effective climate risk management (CRM) actions (also called top-down), while Track 2 entails assessing impacts of interventions aimed at reducing vulnerability and the extent to which such interventions keep development on track (development performance or bottom-up) – Fig. 15.1.

Fig. 15.1

TAMD framework (Adapted from Brooks & Fisher (2014) (Brooks, N., & Fisher, S. (2014). Tracking Adaptation and Measuring Development (TAMD): A step-by-step guide [Toolkit]. London, UK: International Institute for Environmental Development (IIED). Retrieved from http://pubs.iied.org/10100IIED)

The TAMD operational framework⁸ has a set of eight commonly used indicators that can be used to measure top down/climate risk management processes being implemented by government institutions using a score card (these indicators are further described under the top-down process – Track 1). The operational framework then suggests that theories of change (ToCs) be used to measure the change pathways from adaptation interventions to development performance in bottom up processes (Track 2). It further suggests that linkages between climate risk management processes and development performance/adaptive capacity can be shown in a ToC. Thus the development of ToCs can be within one track or between tracks. The researchers therefore chose to measure top bottom processes with a score card and ToCs to show changes in adaptive capacity using bottom up approaches. In addition the researchers used a ToC to make the linkage between Tracks 1 and 2.

2.1 Top-Down (Track 1) Process

For Track 1, county technical officers from the departments of water, livestock, natural resource management, meteorology, planning and the National Drought Management Authority (NDMA) were brought together to identify and prioritize CRM activities required to build adaptive capacity at community level. These activities were screened from the NDMA strategic plan, the Isiolo County Integrated Development Plan (ICIDP), and sectoral plans of the county.

The technical team assessed CRM processes through the use of Brooks score card.⁹ The score card measures CRM indicators in Track 1 through 8 parameters, namely, climate change mainstreaming/integration into planning, institutional coordination, budgeting and finance, institutional knowledge/capacity, use of climate information, planning under uncertainty, participation, and awareness among stakeholders. Under each parameter, there are five questions that need to be answered before scores are assigned. The type of scoring is chosen by stakeholders in terms of weighting (0–4) or percentages. In Isiolo County, percentages were used to depict the extent to which progress against the indicator was being made. The score card and its results are shown in Table 15.1.

Table 15.1 Indicators and assumptions for the integrated ToC

2.2 Bottom-Up (Track 2) Process

Before communities were facilitated to develop ToCs per ward, it was important that communities defined the term resilience in their own context so as to understand how their planned adaptation actions contributed to resilience. The researchers worked with six WAPCs to identify 20 ward adaptation/development interventions covering the water, livestock, and natural resource governance sectors that were in planning phases. Each of these wards was then assisted in developing their own specific ToC, identifying outputs, outcomes, long term impact , indicators and assumptions.

2.3 Linking Track 1 and Track 2

After the top-down and bottom-up processes were completed, a composite theory of change was then developed by the county technical team and the WAPCs. This ToC linked the prioritized county CRM interventions identified through the score card process with the six ward ToCs as shown in Fig. 15.2.¹⁰

Fig. 15.2

Integrated Isiolo ToC. (Ibid)

The methodology used above sought to learn lessons from two questions namely:

• To what extent can participatory processes be used in designing a ToC that links CRM activities (Track 1) with development outcomes (Track 2)?

• How can the framework be used to inform planning at sub-national and community levels?

When the composite ToC was developed and expected changes and indicators were identified in the top-down, bottom-up, Track 1 and 2 linkage processes, the County Planning Unit proceeded to integrate relevant CRM and adaptation actions into the Isiolo County Integrated Development Plan (ICIDP) in order to mainstream adaptation planning and M&E.

The use of a participatory approach in testing the feasibility of TAMD was chosen, as it sought to enhance ownership of the data collected, the analysis, and the dissemination of lessons learned. It also sought to build the evaluative capacity among stakeholders, an approach also supported by Preskill (2009)¹¹ and Preskill and Boyle (2008).¹² This was done by simplifying various climate change and M&E definitions and processes with the county officials and WAPCs e.g. climate variability, maladaptation, outputs, outcomes, impacts , indicators, evaluation and assumptions, before ToCs and M&E plans were developed with facilitation from the researchers.

2.4 Baseline Data

Two types of baseline data were collected from Isiolo. Track 1 (top-down) and Track 2 (bottom-up).

The sources included:

• Key informant interviews using semi-structured questionnaires

• Semi-structured group interviews

• Secondary data sources from county development plans were used for triangulating primary data collected from communities e.g. livestock numbers per ward, number of households accessing potable water.

2.4.1 Track 1 (Top-Down)

For the CRM processes under Track 1, the scores that were agreed upon through the use of the score card were the baseline values. The outputs of this exercise are shown in the results section. This exercise also highlighted the weak areas in CRM in the County, and as such, interventions that could address the weaknesses were prioritized. These were strengthening early warning systems, county budgeting and planning, and county coordination and planning.

2.4.2 Track 2 (Bottom-Up)

After the development of the ward ToCs, communities were given basic training in collecting baseline data against the indicators they had developed to measure their perceptions of resilience/adaptive capacity, for their respective ToCs, with a data collection tool that had been designed by the research team. This data was collected over a period of 3 months by the six wards.

With respect to baseline data verification, the county officials had been tasked to verify the baseline data before the monitoring visit. However but this was not possible as Isiolo is an expansive county and the verification exercise through community visits had not been budgeted for by the county. As a result this exercise had to be done retrospectively and was conducted together with the first monitoring visit which occurred just after the commencement of interventions.

2.5 Output and Outcome Data

Output data, was collected after a period of 9 months, against the indicators in the ward ToCs and the county government score card (Table 15.3). Early outcome data was collected with an outcome assessment tool, after one and a half years to determine whether there were any changes being experienced from adaptation actions being implemented. This tool allowed the ward adaptation planning committees to assess the extent to which outcomes as depicted in their respective ToCs had been achieved through a scoring system. The results of this scoring are depicted in Table 15.2.

Table 15.2 Indicators and assumptions for the Oldonyiro ward ToC

3 Challenges with Implementing the Methodology

Adapted from Karani, I., Kariuki, N., & Osman, F. (2014). Tracking adaptation and measuring development. Kenya research report. London, UK: International Institute for Environmental Development (IIED). Retrieved from http://pubs.iied.org/10101IIED.html

A few challenges were experienced when implementing the described methodology as detailed below:

• Developing adaptation Indicators: As stakeholders were used to developing output indicators as opposed to outcome indicators in development projects, the process of developing adaptation indicators to adequately measure resilience in the longer term proved to be a challenge.

• Use of climate variability information in the development and adjustment of adaptation actions: An adaptation M&E framework assumes that the design of adaptation actions has incorporated climate risk information. It also assumes that climate trends will be continuously monitored throughout project implementation in order to attribute any outcomes to enhanced adaptive capacity as a result of the interventions. However it was found that climate variability data had not been used when designing the adaptation interventions due to its unavailability during the design phase of the actions. In addition technical capacity to downscale climate trends in order to determine baseline scenarios in the county were also limited.

• Counterfactuals: According to the TAMD operational framework, researchers are expected to collect data on attribution and this requires counterfactual data. This became a challenge in Isiolo because the research team had to find a community in Isiolo where climate change adaptation (CCA) interventions were not being undertaken. This proved to be difficult as there are many civil society actors undertaking CCA activities similar to the CAF in other parts of Isiolo. In addition the CAF interventions were public investment goods that were to benefit over 70 % of Isiolo’s population: the remaining population comprises the urban population whose livelihoods are different from the targeted communities. They therefore, did not qualify as good counterfactuals. As such the research team made a decision to develop the before and after approach using the theory of change to measure contribution/attribution to resilience.

4 Results

The TAMD framework uses theories of change to measure progress towards achieving resilience. Theories of change help articulate assumptions behind smaller steps that lead to a long-term goal and the connections between these activities, outcomes and impact . They help present a visual representation on the contribution of a project or combination of projects to an intended outcome.

In Isiolo an integrated ToC at the sub-national (county level) and five theories of change at the community/ward levels was developed by both the county officials and ward committees. Figure 15.2 shows the integrated ToC at county level whilst Fig. 15.3 is an example of one ward ToC.

Fig. 15.3

Example of a ward/community level ToC. (LTS (2013). Ward Committees M&E report)

The indicators and assumptions for the integrated ToC are in shown in Table 15.1.

The indicators and assumptions for the Oldonyiro ward ToC in Fig. 15.3 are shown in Table 15.2.¹³

The communities were asked to discuss the assumptions described in Table 15.2 in order to develop options for risk management. These included; the legalization of traditional natural resource by-laws by the County Assembly which would assist in the enforcement of sound water resource management and would also raise the profile of the water management committees; and the strict vetting and supervision of potential sand dam contractors to enhance minimize the risk of poor dam construction.

For Track 1, Brooks et al. (2012) proposed a scoring system where each CRM indicator is scored against five questions to which the answer is yes, partially, or no, and scored 0, 1 or 2 respectively. The answers to these questions can be aggregated to yield an overall score out of 10 for each indicator, so that changes in the extent and quality of CRM over the various dimensions the indicators represent can be tracked over short time scales e.g. annually by policy and decision makers. However the scoring parameters can be changed by the users of the score card. Thus in Isiolo, the County officials changed the proposed Brooks scoring to percentages which they are more conversant with. The percentages presented in Table 15.3 ¹⁴ against each CRM parameter were agreed upon by the county officials.

Table 15.3 CRM (Track 1) scoring by county officials

The county scored an average of 59.3 % for climate risk management measures with highest scores around public participation in planning and decision making in climate change adaptation as well as coordination of climate change interventions in the county. These scores provided a baseline for climate risk management activities. Subsequently they were used to develop activities needed to strengthen climate risk management and adaptation activities at county level and also formed the basis of the county’s theory of change .

5 Track 1 Score Card Outputs

From the score card process, the county government had prioritised strengthening early warning systems, improving climate finance and budgeting and improving county coordination and planning. The CAPC was able to implement activities within two of the activity areas. The first activity involved purchasing a transmitter for the Isiolo radio station to enhance dissemination of weather and climate information. The expected output indicators for this intervention were on the types and number of information communication products and the percentage of the population reached with climate information within the whole county. Against a baseline figure of 10 % of the population coverage by the transmitter, after the intervention it was reported that the transmitter managed to enhance the coverage to 50 % of the population. However during this feasibility testing, it was not possible to ascertain as to whether households actually received this information and how they used it.

The CAPC also collected livestock data and information to support the development of the Isiolo livestock strategy. They also conducted a workshop to integrate climate change into the Isiolo CIDP. These activities aimed at improving county coordination and planning activities climate change adaptation.

Some outcomes depicted in the integrated ToC (Fig. 15.1) have already been realised with an increase in number of projects targeting infrastructure, agriculture, health, water and sanitation, food security and income generation as well as number of climate change projects financed through county budget allocation.

Progress towards achieving outputs around the improved financing and budgeting is making slow progress however it should be noted that the county water department has provided financing for ward level adaptation activities such as rehabilitation and construction of sand dams. This indicates that implementation of climate change adaptation activities at local level have been able to influence targeted county adaptation financing.

6 Track 2 Outputs and Outcomes

The Track 2 adaptation interventions implemented at ward level by (WAPCs) were similar to development actions with the only difference being that they were formulated through resilience assessments conducted before the TAMD initiative begun. The ward level interventions were categorised as follows:

• Natural resource management

• Construction/rehabilitation of water structures and water management

• Strengthening of traditional resource governance structures

• Construction of other infrastructure (veterinary lab, animal holding yards).

Over 90 % of the activities were completed by the end of the study period. Early outcomes from the interventions was achieved around reduction of distances to water points, increased access to good quality water for the resident and neighbouring communities, increased capacity of traditional natural resource governance committees (dedhas), proper diagnosis of livestock diseases and strengthened local capacity for natural resource management.

With respect to measuring resilience literature has shown that there are no universal or generally applicable indicators of resilience (or of vulnerability or adaptive capacity), as these phenomena are highly context-specific. However, a number of studies have sought to define dimensions of resilience, with each dimension gathering together a suite of related factors that might be represented by context-specific indicators (Alexander 2013¹⁵; Nguyen and James 2013¹⁶). However for the purposes of this study, social or livelihood resilience as defined by Eakin (2012)¹⁷ and Tanner et al. (2015)¹⁸ was used as it was fit for purpose. Additionally Brooks and Fisher (2014)¹⁹ conducted a review of methodologies for measuring resilience and identified the following potential dimensions of resilience²⁰ that can be used to measure livelihood resilience:

• Assets: physical, financial assets; food and seed reserves, etc. (contingency).

• Access to services: water, electricity, early warning systems transport, knowledge and information – to plan for, cope with and recover from stresses and shocks.

• Adaptive capacity: to anticipate, plan for and respond to longer-term changes – for example, by modifying current practice, creating new strategies.

• Income and food access: the extent to which people may be poor or food insecure before the occurrence of a stress or shock.

• Safety nets: includes access to formal and informal support networks, emergency relief and financial mechanisms such as insurance.

• Livelihood viability: the extent to which livelihoods can be sustained in the face of shock/stress, or the magnitude of shock/stress that can be accommodated.

• Institutional and governance contexts: the extent to which governance, institutions, policy , conflict and insecurity constrain or enable coping and adaptation .

• Natural and built infrastructural contexts: the extent to which coping and adaptation are facilitated or constrained by the quality and functioning of built infrastructure, environmental systems, natural resources and geography.

• Personal circumstances: other factors that make individuals more or less able to anticipate, plan for, cope with, recover from and adapt to changes

From the descriptions of resilience above, the research team used a participatory outcome assessment tool to measure the changes anticipated from the adaptation interventions that could contribute to livelihood resilience in a pastoralist context. The results from the assessment indicated that early outcomes were already being realised and included: reduction in livestock disease cases, availability and access to water in water sources for over longer periods i.e. 3–6 months as opposed to 1–2 months, improved household hygiene and reduction in human waterborne disease incidences.

To measure outcome/adaptation benefit achievements, the wards used outcome assessment forms to provide scores on the achievement of any initial outcomes against outcome indicators . Although attaining resilience is a long term objective, ward adaptation interventions have been able to provide benefits around increasing accessibility of water, reduction of violent armed conflicts, and reduction of livelihood diseases which all play important roles in improving resilience of communities in the County. See the example of an outcome assessment form in Table 15.4 from Sericho ward.

Table 15.4 Outcome assessment from Sericho ward

As can be seen from the dimensions of resilience described earlier, communities in Isiolo are already beginning to experience some aspects of enhanced resilience as they are better able to cope with droughts at the household and community levels through enhanced access to clean water over longer drought periods, leading to improved household hygiene. They are also experiencing less armed conflict between communities which increases the success of any adaptation strategies they are involved in.

7 Lessons Learnt

• Adaptation indicators not necessarily different from development indicators depending on the context.²¹ As can be seen from the indicators developed by communities and the dimensions of resilience, most of the indicators are measuring people’s/communities well-being. What distinguishes the two is the contextualization of the results using climate data. However this was not possible at the time of the study.

• Communities that have been trying to adapt to a changing climate are able to understand M&E concepts when simplified, of short and long term changes to their livelihoods due to adaptation interventions and are able to assess progress in change pathways depicted in a theory of change .

• A comprehensive monitoring and evaluation (M&E) system at sub-national level that is designed to collect adaptation outcomes and climate trend information ex-ante is crucial if enhanced resilience is to be proved through an adaptation evaluative framework process.

• It is possible to find and document adaptation outcomes of a community in 3 years if the interventions are designed from resilience assessments, an M&E system is established at the beginning of the intervention, baseline data is collected and verified, monitoring visits are conducted regularly to ensure that the adaptation interventions are implemented effectively and time is taken to document the changes happening in the communities through narratives. This can be regarded as M&E best practice.

8 Implications for Planning Policy and Practice

The results of this feasibility study have elicited a few implications for planning and investment in the use of participatory resilience M&E methodologies as detailed below:

• Using M&E to influence planning: TAMD in Isiolo was applied before the adaptation interventions begun. This had two advantages (a) the county adaptation committee were able to prioritise the activities that needed to be done under climate risk management and (b) the county adaptation committees was able to collect baseline data against indicators they had designed for CRM. In this way information that had never been collected before was now available for decision making and future planning on CRM. This ex ante M&E fits within the development evaluation approach described by Patton (2010)²². It is also proposed by the World Bank²³ for new or redesigned poverty and inequality reduction programs. This is because despite the upfront investment costs, this method can be cost effective in the long term as it allows for the adjustment and refinement of programs before implementation, and programs are likely to be better targeted as a result. This method can also provide useful information on the political consequences of new programs and therefore provide for the design of appropriate risk mitigation measures before implementation by decision makers.

• Resilience measurement by communities and planning: Project or program M&E is usually undertaken by independent individuals or institutions. During an evaluation exercise, it can become frustrating if the relevant data or information was not collected during the course of the project or packaged appropriately. The advantage of empowering the county and ward committees with tools for collecting baseline and monitoring information increases the chances of a better quality evaluation of community resilience. This is different from normal evaluations in which the target communities are not involved in defining their indicators according to their own perceptions. This acknowledgement of the usefulness of subjective measurements of resilience is relatively recent and has been proposed as complementary to the traditional evaluation methods by Jones and Tanner (2015)²⁴ for planning and decision making. Through subjective resilience measurement, there is a greater understanding of household factors that contribute to resilience and policy makers/decision makers can design and plan for programs that enhance these factors in the long term and avoid introducing or planning for programs that have the potential to be maladaptive to communities.

• Replication and scale-up of subjective resilience measurement methods: Replication of participatory methodologies of measuring resilience such as TAMD can be beneficial for climate risk management planning by sub-national governments and adaptation planning for targeted communities. However up scaling to national level may prove challenging (Jones and Tanner 2012) especially because of initial investment. A cost and values study conducted in Kenya on TAMD concluded that the ‘returns of using TAMD as a resilience M&E system are likely to be considerable, despite uncertainty. This is based only on individual indicators of avoided losses, expenditures and investment requirements. In reality, TAMD will have a system-wide impact , causing many costs to fall simultaneously and generating greater investment returns (Barrett 2014).²⁵ In addition Barrett states that his analysis did not factor in future escalation of climate change effects. This suggests that the likelihood of even higher Net Present Values of TAMD in the future.