1 Executive summary

The Mesoamerican Agroenvironmental Program (MAP-Norway) is a multi-dimensional rural development program implemented in Central America since 2009, working with smallholder families, producer organizations, governmental organizations, and regional governance platforms. To monitor, assess, and evaluate the effects of the program on its beneficiaries, MAP-Norway uses a series of indicators that allow project managers and donors to adapt and follow-up on the interventions. Because gender is a cross-cutting theme in the program, gender indicators are used at all levels: families, producer organizations, and governmental organizations and governance platforms. In this document, we use the experience of MAP-Norway to critically assess these indicators, considering their potential usability in the monitoring and evaluation of climate-smart agriculture (CSA) initiatives. Furthermore, we propose a series of other indicators that capture various dimensions of gender relations. These indicators can be used to assess the effect of CSA practices, services, and technologies on equity in decision-making, women’s empowerment (including economic empowerment), intra-household food security, and equity in ownership over productive resources, among others, thus providing evidence that can help better design and target CSA interventions.

2 Context

The Central American rural population represents 41.41% of the total rural population of Latin America. 62.7% of the people living below the poverty line in Central America are located in rural areas (PRESANCA 2011). For these people, agriculture represents their main livelihood. It contributes to food security (Tucker et al. 2010) and income. In Central America, family farming consists of more than 2 million families, accounting for about 50% of the Central American agro-industry sector and representing 96% of the total basic grain producers (PRESANCA 2011).

Women’s role in agriculture has been overlooked (González and Macleod 2010; USAID 2013), even though women make up around 43% of the agricultural labor force in developing countries (FAO 2011). Rural women in Latin America and the Caribbean represent 20% of the labor force (FAO 2017). While official data indicates that more men than women are active in agriculture in Latin America and the Caribbean overall, agriculture constitutes 80% of women’s economic activities in LDCs, including in Latin America. As well, women’s work activities in subsistence agriculture are often underestimated or excluded for reasons such as limitation of data sources, gender norms about who is a farmer, and concepts and definitions adopted in data collection (UN 2015; Twyman et al. 2015). Besides their reproductive roles, such as caregivers for family members, women are responsible for producing grains, vegetables, and small animals for household consumption which play an important role in household food security. In rural areas in Latin America, women (and children) are also in charge of water and fuel supply for the family needs. These activities are often considered “duties” and not “work.” Usually, social and cultural norms limit women’s participation in public spaces, excluding women’s voices from decision-making processes at the community level, and even though they have less access to production resources (seeds, inputs as well as technical and financial assistance), women play an important role in the transmission of local knowledge about certain agricultural and conservation practices (Gutiérrez-Montes et al. 2012).

Rural women are particularly vulnerable to the effects of climate variability and change due to their low adaptive capacity (IICA 2015). What makes women more vulnerable is that they have fewer endowments and entitlements than men, less access to information and services, increasingly heavy agricultural workloads (FAO 2011), unequal survival opportunities (limitations of mobility), lack of participation in decision-making (Stock 2012), and little control over key resources (Okali 2012). However, Acosta et al. (2015) point out that policy-makers should avoid portraying women as a group that is homogeneously affected by climate change, because that engenders an oversimplified vision of the problem and a discourse that tends to disregard nuances in the effects of socioeconomic and cultural factors. Diversity among rural women (indigenous, afro descendants, youth, farmers, etc.) is also overlooked.

A reduction of gender inequalities in agriculture could have positive results for the sector. According to FAO (2011: 5) “If women had the same access to productive resources as men, they could increase yields on their farms by 20-30%, this could rise agricultural output in developing countries by 2.5-4%, which could in turn reduce the number of hungry people in the world by 12-17%”. Improvement in women’s education is one of the most important policy strategies to reduce poverty and increase agricultural productivity, because increases in women’s capabilities expand their opportunities and empower them to exercise their choices (Quisumbing and Meinzen-Dick 2001). Achieving gender equity, primarily through investing in women, may lead to greater reductions in poverty, faster economic growth, and significant improvements in family health, nutrition, education, and quality of life. Evidence demonstrates that gender integration makes development efforts more effective (USAID 2013). Furthermore, enhancing women’s decision-making power also results in greater well-being of children, and households in general (Kabeer 2005).

Greater participation of women can be particularly important in technology adoption. Projects centered around the climate-smart agriculture (CSA) approach usually promote the adoption of technologies—as well as practices and services—aimed at increasing agricultural productivity while enhancing producers’ climate adaptation and mitigation capacities.Footnote 1 If women have equal access to such technologies and practices and take ownership over the resulting benefits, CSA may have a more significant effect on family well-being. To better understand the relationship between CSA, gender, and rural livelihoods, there is a need for well-defined and efficient indicators (SMART indicators) that allow project managers and policy-makers to assess and evaluate CSA programs or interventions in terms of their impact on gender relations.

3 Mesoamerican Agroenviromental Program

MAP is a platform that links research, education, and technical assistance to support rural sustainable development while reducing rural poverty in Mesoamerica. MAP’s first phase (2008–2013) promoted strategies and practices for sustainable land management (SLM) using the sustainable livelihoods approach and the community capitals framework with a territorial approach (Gutiérrez-Montes and Ramirez 2016) to improve production, competitiveness, and to address environmental issues that affect the most important agricultural and natural resources sectors of the region. MAP’s second phase, called MAP-Norway (2013–2017), promotes the climate-smart territory (CST) approachFootnote 2 to address issues such as poverty, food, and nutrition insecurity, gender inequality, degradation of ecosystem services and vulnerability to climate change (Box 1) (CATIE 2013).

Box 1 Definition of CST

MAP-Norway defines a climate-smart territory as “social and geographic spaces where the actors collaboratively manage ecosystem services to equitably improve human well-being, continuously optimizing land use and mitigation and adaptation to climate change” (Wallace Conference, cited in Louman et al. 2015, p. 77).

MAP-Norway focused its work at three levels: (i) local, with families; (ii) regional, with business organizations and territorial governance platforms; and (iii) national, with governmental organizations. MAP-Norway reached 5000 smallholder families, 30 business organizations, 8 territorial platforms, and 8 governmental organizations (CATIE 2013).

Gender strategy

Both phases of MAP promoted gender equity and social inclusion to contribute to the creation of an enabling environment for human development. This has been captured in the gender strategy of MAP (Siles et al. 2012) and MAP-Norway (Siles et al. 2015). This last strategy comprises four main strategic axes (Box 2) which are addressed by the following actions: (i) to promote positive changes in gender roles at the household level, access, use, and control over resources, equal participation of household members in decision-making, and balanced distribution of the division of labor and responsibilities from within the household; (ii) to promote more equity in decision-making spaces in business organizations; (iii) to incorporate concepts of gender and equity in plans and co-management action plans, generated and disseminated tools and methodologies related to gender and equity; and (iv) to integrate a gender approach in all documents generated and in all program outcomes (Siles et al. 2015).

Box 2 Axes of MAP-Norway’s gender strategy

1. Development of the capitals of rural families, especially women and youth, so that they diversify their livelihoods and empower themselves.

2. Development of entrepreneurial, associative, and innovation capacities of women and youth so that they participate in the main value chains of the key territories of MAP.

3. Management and diffusion of knowledge related to gender and equity.

4. Mainstreaming of the gender approach in the management of MAP-Norway

MAP participatory approach includes (i) strengthening farming capacities of all family members through Farmer Field School (FFS) with husbands, wives, and youthFootnote 3 invited to all FFS sessions. In this way, all the family members are able to learn, share experiences, exchange knowledge, and apply more and better knowledge and skills to improve their farms (Gutiérrez-Montes and Ramirez 2016); (ii) providing technical assistance to business organizations and value chains linked to the families who benefit from the program through the Territorial Business Training Schools (TBS)Footnote 4 and the Business Technical Assistance program; (iii) advocating the incorporation of gender equity and inclusiveness within local governments actions through the Trinational Gender School (TGS)Footnote 5; and (iv) generating new knowledge and scaling up and out knowledge and lessons learned.

3.1 MAP monitoring and evaluation system

MAP-Norway’s monitoring and evaluation system (MAP’s M&ES) is a tool to monitor its actions, as well as to measure progress toward expected results. The system is based on the program’s logical framework and provides the option to perform simple or complex information queries (Mercado and Aguilar 2015).

To monitor progress throughout the period of implementation, MAP-Norway has used a set of quantitative indicators mapping to each of its five outcomes (Box 3). These indicators are grouped in (i) 22 performance indicators that measure progress of actions toward achievement of expected outputs, which are evaluated every year in the Annual Report (Mercado 2014 and 2015), and (ii) 12 outcome indicators that measure progress in the change sought at the end-user level, both in the short and medium term. A baseline study was carried-out in 2013 (Beer et al. 2013), with two follow-up monitoring studies conducted in 2015 and 2016 (Mercado et al. 2015). Gender equity, as a cross-cutting theme in MAP-Norway, is present at all scales of intervention: household/farm, business organizations, and territorial platformsFootnote 6/governmental actors. Therefore, gender indicators are embedded within MAP’s performance and outcome indicators. In this document, we give special attention to the outcome indicators. Table 1 presents the list of indicators assessed in this document.

Box 3 MAP-Norway’s outcomes

1. 5000 families increased their assets with improved equity in family decision-making for food and nutrition security

2. 5000 farms and home gardens increased and agroecological and agroforestry production diversified

3. 30 business organizations increased entrepreneurial capacities and access to markets

4. 6 platforms improved enabling conditions to implement the Climate-Smart Territory (CST) approach

5. CST approach scaled up and out

Table 1 MAP-Norway quantitative gender indicators to measure progress toward expected results at different scales of intervention
Full size table

Data regarding household and farm activities were collected using a survey that was undertaken with heads of households. For business organizations, territorial platforms, and governmental organizations, a survey was undertaken with a representative of each organization and the information received triangulated with official documents such as strategic plans and annual operating plans. Finally, to assess changes at the outcome level, we used generalized linear mixed effects models with post-hoc tests for multiple comparisons and chi-square tests.

4 Assessment of the gender indicators used in MAP-Norway

According to Tayyib et al. (n.d.), indicators should be specific, valid, reliable, sensitive, user-friendly, and cost-effective. Hunt (2011) suggests that gender equity indicators should measure: (i) differences in participation, benefits, outcomes, and impacts for women, men, boys and girls; (ii) changes in gender relations; and (iii) how these changes affect the achievement of development objectives, particularly economic growth, poverty reduction, and sustainable development (Hunt 2011 cited in ADB and AUSAID 2013).

Based on MAP-Norway’s experience, in this document, we critically assess MAP’s gender indicators. We identify the strengths and weaknesses of the indicators regarding how easily measurable they are and their power of explanation. As follows, we propose adjustments to improve the effectiveness and efficiency of the indicators. We expect that both the assessment and the adjustments proposed will be useful for future interventions with similar approaches to MAP-Norway (Table 2).

Table 2 Strengthens and weaknesses of MAP-Norway outcome indicators and the adjustment proposed
Full size table

In the experience of MAP-Norway, it is relatively easy to obtain most of the data used to inform the indicators used (indicators 2 to 7). These indicators provide an overview of women’s roles: productive, reproductive and business activities (1, 3, and 4), gender awareness (from 5 to 8), women’s participation (3 and 4) within an organization and changes in gender relations at the household level (1). Most of the indicators, however, do not provide an accurate assessment of what is happening within households and organizations in terms of gender relations. Indicators that rely on official documents or simply on the number of women in an organization fail to evaluate if there are actual changes in gender relations over time. The sheer increase in the membership of women in an organization could be the result of compliance with affirmative actions, quotas, or mandate, but that does not necessarily mean that men and women have equal voice and are treated equally. Such output indicators fail to capture changes over time in organizational culture, which is so important for genuine equity.

Regarding indicator 1, decades of research have shown how biased the opinions of one gender tend to be about the actions, beliefs, and goals of the other gender. MAP experience has also shown that data obtained from household heads alone show inflated participation rates of women (Mercado et al. 2015). MAP experience also suggests that observations of change in household decision-making require long-term monitoring, as such change does not happen quickly.

In some cases, the quantitative data is not sufficient and needs to be complemented with qualitative analyses, especially in cases when a development program is trying to assess changes in behavior or cultural norms that involve processes such as changes in gender roles within household, decision-making, women’s empowerment, or increased knowledge. These impacts are more difficult to measure; therefore, they are usually overlooked (Meinzen-Dick et al. 2004).

Analysis of qualitative data is usually more time consuming than quantitative data, but both are important to understand the impact of an intervention. Qualitative data could strengthen the understanding and interpretation of quantitative data (mixed methods approach), thus backing up statistical results. Mixed sampling methods, which are flexible, can be used in the collection of such qualitative data from small groups, with varying characteristics; qualitative data that can be used to strengthen the understanding and interpretation of quantitative data (Creswell 2003; Bamberger et al. 2010; Newing et al. 2011). This data can be collected through participatory techniques, such as key informant semi-structured and/or open interviews and focus groups. Such techniques build on quantitative analyses and allow participants the opportunity to build upon quantitative analyses and reflect upon their own situation and participation in a given program.

To complement the information obtained for our indicators, we conducted a series of focus groups to capture the opinion of men, women, and youth separately (24 in total). Each focus group represented an opportunity to analyze and reflect on varying MAP-Norway impacts and how participants perceived program benefits (Box 4).

Box 4. Assessing changes in decision-making and gender roles in the Central Region of Nicaragua and in Trifinio. Some quotes from the key stakeholders at local level

Changes in decision-making

Young participants pointed out that before MAP-Norway, only the father was in charge of household decision-making. At present, there is a broader participation, parents and children, now:

“We work in a more organized way” … “there is more communication between parents and children” (Young man Nicaragua).

Participation in commercialization

Women highlighted increased participation in use, access and control of income:

“We produce and make decisions together about going into the market… we have control over the money and now we have the power” (Women from Nicaragua).

5 Proposed indicators

Based on MAP-Norway’s experience, we suggest the following dimensions and indicators related to the productivity pillar of CSA. The indicators are organized in two levels: (i) at local level, we present indicators that allow the measurement of changes in household decision-making processes and (ii) at territorial level, we suggest indicators that allow us to measure gender equity in decision-making processes within business organizations, and the incorporation of gender equity principles in planning documents developed by governance platforms or governmental offices with presence in the territory. For each of the proposed indicators, we present the reasons that justify our suggestion and/or the cases where similar indicators were applied; use of existing indicators allows us to usefully compare results with other projects/programs and with other study families (beneficiaries and non-beneficiaries), making more evident the impact of the intervention (Table 3).

Table 3 Gender outcome indicators suggested for development programs in agricultural sector of Central America in terms of the productivity pillar of CSA
Full size table

In terms of adaptation and mitigation pillars, there is enough evidence that women living in rural areas of developing countries will face disproportionate effects related with climate change and climatic extreme events (Aguilar 2009; Denton 2000, 2002). Therefore, there is an urgent need to develop specific indicators for the adaptation and mitigation pillars. For example, the ratio of female headed households that increase efforts to collect water for household use and for use within the home garden; diversification of income sources; or increased number of women reporting the inclusion of trees in their productive system.

Data to measure the following indicators, related to the productivity pillar (Table 3) should be collected through mixed methods (surveys, semi-structured interviews, and focus groups), applying them in some cases not only to the heads of households but also to his/her spouse to capture both (men’s and women’s) perceptions. In the case of business organizations, platforms, or governmental organizations, the survey and/or semi-structured interview should be addressed to its representatives. In some cases, the information could be collected through participatory activities such as focus groups.

6 Recommendations

  • Mixed methods approaches (combining surveys, focus groups, and semi-structured interviews) have been found useful in assessing gender roles, as well as differences in participation and decision-making.

  • Project personnel must consider the gender of their respondents, interviewing both the head of household and his/her spouse, to minimize gender bias in responses.

  • Include indicators on how men and women perceive the changes generated by the program/intervention and if there is greater gender awareness in terms of productivity, adaptation, and mitigation to climate change.

  • Make sure that the indicators are measuring benefits and outcomes for women and men.

  • It is difficult to show transformative changes in people’s behavior or cultural norms because it takes time. Performance indicators must be included when developing a gender evaluation framework, differentiating advances within the impact pathway.

  • There is enough evidence supporting the need to have budget allocations that consider the relevance of qualitative indicators assessment.