Contributing to the construction of a framework for improved gender integration into climate-smart agriculture projects monitoring and evaluation: MAP-Norway experience

The Mesoamerican Agroenvironmental Program (MAP-Norway) is a multi-dimensional rural development program implemented in Central America since 2009, working with smallholder families, producer organizations, governmental organizations, and regional governance platforms. To monitor, assess, and evaluate the effects of the program on its beneficiaries, MAP-Norway uses a series of indicators that allow project managers and donors to adapt and follow-up on the interventions. Because gender is a cross-cutting theme in the program, gender indicators are used at all levels: families, producer organizations, and governmental organizations and governance platforms. In this document, we use the experience of MAP-Norway to critically assess these indicators, considering their potential usability in the monitoring and evaluation of climate-smart agriculture (CSA) initiatives. Furthermore, we propose a series of other indicators that capture various dimensions of gender relations. These indicators can be used to assess the effect of CSA practices, services, and technologies on equity in decision-making, women’s empowerment (including economic empowerment), intra-household food security, and equity in ownership over productive resources, among others, thus providing evidence that can help better design and target CSA interventions.

families, producer organizations, governmental organizations, and regional governance platforms. To monitor, assess, and evaluate the effects of the program on its beneficiaries, MAP-Norway uses a series of indicators that allow project managers and donors to adapt and followup on the interventions. Because gender is a cross-cutting theme in the program, gender indicators are used at all levels: families, producer organizations, and governmental organizations and governance platforms. In this document, we use the experience of MAP-Norway to critically assess these indicators, considering their potential usability in the monitoring and evaluation of climate-smart agriculture (CSA) initiatives. Furthermore, we propose a series of other indicators that capture various dimensions of gender relations. These indicators can be used to assess the effect of CSA practices, services, and technologies on equity in decisionmaking, women's empowerment (including economic empowerment), intra-household food security, and equity in ownership over productive resources, among others, thus providing evidence that can help better design and target CSA interventions.

Context
The Central American rural population represents 41.41% of the total rural population of Latin America. 62.7% of the people living below the poverty line in Central America are located in rural areas (PRESANCA 2011). For these people, agriculture represents their main livelihood. It contributes to food security (Tucker et al. 2010) and income. In Central America, family farming consists of more than 2 million families, accounting for about 50% of the Central American agro-industry sector and representing 96% of the total basic grain producers (PRESANCA 2011).
Women's role in agriculture has been overlooked (González and Macleod 2010;USAID 2013), even though women make up around 43% of the agricultural labor force in developing countries (FAO 2011). Rural women in Latin America and the Caribbean represent 20% of the labor force (FAO 2017). While official data indicates that more men than women are active in agriculture in Latin America and the Caribbean overall, agriculture constitutes 80% of women's economic activities in LDCs, including in Latin America. As well, women's work activities in subsistence agriculture are often underestimated or excluded for reasons such as limitation of data sources, gender norms about who is a farmer, and concepts and definitions adopted in data collection (UN 2015;Twyman et al. 2015). Besides their reproductive roles, such as caregivers for family members, women are responsible for producing grains, vegetables, and small animals for household consumption which play an important role in household food security. In rural areas in Latin America, women (and children) are also in charge of water and fuel supply for the family needs. These activities are often considered Bduties^and not Bwork.^Usually, social and cultural norms limit women's participation in public spaces, excluding women's voices from decision-making processes at the community level, and even though they have less access to production resources (seeds, inputs as well as technical and financial assistance), women play an important role in the transmission of local knowledge about certain agricultural and conservation practices .
Rural women are particularly vulnerable to the effects of climate variability and change due to their low adaptive capacity (IICA 2015). What makes women more vulnerable is that they have fewer endowments and entitlements than men, less access to information and services, increasingly heavy agricultural workloads (FAO 2011), unequal survival opportunities (limitations of mobility), lack of participation in decision-making (Stock 2012), and little control over key resources (Okali 2012). However, Acosta et al. (2015) point out that policy-makers should avoid portraying women as a group that is homogeneously affected by climate change, because that engenders an oversimplified vision of the problem and a discourse that tends to disregard nuances in the effects of socioeconomic and cultural factors. Diversity among rural women (indigenous, afro descendants, youth, farmers, etc.) is also overlooked.
A reduction of gender inequalities in agriculture could have positive results for the sector. According to FAO (2011: 5) BIf women had the same access to productive resources as men, they could increase yields on their farms by 20-30%, this could rise agricultural output in developing countries by 2.5-4%, which could in turn reduce the number of hungry people in the world by 12-17%^. Improvement in women's education is one of the most important policy strategies to reduce poverty and increase agricultural productivity, because increases in women's capabilities expand their opportunities and empower them to exercise their choices (Quisumbing and Meinzen-Dick 2001). Achieving gender equity, primarily through investing in women, may lead to greater reductions in poverty, faster economic growth, and significant improvements in family health, nutrition, education, and quality of life. Evidence demonstrates that gender integration makes development efforts more effective (USAID 2013). Furthermore, enhancing women's decision-making power also results in greater well-being of children, and households in general (Kabeer 2005).
Greater participation of women can be particularly important in technology adoption. Projects centered around the climate-smart agriculture (CSA) approach usually promote the adoption of technologies-as well as practices and services-aimed at increasing agricultural productivity while enhancing producers' climate adaptation and mitigation capacities. 1 If women have equal access to such technologies and practices and take ownership over the resulting benefits, CSA may have a more significant effect on family well-being. To better understand the relationship between CSA, gender, and rural livelihoods, there is a need for well-defined and efficient indicators (SMART indicators) that allow project managers and policy-makers to assess and evaluate CSA programs or interventions in terms of their impact on gender relations.

Mesoamerican Agroenviromental Program
MAP is a platform that links research, education, and technical assistance to support rural sustainable development while reducing rural poverty in Mesoamerica. MAP's first phase (2008)(2009)(2010)(2011)(2012)(2013) promoted strategies and practices for sustainable land management (SLM) using the sustainable livelihoods approach and the community capitals framework with a territorial approach (Gutiérrez-Montes and Ramirez 2016) to improve production, competitiveness, and to address environmental issues that affect the most important agricultural and natural resources sectors of the region. MAP's second phase, called MAP-Norway (2013-2017), promotes the climate-smart territory (CST) approach 2 to address issues such as poverty, food, and nutrition insecurity, gender inequality, degradation of ecosystem services and vulnerability to climate change (Box 1) (CATIE 2013).
Box 1 Definition of CST MAP-Norway defines a climate-smart territory as Bsocial and geographic spaces where the actors collaboratively manage ecosystem services to equitably improve human well-being, continuously optimizing land use and mitigation and adaptation to climate change^(Wallace Conference, cited in Louman et al. 2015, p. 77).
MAP-Norway focused its work at three levels: (i) local, with families; (ii) regional, with business organizations and territorial governance platforms; and (iii) national, with governmental organizations. MAP-Norway reached 5000 smallholder families, 30 business organizations, 8 territorial platforms, and 8 governmental organizations (CATIE 2013).
Gender strategy Both phases of MAP promoted gender equity and social inclusion to contribute to the creation of an enabling environment for human development. This has been captured in the gender strategy of MAP (Siles et al. 2012) and MAP-Norway (Siles et al. 2015). This last strategy comprises four main strategic axes (Box 2) which are addressed by the following actions: (i) to promote positive changes in gender roles at the household level, access, use, and control over resources, equal participation of household members in decisionmaking, and balanced distribution of the division of labor and responsibilities from within the household; (ii) to promote more equity in decision-making spaces in business organizations; (iii) to incorporate concepts of gender and equity in plans and co-management action plans, generated and disseminated tools and methodologies related to gender and equity; and (iv) to integrate a gender approach in all documents generated and in all program outcomes (Siles et al. 2015).
Box 2 Axes of MAP-Norway's gender strategy

MAP monitoring and evaluation system
MAP-Norway's monitoring and evaluation system (MAP's M&ES) is a tool to monitor its actions, as well as to measure progress toward expected results. The system is based on the program's logical framework and provides the option to perform simple or complex information queries .
To monitor progress throughout the period of implementation, MAP-Norway has used a set of quantitative indicators mapping to each of its five outcomes (Box 3). These indicators are grouped in (i) 22 performance indicators that measure progress of actions toward achievement of expected outputs, which are evaluated every year in the Annual Report (Mercado 2014 and2015), and (ii) 12 outcome indicators that measure progress in the change sought at the enduser level, both in the short and medium term. A baseline study was carried-out in 2013 (Beer et al. 2013), with two follow-up monitoring studies conducted in 2015 and 2016 . Gender equity, as a cross-cutting theme in MAP-Norway, is present at all scales of intervention: household/farm, business organizations, and territorial platforms 6 /governmental actors. Therefore, gender indicators are embedded within MAP's performance and outcome indicators. In this document, we give special attention to the outcome indicators. Table 1 presents the list of indicators assessed in this document.
Box 3 MAP-Norway's outcomes 1. 5000 families increased their assets with improved equity in family decision-making for food and nutrition security 2. 5000 farms and home gardens increased and agroecological and agroforestry production diversified 3. 30 business organizations increased entrepreneurial capacities and access to markets 4. 6 platforms improved enabling conditions to implement the Climate-Smart Territory (CST) approach 5. CST approach scaled up and out Data regarding household and farm activities were collected using a survey that was undertaken with heads of households. For business organizations, territorial platforms, and governmental organizations, a survey was undertaken with a representative of each organization and the information received triangulated with official documents such as strategic plans and annual operating plans. Finally, to assess changes at the outcome level, we used generalized linear mixed effects models with post-hoc tests for multiple comparisons and chi-square tests.

Assessment of the gender indicators used in MAP-Norway
According to Tayyib et al. (n.d.), indicators should be specific, valid, reliable, sensitive, userfriendly, and cost-effective. Hunt (2011) suggests that gender equity indicators should measure: (i) differences in participation, benefits, outcomes, and impacts for women, men, boys and girls; (ii) changes in gender relations; and (iii) how these changes affect the achievement of development objectives, particularly economic growth, poverty reduction, and sustainable development (Hunt 2011 cited in ADB and AUSAID 2013).
Based on MAP-Norway's experience, in this document, we critically assess MAP's gender indicators. We identify the strengths and weaknesses of the indicators regarding how easily measurable they are and their power of explanation. As follows, we propose adjustments to improve the effectiveness and efficiency of the indicators. We expect that both the assessment 6 Territorial platforms are considered a multiple stakeholders mechanism toward territorial governance and the adjustments proposed will be useful for future interventions with similar approaches to MAP-Norway (Table 2).
In the experience of MAP-Norway, it is relatively easy to obtain most of the data used to inform the indicators used (indicators 2 to 7). These indicators provide an overview of women's roles: productive, reproductive and business activities (1, 3, and 4), gender awareness (from 5 to 8), women's participation (3 and 4) within an organization and changes in gender relations at the household level (1). Most of the indicators, however, do not provide an accurate assessment of what is happening within households and organizations in terms of gender relations. Indicators that rely on official documents or simply on the number of women in an organization fail to evaluate if there are actual changes in gender relations over time. The sheer increase in the membership of women in an organization could be the result of compliance with affirmative actions, quotas, or mandate, but that does not necessarily mean that men and women have equal voice and are treated equally. Such output indicators fail to capture changes over time in organizational culture, which is so important for genuine equity.
Regarding indicator 1, decades of research have shown how biased the opinions of one gender tend to be about the actions, beliefs, and goals of the other gender. MAP experience has also shown that data obtained from household heads alone show inflated participation rates of women . MAP experience also suggests that observations of change in 3 Number of women in administrative and technical roles within the business organizations It is relatively easy to obtain the data.
Provide an overview of women's role within the organization.
Women could be included just to fulfill an affirmative action, a quota or a mandate. Should be complemented with a qualitative indicator, such as women's self-perception of her role within the organization 4 Number of women that are part of a Board of Directors within the business organization 5 Number of business organizations that have recruitment processes that address gender equity issues It is relatively easy to obtain the data.
It reflects that the organization is aware of the importance of inclusion of women.
To have gender considerations in official documents does not necessarily translate into more equitable treatment in practice Should be coupled with other indicators to ensure that these mechanisms are being applied effectively. Such indicators could assess qualitatively how men and women perceive gender relations within the organization.
6 Number of business organizations that have gender sensitive statutes Allows to track the continuity in the incorporation of the gender equity principle To have gender considerations in official documents does not necessarily translate into more equitable treatment in practice; Documents are not always readily available Should be coupled with data that indicates whether the actor is currently applying gender-related affirmative actions household decision-making require long-term monitoring, as such change does not happen quickly.
In some cases, the quantitative data is not sufficient and needs to be complemented with qualitative analyses, especially in cases when a development program is trying to assess changes in behavior or cultural norms that involve processes such as changes in gender roles within household, decision-making, women's empowerment, or increased knowledge. These impacts are more difficult to measure; therefore, they are usually overlooked (Meinzen-Dick et al. 2004).
Analysis of qualitative data is usually more time consuming than quantitative data, but both are important to understand the impact of an intervention. Qualitative data could strengthen the understanding and interpretation of quantitative data (mixed methods approach), thus backing up statistical results. Mixed sampling methods, which are flexible, can be used in the collection of such qualitative data from small groups, with varying characteristics; qualitative data that can be used to strengthen the understanding and interpretation of quantitative data (Creswell 2003;Bamberger et al. 2010;Newing et al. 2011). This data can be collected through participatory techniques, such as key informant semi-structured and/or open interviews and focus groups. Such techniques build on quantitative analyses and allow participants the opportunity to build upon quantitative analyses and reflect upon their own situation and participation in a given program.
To complement the information obtained for our indicators, we conducted a series of focus groups to capture the opinion of men, women, and youth separately (24 in total). Each focus group represented an opportunity to analyze and reflect on varying MAP-Norway impacts and how participants perceived program benefits (Box 4).
Box 4. Assessing changes in decision-making and gender roles in the Central Region of Nicaragua and in Trifinio. Some quotes from the key stakeholders at local level Changes in decision-making Young participants pointed out that before MAP-Norway, only the father was in charge of household decision-making. At present, there is a broader participation, parents and children, now: BWe work in a more organized way^… Bthere is more communication between parents and children^(Young man Nicaragua). Participation in commercialization Women highlighted increased participation in use, access and control of income: BWe produce and make decisions together about going into the market… we have control over the money and now we have the power^(Women from Nicaragua).

Proposed indicators
Based on MAP-Norway's experience, we suggest the following dimensions and indicators related to the productivity pillar of CSA. The indicators are organized in two levels: (i) at local level, we present indicators that allow the measurement of changes in household decisionmaking processes and (ii) at territorial level, we suggest indicators that allow us to measure gender equity in decision-making processes within business organizations, and the incorporation of gender equity principles in planning documents developed by governance platforms or governmental offices with presence in the territory. For each of the proposed indicators, we present the reasons that justify our suggestion and/or the cases where similar indicators were applied; use of existing indicators allows us to usefully compare results with other projects/ programs and with other study families (beneficiaries and non-beneficiaries), making more evident the impact of the intervention (Table 3). In our focus groups, we used the WEAI as the basis for guiding the sessions. We constructed the protocols based on the five dimensions of empowerment proposed by the WEAI. *Data should be collected through focus groups or in semi-structured interviews addressed to heads of household and his/her partner **Data may be collected through through focus groups or in semi-structured interviews addressed to heads of household and his/her partner providing a Likert scale In terms of adaptation and mitigation pillars, there is enough evidence that women living in rural areas of developing countries will face disproportionate effects related with climate change and climatic extreme events (Aguilar 2009;Denton 2000Denton , 2002. Therefore, there is an urgent need to develop specific indicators for the adaptation and mitigation pillars. For example, the ratio of female headed households that increase efforts to collect water for household use and for use within the home garden; diversification of income sources; or increased number of women reporting the inclusion of trees in their productive system. Data to measure the following indicators, related to the productivity pillar (Table 3) should be collected through mixed methods (surveys, semi-structured interviews, and focus groups), applying them in some cases not only to the heads of households but also to his/her spouse to capture both (men's and women's) perceptions. In the case of business organizations, platforms, or governmental organizations, the survey and/or semi-structured interview should be addressed to its representatives. In some cases, the information could be collected through participatory activities such as focus groups.

Recommendations
& Mixed methods approaches (combining surveys, focus groups, and semi-structured interviews) have been found useful in assessing gender roles, as well as differences in participation and decision-making. & Project personnel must consider the gender of their respondents, interviewing both the head of household and his/her spouse, to minimize gender bias in responses. & Include indicators on how men and women perceive the changes generated by the program/intervention and if there is greater gender awareness in terms of productivity, adaptation, and mitigation to climate change. & Make sure that the indicators are measuring benefits and outcomes for women and men. & It is difficult to show transformative changes in people's behavior or cultural norms because it takes time. Performance indicators must be included when developing a gender evaluation framework, differentiating advances within the impact pathway. & There is enough evidence supporting the need to have budget allocations that consider the relevance of qualitative indicators assessment.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.