Pilot testing and evaluation of a toolkit for menstrual hygiene management in emergencies in three refugee camps in Northwest Tanzania

Abstract

Displaced adolescent girls and women face many challenges managing their monthly menstrual flow with dignity and comfort in various challenging settings around the world, such as refugee camps, informal settlements, and while in transit across geographies as they flee disaster or conflict. Menstrual hygiene management requires easy access to safe, private water and sanitation facilities, along with appropriate menstrual materials and supplies, discreet disposal and waste management, and basic information on menstrual hygiene for displacement contexts. Yet, a significant gap exists in terms of available guidance on effective, coordinated multi-sectoral approaches for a complete menstrual hygiene management response. This paper describes one effort to address this gap, the development and pilot testing of the Menstrual Hygiene Management in Emergencies Toolkit in three camps hosting Burundian and Congolese refugees in Northwest Tanzania. Multiple methods were used to evaluate the implementation of the toolkit, which included a process and endline evaluation. Key findings included the identification of content gaps in the draft toolkit, the mapping out of a training and capacity building approaches needed for integrating menstrual hygiene management into ongoing programming, the relevancy and appropriateness of the guidance prescribed, and the potential for novel approaches to be identified by both water, sanitation and hygiene (WASH) and non-WASH sector actors subsequent to sufficient training. Important lessons from this exercise may be useful for the introduction of MHM programming into future global humanitarian response efforts.

Introduction

Displaced adolescent girls and women face significant barriers to managing monthly menstruation in a safe, private, and dignified manner. Humanitarian crises exacerbate menstrual management challenges, given girls’ and women’s frequent lack of access to basic materials and disposal (International Federation of Red Cross (IFRC) 2013; Kågesten et al. 2017). They must manage menstruation in overcrowded camps and informal settlements, which often lack private and safe toilets and sanitation facilities (especially at night) for changing and for washing materials (Kågesten et al. 2017; Parker et al. 2014; Schmitt et al. 2017; Sommer 2012). The disposal of menstrual waste (Parker et al. 2014) is often strongly influenced by cultural beliefs and societal taboos around menstrual blood and a need for discretion. This results in a variety of practices including burning, burying, or directly dropping materials into latrines (McMahon et al. 2011; Sommer and Sahin 2013). These issues may create safety risks including exposure to gender-based violence while women and girls discreetly access sanitation facilities (Sommer et al. 2014) or dispose of waste during the early morning or nighttime (Sahoo et al. 2015; Schmitt et al. 2017). In addition, basic information on menstrual hygiene management (MHM), especially targeting pubescent girls who may first experience menarche while displaced, is often scarce and particularly needed during more protracted emergencies (Martin and Anderson 2017; Schmitt et al. 2017).

A global review conducted in 2012 highlighted numerous issues around existing organizational responses for MHM in humanitarian emergencies, including an overall lack of evidence, practical guidance, monitoring measures, and coordination across sectors and organizations (Sommer 2012; Sommer et al. 2016). The majority of guidance available was often concentrated in the water, sanitation and hygiene (WASH) sector and did not incorporate the range of multi-sector actors needed for a cohesive and effective response (Sommer 2012).

In response, the International Rescue Committee (IRC) and Columbia University’s Mailman School of Public Health (CU MSPH) created a new research and practice partnership in 2015 to focus on three objectives for MHM in humanitarian response: (1) describing the evidence base, (2) developing effective cross-sectoral guidance for programming to improve MHM outcomes for girls and women, and (3) developing evidence-based monitoring measures. This effort culminated in the development of the Menstrual Hygiene Management in Emergencies Toolkit, a cross-sectoral resource (Sommer et al. 2017). The toolkit development process involved a range of research, piloting, and evaluation activities collectively aimed at achieving the three objectives. The toolkit was then piloted and evaluated in an ongoing humanitarian response to the displacement of Burundian and Congolese refugees in Tanzania.

The aims of the pilot and evaluation described here were to systematically monitor and describe the implementation of the toolkit in an ongoing humanitarian emergency, to capture in real-time the lessons learned from the perspectives of practitioners addressing MHM, and to refine the MHM in Emergencies Toolkit.

Methods

Preliminary activities

From 2015 to 2016, formative research was conducted to identify knowledge gaps around MHM in emergencies. This included a desk review of gray and scientific literature, key informant interviews with humanitarian staff from headquarters and regional offices of humanitarian organizations, and qualitative assessments with adolescent girls and women and response staff in two emergency contexts (internally displaced persons (IDP) camps in Rakhine State, Myanmar, and Syrian refugees living in host communities/informal settlements in Lebanon) (Schitt et al. 2017, Sommer et al. 2016).

The formative learning was also used for the development of the MHM in Emergencies Toolkit. The initial draft of the toolkit targeted four main sectors (WASH; Protection, including Women’s Protection and Child Protection; Health; Education). It was completed in February 2016 and disseminated across a network of multi-sectoral reviewers (n = 50) for further critique. In March 2016, a workshop brought together cross-sectoral humanitarian practitioners (n = 28) from 16 organizations to provide further technical inputs into the toolkit. The feedback ranged from identifying gaps in content and proposed sectoral involvement, improving the format and design, to advocating for the inclusion of more operational guidance to enhance cross-sectoral coordination. After integrating the feedback, the toolkit was readied for piloting in an ongoing humanitarian emergency, in October 2016 (see Table 1). Over 40 organizations were involved across all the phases of the toolkit development.

Table 1 Toolkit development process timeline

The toolkit document

The pilot draft of the toolkit consisted of 12 chapters covering needs assessment, materials and supplies, water and sanitation facilities, menstrual waste disposal, and guidance on integrating MHM programming into sectoral programming—including WASH, Protection, Education, and Health, and monitoring and evaluation (M&E). The toolkit also included a few simple diagrams, checklists, assessment tools, and training materials. The toolkit framework operates from the central premise that a complete MHM response includes three components: (1) menstrual materials and supplies, (2) MHM supportive facilities, and (3) MHM information (see Fig. 1). Two versions of the toolkit were piloted, including a full and a mini guide. The full guide included extensive technical guidance, case studies, checklists, and diagrams. The mini guide provided a concise summary of the more extensive resource as it was intended for readers that lacked sufficient time to review the larger text given competing priorities. The toolkit was provided in print and digital formats to over 14 organizations.

Fig. 1

Three essential components of an MHM response

Study setting

The pilot was conducted in three refugee camps located in the Kigoma Region of Tanzania: Nyaragusu, Mtendeli, and Nduta. When the pilot began in October 2016, there were 66,370 Congolese refugees currently residing in the Nyaragusu camp, many having lived in Tanzanian camps for 20 years (UNHCR 2016). Starting in the early 2015, a surge of Burundian refugees began crossing the borders due to political violence and unrest, with population estimates at 152,581 (UNHCR 2016). Nyaragusu is comprised of both Congolese and Burundians populations, and Mtendeli and Nduta designated for newly arriving Burundians. Nyaragusu had more permanent structures, including brick households and permanent schools, protection centers, and a hospital. The newly arrived Burundian populations were living in temporary shelters, including tents and corrugated metal structures. There were semi-permanent youth and women’s centers, schools, and health centers in Mtendeli and Nduta.

Twelve border points received refugees along the Burundian border staffed by IRC clinical staff member and community health workers. They conducted health screenings for the arriving refugees. In select locations, protection staff conducted gender-based violence screenings and referrals. Refugees would stay at the border point anywhere from 1 to 6 days, residing in large compounds, usually separating males from females and children.

International and local humanitarian organizations provided services, including WASH, education, protection, non-food item (NFI) distribution, health, and Camp Coordination and Camp Management (CCCM). MHM activities were usually stand-alone and primarily focused on the provision of reusable pads. The lack of a clear framework for a multi-sectoral MHM response made the context relevant for a pilot and evaluation of the toolkit. For example, a WASH actor was providing menstrual hygiene kits, in addition to setting up a reusable menstrual pad income generating group with select women in Nyaragusu. Another organization focused on the menstrual material needs of adolescent girls, providing menstrual hygiene kits and basic education to school-going girls in Mtendeli. Other organizations constructed female-friendly toilets for a secondary school in Nduta and developed small-scale reusable pad income generating schemes in Nyaragusu. As many humanitarian contexts today will have limited MHM activities underway, the Tanzania context provided an appropriate context in which to pilot the toolkit’s holistic, coordinated, systematic approach.

Implementation of the toolkit

Two staff members, one WASH and one protection, were based in the Kigoma Region to support the introduction of the toolkit. Prior to the launch of the pilot, two global WASH technical advisors (IRC) and one researcher (CU) joined the field-based staff to deliver a training of trainers’ workshop in October 2016 with 34 participants, ranging across sectors and 13 organizations and the camps and border points. The 2-day workshop focused on introducing the toolkit, discussion and sharing of learning and practices, exercises to identify the prioritization and scale of MHM needs, and discussions on enhancing coordination across organizations and sectors. Subsequently, the two-person (WASH and protection) field-based implementation team conducted technical trainings with managerial and field staff. They supported advocacy of high-level partners across clusters and organizations to solicit their collaboration through presentations at cluster meetings and advocacy for MHM inclusion in planning and activities. They also helped assess the feasibility of the indicators recommended in the draft toolkit. Lastly, they provided limited technical input for the implementation and facilitation of a few small projects aimed at testing new approaches to address specific MHM gaps identified, including generating learning on the process for integrating new MHM activities within existing operations and the acceptability of these approaches to the target population. For the actual evaluation of the pilot, a two-person CU team returned to the field to conduct data collection activities.

Assessment methods

There were three research components to the pilot implementation and evaluation: (1) formative assessment (October 2016), (2) process evaluation (November 2016 to April 2017), and (3) final evaluation (March to April 2017). This paper focuses on the findings generated from the process (2) and final evaluation (3) phases. Different methodologies were used to enhance the breadth of learning and triangulate findings. Data sources were primarily qualitative, but quantitative data were collected in terms of estimating numbers and proportions of women and girls affected and the outcomes of environmental inspections of MHM infrastructure. The methods used in the formative assessment are described in two published articles (Schitt et al. 2017, Sommer et al. 2016).

The M&E framework (see Fig. 2) utilized for the pilot examined the following objectives: (1) Was the toolkit implemented as intended, (2) What strategies enhanced capacity to carry out MHM among humanitarian organizations, and (3) Was the toolkit guidance both feasible to implement and acceptable by practitioners. Beyond these objectives, the team also sought to document more general learning that arose from observing the challenges with implementing MHM components into an ongoing response.

Fig. 2

Monitoring and evaluation framework

Formative assessment (October 2016)

The formative assessment was conducted at the onset of the pilot in the form of a rapid needs assessment which sought to provide a basic understanding of the MHM experiences and challenges facing girls and women from the perspective of both the refugee population and program staff. This included key informant interviews and Focus Group Discussions (FGD). In addition, a stakeholder analysis of key actors and agencies was also conducted, which included mapping out previous and ongoing MHM-related activities. This formative learning was utilized primarily to inform the content of the training of trainers’ workshop and to help identify key gaps in current response efforts.

Process evaluation (November 2016 to April 2017)

Monthly monitoring reports were used to track progress with the introduction and uptake of the toolkit, toolkit capacity building activities, and outcomes related to the desired impacts of improved MHM response capacity and coordination. This included tracking ongoing MHM programming, monitoring (needs assessment reports, post-distribution monitoring (PDM) reports), and trainings. Monthly environmental inspections of MHM supportive infrastructure were conducted at the onset and completion of the project to identify design gaps and areas for improvement, such as examining toilet facilities, washing facilities, and disposal systems at both the household and facility levels.

Final evaluation (March to April 2017)

Final evaluation activities focused on the effects of the toolkit and MHM response felt by beneficiaries and practitioners. This included (1) key informant interviews (KII) with cross-sectoral staff involved in roll-out activities, (2) Focus Group Discussions (FGD) with adolescent girls and women, and (3) KIIs with adolescent girls and women that had recently passed through the border points providing improved MHM services. Observational inspections were also conducted across the various camps, including sanitation facilities across all three camps (including schools, women’s centers, households) and both toilets and washing spaces at the border points.

Sample and recruitment

Purposive sampling with maximum variation was used to ensure diversity in the organizations, sectors and/or age groups, and camps selected for the KIIs and FGDs (see Table 2). Key informants (n = 21) from humanitarian organizations were interviewed with a semi-structured interview guide that included topics such as the acceptability of the toolkit, challenges and barriers to implementation, and perceptions of training activities. A diverse range of participants was selected from varying sectoral response types and levels. The sample for the FGDs included adolescent girls and women between the ages of 14-49 years. Protection and education sector staff from various organizations participating in the pilot were responsible for the recruitment and selection of women and girls to participate in the FGDs, with the number of participants recruited based on the learning from the formative assessments around saturation being reached. We reached saturation relatively quickly given the constraints of girls’ and women’s MHM experiences within the camp context. A larger amount of FGDs was conducted with the Burundian population given their more recent arrival in the camps and our focus on the current crisis. The groups were stratified into three age groups (14–18, 19–25, 26–49 years of age) to ensure better comfort and participation. Semi-structured interviews covered gaps and challenges related to MHM services, perceptions and limitations of existing MHM activities, and recommendations for improvement. A total of ten FGDs per age group (n = 6; total n = 117) occurred in the three camps and KIIs with adolescent girls and women for KII (n = 7) who arrived at border points providing MHM services.

Table 2 Number of participants

Data collection

Process and final data collection occurred over a 6-month period from October 2016 through March 2017. The pilot team staff were limited to supporting process evaluation data collection and documentation while the CU MSPH research staff (n = 3) conducted final evaluation activities. The formative data collection was conducted by a joint team from both institutions. All research activities were conducted in confidential settings, with female facilitators and a female translator fluent in Kiswahili (FGDs with Congolese participants) and Kirundi (FGDs with Burundian participants). All KIIs with staff were conducted in English. Kiswahili or Kirundi are the two primary languages spoken by the refugee populations. Given girls’ and women’s expressed discomfort with the use of the tape recorder, and noise pollution (e.g., rains on roof, crowded density of camps), tape recorders were not utilized. Instead, the FGDs and KIIs were conducted with a translator live translating responses into English. On average, FGD were completed in 65 min, while the KIIs with girls and women were conducted in 40 min and the KII with staff ranged from 1 to 1.5 h. To capture the maximum amount of data, careful note-taking was conducted by the two team members, capturing verbal and non-verbal responses. All participants provided oral informed consent prior to the data collection.

Data analysis

Data analyzed included monthly field notes from the implementation team, program documentation (reports, memos, presentations), FGD and KII transcripts, observational checklists, and environmental assessments. The monthly field notes and other program documentation were closely reviewed and categorized into a table to assess their relevance to the effects of the toolkit on capacity building, feasibility of implementation, and acceptability by practitioners. Two researchers (CU research team) reviewed the transcripts in Dedoose (Dedoose Version 7.0.23, 2016) collaboratively to determine key themes while debriefing across each step of data analysis. These themes were used to develop a codebook which supported the researchers in the application of a deductive content analysis methodology (Elo and Kyngäs 2008). This included systematically identifying the major themes arising, coding and condensing the text based on these thematic areas, and summarizing each thematic area identified. The significant themes from the data were shared with the larger research team for further validation and discourse. The results of the review of monthly field notes and the qualitative findings were integrated into the results described below.

All study procedures were approved by the IRBs from Columbia University, IRC, and the Tanzania Commission for Science and Technology.

Results

Three major thematic areas emerged from the analysis: (1) the fidelity of implementation of the toolkit setting including felt effects by beneficiaries and practitioners, (2) the enhancement of staff capacity, and (3) the acceptability of the toolkit.

Fidelity of implementation of the toolkit and potential effects

Over 12 organizations participated in training, toolkit dissemination, and technical engagement with the pilot team. The staff continuously conducted higher-level advocacy with the United Nations High Commissioner for Refugees (UNHCR), coordination and interagency, WASH, shelter, Camp Coordination and Camp Management (CCCM), non-food items (NFIs), and education working group meetings. This was concluded to be valuable in terms of positioning good MHM practice on the agenda during the period when standard operating procedures for various sectors were being drafted for improved response in the camps.

In terms of the uptake of the triad response (materials and supplies, facilities, and information), its complete application during the pilot implementation proved challenging. However, challenges became apparent from the provision of an insufficiently comprehensive response in the camps. For example, MHM supportive toilets (e.g., with a door, lock, water) introduced at some schools were not always used by girls because they had not been given appropriate menstrual supplies. As one Burundian adolescent girl explained:

We can’t [change at school] because when you wear the reusable [pads] it is difficult to change here. There is nowhere to put the used pads. Because we come with one we are wearing and don’t have a bag to put it in and there may be smells.

Through trainings and advocacy, the pilot team articulated the need for a more comprehensive response targeting the three key components of MHM (information, supplies, infrastructure, see Fig. 1). Indeed, the formative assessment indicated that prior to the toolkit introduction, distribution of menstrual materials was the most common MHM response activity being implemented. During the pilot period, partners were found to continue to prioritize materials distribution, including reusable materials (e.g., blanket coverage distributions), promotion of small-scale reusable pad making income generating schemes, and the provision of emergency disposable pads (at hospitals and protection centers). The evaluation indicated that the introduction of the toolkit enhanced the discussion around the appropriateness and adequacy of coverage of ongoing menstrual material distributions. This included efforts across organizations to standardize the type of menstrual materials and supplies being included during routine distributions of MHM kits to enhance consistency and coverage.

Table 3 Overview of small-scale MHM projects introduced by the pilot team

In addition, hygiene promotion education on how to use these new supplies (e.g., to reuse and not throw away pads) in the new MHM supportive facilities was also needed. In addition, incomplete distributions were observed that did not adequately incorporate the range of menstrual supplies and materials needed, such as the provision of reusable pads without an adequate supply of soap or incorrect sizes of underwear which hindered the ability to use reusable pads. Discussion of the toolkit, including its full guidance on MHM supplies, at working group meetings appeared to facilitate partners communicating with each other about the relevance and appropriateness of the items being distributed (e.g., underwear, reusable pads, a kanga (piece of local cloth), 10-L bucket, and soap), and the need to ensure that MHM kit content is not duplicated during distributions by various organizations and agencies.

The toolkit introduction was also observed to increase dialog around the need for improved sanitation infrastructure. For example, following the initial workshop, one WASH NGO requested technical support to address MHM-friendly sanitation facilities under construction at three schools in Nyarugusu camp. This led to joint assessments by the NGO and toolkit team to evaluate the proposed design based on toolkit standards and propose feasible corrective actions, such as ensuring locks and tighter doorframes, improved drainage mechanisms for washrooms to enhance privacy, and clearer delineation (and spacing) between male and female facilities. The same NGO subsequently introduced minor modifications (i.e., lockable doors) within toilet and bathing facilities at protection centers, other schools, and at the border points. For toilet facilities located at the household level, there remained a lack of improvements to toilet infrastructure subsequent to the toolkit introduction, including that communal facilities (toilets shared by multiple families) were deemed as not private, safe, hygienic, or comfortable locations for managing menstruation, especially at night.

In terms of menstrual waste disposal, the toolkit was not found to greatly influence practice for a range of reasons, including the shortage of timing and funds to cover a relatively large intervention. However, learning was gained in the challenges with disposal. The formative evaluation found relatively little attention among actors directed towards addressing disposal, which could have been partly related to a predominance of reusables being distributed. However, waste disposal was mentioned as a priority by WASH actors given the implications of menstrual materials for desludging latrines. Both Burundian and Congolese girls and women indicated strong preferences for putting used menstrual materials directly into household latrine pits. In addition, there were strong taboos surrounding menstrual blood and its linkages to witchcraft which posed a challenge to identifying other disposal approaches, and in particular, women’s fears that cleaners would be able to identify their used materials. At schools and protection centers where pit latrine and flush toilet were present, disposal issues were of greater concern given the frequency of blockages from menstrual waste. Overall, there remained a lack of consensus across organizations and agencies on improving disposal approaches, disposal and waste management design options, and appropriate strategies for engaging with girls and women given cultural sensitivities. These conceptual blockages appeared to delay movement in this area prior to and during the pilot.

The pilot team was also successful in helping to facilitate a few small-scale activities in conjunction with specific sectoral leads, both WASH and other sectors, which aimed to target particularly vulnerable populations (Table 3).

Consultation of girls and women on MHM was emphasized during the pilot by the Health WASH, NFI, and protection actors. This included increased efforts at women’s centers, reproductive health clinics, mental health centers, and schools using group discussions and interviews, similar to what is recommended by the toolkit. Although the majority of staff trained articulated the importance of direct consultation, it became apparent that many staff required direct support and coaching with initiating such activities, partly due to their own discomfort discussing menstruation. This need was illustrated by requests made from specific units and organizations for support in providing these consultations and tailoring existing guides from the toolkit to better suit their needs. The consequences of insufficient consultation also became apparent. For example, new MHM supportive toilets with a separate MHM cubicle were introduced in a few select school settings. However, upon hearing about these designated “MHM units” at the schools, girls immediately expressed concerns about the stigma that would surround the usage of such a cubicle. As one WASH officer from a local NGO explained:

…Ladies [female students] are suspicious [saying] ‘why is this the facility that is being used?...when I’m going [to the menstruation room}, I don’t want others to know.’ This special room for MHM shouldn’t be the case because when we go in then people will know.

The recommendation from girls was that all the cubicles should be menstruation units, indicating how direct beneficiary consultation, even if following the introduction of new facilities can identify uptake barriers and inform the design of future facilities.

Consultations did enable more clarification around girls’ and women’s menstrual materials for some organizations that appeared to harbor misconceptions. For example, many response staff suggested that the displaced girls and women preferred using the same materials they had used prior to displacement (strips of kanga or cloth) for managing their menstruation. As one international WASH officer described:

…not many [girls and women] like the reusable pads because it is different than what they used.

In the past, they used kangas and rags.

However, consultations with girls and women during the pilot indicated that the majority of girls and women preferred pre-made reusable pads if available. Reusable pads were perceived as reducing the likelihood of blood leaks onto their clothing and thus enhanced mobility during daily activities. As one Burundian woman explained:

…sometimes if we don’t have [reusable] pads, we can’t move from one place to another. So, we can’t move from home, we can’t go anywhere.

Reusable pads were also described as being better for those experiencing heavier bleeding which required them to change their cloth numerous times in a day. As another Burundian woman explained:

“…our bodies are different, some have more blood and some have little. For those with a lot of blood, the reusables are better. They can change up to 4 times a day and those are much better.”

When directly asked to select their preferences between cloth or reusables pads, several women indicated that their use of cloth prior to displacement was the consequence of poverty rather than preference.

One of the most effective examples of uptake (and hence feasibility) of the toolkit emerged from observations of activities being conducted by an NGO with multi-sectoral responsibilities. During an internal meeting that included the health, protection, and education teams, the NGO staff divided up current and future MHM needs into activities that could be carried out with existing resources, additional resources, and new creative and important ideas to be carried out if additional resources were acquired. Funds were identified for one of the creative and important ideas to be led by the health sector focused on the menstrual needs of refugees arriving at border points. Rapid consultations directly with girls and women indicated the range of challenges experienced while traveling or being received at border points, reception centers and camps. In response, an intervention was designed in coordination with the toolkit guidelines, to provide girls and women with menstrual hygiene kits at the border. A screening method was introduced to identify girls and women needing supplies in the form of a question embedded within their existing health screening protocol. The integration of MHM into the health sector team’s existing scope of work, including monthly reporting, ensured that MHM was packaged not as a new project but rather as a routine aspect of programming. The sanitation facilities at the border points were also improved with door shutters, locks, signage for sex segregation, and hand washing facilities with soap located close to the facilities.

The enhancement of staff capacity on MHM

In order to build staff capacity in the pilot context, recognizing that a holistic MHM response was a relatively new concept for many local and international staff, a range of different strategies were used to support toolkit uptake and promote the translation of key MHM concepts. This included supporting toolkit trainings that introduced the resource materials, targeting a range of levels (leadership, cluster-level, camp management, and field staff). The need for such trainings emerged during the course of the pilot, given the novelty of presenting MHM as a three-pronged strategy to be integrated into various sectoral response activities, and an identified need for more sustained guidance on how to mainstream toolkit recommendations. Overall, there was found to be broad consensus on the value of the toolkit and the training activities for improving basic MHM understanding and technical knowledge around an MHM response. The trainings were described by response staff as essential for ensuring that the toolkit’s key concepts were retained. As one health staff member explained:

Most people understood MHM after they went to training…like wow this is important. One engineer came to me and said, “this training was really helpful. Now if I’m planning a latrine I know I should put 1,2, 3, 4, 5 to support MHM. So that was just through training…we can keep explaining [to] see the big picture of women, to see what they are going through when they are menstruating.”

The trainings were also perceived as an important precursor to using the toolkit given their value for breaking down discomfort discussing menstruation. Several respondents indicated an improved comfort conversing on MHM following the toolkit introduction trainings. This included a reported improved capacity to discuss MHM with their colleagues of both genders and those working in other sectors who were needed to help inform program design considerations. A WASH engineer working in the Nyaragusu camp described the impact of improved staff dialog resulting from a training activity:

It is through the training, then because we are not afraid of each other when talking of these issues [menstruation]… we are not that much ashamed to give each other information. You are female and we are male and we must communicate about these issues. What is workable and what is clear – we can sit as designers and sector representatives [staff from different sectors] and they can link directly to beneficiaries and give us this information.

In addition, the training meetings were also observed to be useful opportunities for enhancing consensus on sectoral roles and responsibilities. This was articulated well by an education actor who explained:

…having them in one room and you train them…then everyone goes with the same understanding of how MHM should be handled in Education, in Women’s Protection, in CBR [community-based rehabilitation].

As in many emergencies, MHM is perceived to be a WASH responsibility; the toolkit trainings were also perceived to be important for convincing other sectoral actors that they had a role to play in supporting MHM and of the importance of cross-sectoral coordination. One community services coordinator described her shifting views on sectoral responsibility following the toolkit workshop:

I had a very different perspective because I never saw MHM as part of community services…It was always part of WASH for us. It was the responsibility of the NFI people. But after the workshop, I was able to see that actually it cuts through all the sectors. That it was the responsibility of all the sectors. It is very difficult to separate such issues now.

These training-related discussions were also perceived to be opportunities for clarifying sectoral roles and brainstorming on ideas about how best to integrate MHM into existing programming, with or without additional funding or resources. The pilot team intentionally presented MHM as not a new program or separate activity but rather a critical aspect of a response that should become an integrated component of routine response.

The trainings were also perceived as important tools for promoting the diffusion of MHM across sectors, organizations, and staff levels. This included ensuring that MHM was included on the agendas at cluster meetings involving a range of inter-agency actors and leadership. One WASH actor described the role of training in serving as a catalyst for the inclusion of MHM in coordination meetings and for breaking down taboos around discussing it:

I think that the workshop came at right time. Before that we were not really openly talking about that [MHM] as partners…when you brought us together, then even in our WASH coordination meetings we started to discuss it. Previously it was not discussed.

However, the potential impact of the toolkit was also seen as limited if its introduction, and the trainings were not also accompanied by strategies for generating high-level buy-in and recognition of the issue. This included ensuring that MHM was routinely discussed at cluster-level meetings. A high-level WASH advisor explained the importance for this dialog in terms of ensuring that an MHM response becomes routinized in emergencies:

What the “toolkit coordinator” did the other day in the WASH coordination meeting is important – more so than the workshop. The workshop people come and they listen and they forget about the toolkit. But once it is said constantly in a WASH coordination meeting. During those meetings… we report on water, we report on sanitation, we report on hygiene promotion, but let us also be reporting on MHM. For example, we have been distributing the pads, we have the challenge of disposal…it [MHM] needs to be echoed in these coordination meetings more and more if it will begin to stick in people’s minds.

Trainings however remain limited in their scope if not accompanied with inter-agency sectoral leadership, including cluster leads mandating the inclusion of MHM within routine reporting and activity updates over time. Most notably, a repeated observation heard throughout the pilot period was that despite an NGO having representatives at the initial training workshop conducted in October, most staff at the field level were not aware of the toolkit and appeared untouched by that training. Partners did not cascade the concept within their organizations after the first workshop to a large extent, thus necessitating more and more engagements in addition to efforts to reach organizational leaders.

The acceptability and usability of the toolkit

The pilot generated a range of practical insights related to the perceived value and acceptability of the toolkit by practitioners. Across sectors and organizations, there was consensus on the need for introducing structured guidance on MHM into response operations. Practitioners also noted the value of having MHM guidance tailored for emergencies as opposed to relying on existing development context resources, as one WASH actor explained: “I like both versions [short and full]. In the past, we never had any sort of toolkit… Last year, we were using puberty resources and they weren’t tailored for the emergency; it was for development work.”

The provision of the more streamlined mini guide (~ 34 pages) was seen as a valuable contribution given its brevity and summarization of key concepts of the 100-page full guide. Respondents also requested increased visuals, such as diagrams in the document, in order to enhance usability and information retention. As one shelter actor explained:

I didn’t get much time to pass through [the toolkit] but I have seen some pictures [diagrams]. If it can be made to look more like that [pointing to the diagram of a female friendly toilet.]

The diagrams were also perceived as especially useful for staff who may not speak English as a first language.

Findings also identified several important content gaps in the draft toolkit. For example, there was deemed to be a lack of content targeting activities relevant to the shelter sector. Both program staff and girls and women highlighted the challenges associated with MHM within shelters as girls and women frequently lacked any privacy or space for changing or drying menstrual materials in these spaces. A Congolese adolescent girl living in Nyaragusu described this issue, explaining how “it is a challenge to ever be alone. Sometimes you find many people in the house – mother, father, sisters, brothers. It becomes very difficult to change [menstrual materials].” In addition to household shelters comprised of families, the changing of menstrual materials was also identified as an issue in the communal shelters found at the border points and camp reception centers. As one health staff at the border point explained, “they have one single shelter with no partitions. So, if she needs to change her pad, she is not alone…so they have to go out [to change].” Recommendations were made by both staff and displaced women advocating for the inclusion of partitions or changing rooms in shelter structures to enhance the comfort of girls and women, especially for changing of materials during the nighttime.

Another key content gap identified included insufficient guidance on how best to support vulnerable populations. Special needs populations were also emphasized, such as girls and women with physical and mental disabilities and their caregivers. A community-based rehabilitation (CBR) coordinator further clarified the rationale for specific design considerations for toilet facilities for special needs girls and women:

…many cannot walk, or they have mental issues, and they are sharing the latrines with everybody. When they used the shared latrines, it is difficult. She will need extra care and extra consideration on what type of facilities should be designed. I don’t think it was included in the toolkit. There was not enough detail on that.

The men really suffer, they just do not know how and they came [to us] because they were anxious about how to handle these issues… these men are taking care of women; one client has a sister and a brother who are mentally challenges and he’s the only caregiver. He would come… for help about his sister’s menstruation. We did not have any guidelines on how to address it.

Another vulnerable population emphasized during the pilot was out-of-school girls, including those with physical and mental disabilities. For example, one Congolese woman with a mentally disabled daughter explained:

I have a child with mental problems. During the menstrual period for her, I share my own menstrual cloths with my child. I think it is a problem that I have to share my own menstrual materials with my children.

In addition, the unique needs of girls and women in transit (traveling or arriving at border points or reception centers) arose as a gap in the draft toolkit content. The rapid needs assessment highlighted the specific challenges they face, such as inadequate materials, challenges around washing themselves privately, and washing or disposing of menstrual materials. One woman described her challenges en route to the camps:

When we left Congo, it was a long distance to walk. Some of us were bleeding on the road, and we only had a kanga and underwear. It was very difficult. We had nowhere to clean ourselves.

The discreet disposal of frequently perceived to be a taboo menstrual waste, while on the road, created particular challenges for girls and women, requiring them to depart the road for bush areas or forests to privately change their menstrual materials. This in turn introduced safety concerns. Oftentimes, due to necessity, they indicated having to discard of used materials (such as stained clothes) directly on the roadside, materials that represented important items of their limited belongings. As one young Congolese woman explained, “I started to menstruate while I was on the road traveling. I was just wearing underwear so I pulled off the underwear and just threw it anywhere and put on a new pair.”

Another content gap identified by practitioners included a desire for additional training materials to support diffusion efforts around the topic of MHM and the toolkit guidance, with more information requested on how to best sensitize staff and generate buy-in across different levels. One NFI actor suggested a solution:

“I’d have to extract the information [from the toolkit] and present it in a very simple way. Some of these guys are not very good with English, but if I were to present it to my national staff I would present it in a way that they can understand. Depending on the country, if it was in Tanzania, I would present it in Swahili...Depending on the culture, I know some of the information is not okay with every culture. So, a trainer’s guide on how to use it [the toolkit] would be helpful.”

Other more specific technical areas identified for toolkit improvement included guidance on how to more accurately define the parameters for designing MHM supportive water and sanitation facilities. This subsequently led to the development of the minimum standards for female-friendly toilets and bathing spaces (see Table 4) in the revised final version of the toolkit. This table, developed after consultation with a range of WASH staff, aimed to provide a more uniform definition of the key components involved in the design and construction of supportive MHM facilities, taking into account the variation across institutional settings and household contexts.

Table 4 Minimum requirements for female-friendly facilities

Lastly, the MHM indicators developed for the toolkit were also extensively revised and simplified based on the feedback from practitioners. This simplification process involved consolidating them to fit within the three key components previously defined: (1) materials and activities, (2) MHM supportive facilities, and (3) MHM information. In addition, the draft monitoring tools and needs assessment guides were simplified based on the feedback expressed regarding their length and appropriateness for rapid implementation.

Discussion

This pilot and evaluation provided valuable insight related to the appropriateness and usability of the MHM toolkit during a humanitarian emergency. The findings indicated that the toolkit covered a useful range of content, was appropriately designed for the audience, and was valued by humanitarian staff as an important new resource. The critical importance of the trainings at both the onset and throughout the duration of the toolkit pilot was one of the most critical areas of learning, including the trainings’ importance as a tool for enhancing staff comfort in discussing the topic and for improving basic understanding, technical capacity, and promoting multi-sectoral coordination from the outset. Promotion efforts across the various levels of a given response also identified the need to mitigate the potential for MHM to be perceived as a standalone activity requiring additional work or new ownership.

The pilot demonstrated that like any new concept, there needs to be a longer-term plan for (1) training staff across levels, (2) internalizing the concept within humanitarian organizations, (3) and engagement and advocacy for higher-level coordination. Without these efforts, the likelihood of meaningful utilization of the toolkit for improved MHM practice is low. Therefore, uptake at the global and organizational levels would be interesting to explore further. As with other guidelines, like the Minimum Initial Service Package (MISP) for reproductive health in emergencies, a longer-term dissemination process that can foster strong buy-in across sectoral actors and humanitarian leadership can help to ensure that MHM is framed as an integrated and essential component of a routine response (Foster et al. 2017).

The study also generated valuable insight on the importance of non-WASH actors to identify and initiate new opportunities and roles to support MHM. As MHM responses are often led by the WASH sector in a given emergency, other sectors are often not clear on the scope of their supporting (or leading) role to ensure a comprehensive MHM response (Schmitt et al. 2017; Sommer et al. 2016). The pilot provided an important example of the health sector determining their role in addition to effectively engaging with WASH actors to address facility-focused improvements in areas of overlapping operations. Not only was this an effective example of health sector leadership in MHM but it also illustrates the importance of integrating MHM into ongoing operations. In addition, the health sector integrated an additional menstruation screening question into their existing questionnaire and began tracking this information during routine monthly monitoring, highlighting how MHM can harmoniously be integrated into existing programming if carefully framed and well designed.

The pilot study provided evidence that there is an ongoing need for implementing organizations to continue to document their learning and, when needed, carry out studies to build the MHM evidence base around cost-effective approaches to addressing MHM in emergency contexts. Additional evaluation of cross-sectoral interventions for improving MHM response in a range of response settings (e.g., camp vs. non-camp based, regional/geographic, urban vs. rural, different cultural beliefs around MHM) is essential. This includes developing and testing disposal mechanisms for menstrual waste, mechanisms for washing and drying materials discreetly, and the construction and maintenance of female-friendly toilets. This could be modestly done by first documenting and disseminating practical examples, as illustrated through the range of case studies generated for the toolkit, which have proven valuable for promoting best practices and innovation, anticipating barriers and mitigating the repetition of mistakes. For example, in the pilot, the preferences displayed and discussed by women and girls to dispose of materials in latrines may lead to developing and testing the uptake, use, and maintenance of deeper latrine pits for this purpose.

A future widespread uptake of the toolkit will be largely dependent on the translation of the key concepts and learning into the organization and sector-specific guidelines, operational plans, and funding proposals. Only then will the appropriate planning for MHM responses be incorporated into response plans and donor proposals that can be carried out by multi-sector NGOs and across sectors. Although the toolkit will serve as a valuable resource for a range of actors to use when trying to understand the scope of integrating MHM into response efforts, it is necessary for these various organizations and agencies to translate and personalize this learning to fit within their existing operational and M&E frameworks. The greatest uptake will likely occur when the main tenets of this toolkit are incorporated into cluster-level and/or organizational-level internal guidance documents (Child Protection Working Group 2012; Inter-Agency Standing Committee 2015; Inter-Agency Working Group on Reproductive Health in Refugee Situations 2011; Sphere Project 2011). This includes ensuring that MHM is appropriately addressed in a range of sectoral guidelines, program plans, job descriptions, monitoring tools, and other relevant documentation. Finally, the sustainability of MHM being included will remain contingent on its visibility and acceptance by the donor community at large coupled with consistent inclusion within funding proposals, objectives which although critical go well beyond the scope of this project.

Limitations

There are a few limitations of the pilot that are important to note. Although the recruitment of two dedicated cross-sectoral staff for leading the pilot project generated a great deal of value and learning to the pilot, it also created an artificial environment with a low likelihood for replication in future humanitarian responses (unless MHM focal points are hired for short-term assistance in the near future until MHM becomes more mainstream). However, given the organizational capacity constraints during the period of implementation and surging numbers of refugee arrivals, their presence was determined to be essential for achieving the pilot objectives. Constrained financial resources for implementation also posed a limitation, reducing the impact of the toolkit pilot, especially in terms of the capacity to design and test hardware related solutions (e.g., facilities, materials, and supplies). Another challenge, common to most emergency contexts, was the high rates of staff turnover. This likely diminished staff retention of toolkit content and highlights the need for repeated trainings and documented guidance until MHM becomes a more standardized response. External challenges included (1) the timing of the pilot at end of the fiscal year and (2) the rapid influx of new Burundians into Nduta and Mtendeli camps over the course of the project that in turn emphasized the difficulty for organizations to rapidly and coherently broaden the priorities beyond the provision of menstrual materials and supplies. Lastly, the small size of the toolkit research and development team (blinded – NGO/University) resulted in research member involvement across the formative research, toolkit drafting process, and pilot evaluation. To try and enhance the objectivity, the NGO implemented the pilot of the toolkit in the Tanzanian camps, while the university conducted the evaluation of the pilot.

Conclusion

The evaluation of the implementation of the MHM in Emergencies Toolkit during an ongoing response operation in Tanzania yielded an important practical learning related to the process for introducing new technical guidance for enhancing MHM support during emergencies. Multi-level trainings coupled with top-down leadership affirming the importance of MHM within routine operations were found to be essential to the success of the MHM response. Furthermore, enhanced clarity on both sectoral and organizational (NGO, agencies) roles in supporting MHM as a priority, which can be facilitated through more consistent cross-sectoral dialog and the integration of MHM across internal planning, budgeting, and trainings at both the organizational and donor levels. However, such mainstreaming will only prove possible if MHM, across each organizational tier, is perceived as an essential component rather than supplementary response activity.