Monitoring the Ebola Crisis Using Mobile Phone Surveys
- 7.3k Downloads
The recent proliferation of mobile phones enables the collection of high-quality data on an almost real-time basis. This chapter presents an innovative approach to data collection for crisis monitoring, combining baseline data from face-to-face household surveys with subsequent mobile phone-based interviews, based on the ongoing Listening-to-Africa project and data collected in Sierra Leone and Liberia during the 2014 Ebola crisis. It finds that mobile phone interviews are valuable tools for crisis monitoring and suggests that the proactive creation of sampling frames of respondent characteristics and re-contact information would allow for a rapid response to unforeseen crises.
1 The Data Demand Challenge
The outbreak of the Ebola virus disease in West Africa in 2014 constituted one of the gravest global health emergencies of recent years.1 The Ebola outbreak originated in rural Guinea in December 2013, and then spread across the country and to the neighboring countries of Liberia and Sierra Leone. The pandemic continued for two years and the World Health Organization (WHO) only declared Liberia free of Ebola in May 2015, Sierra Leone in November 2015, and Guinea in December 2015. By the end of the crisis, the epidemic had claimed more than 11,300 lives in these three countries, including over 500 frontline healthcare workers.2
In addition to its effects on people’s health, Ebola caused widespread economic disruption. At the height of the epidemic, schools, and markets were closed, government workers were placed on furlough, social gatherings were banned, transportation restrictions were placed on people and goods, and international borders were closed. Therefore, in addition to the health monitoring by the WHO, there was an urgent need for just-in-time data in order to monitor the economic impact of Ebola on livelihoods and wellbeing. Given the epidemic, however, it was impossible to deploy enumerators to the field to collect information from households and communities through face-to-face interviews.
2 The Innovation
The proliferation of mobile phone networks and inexpensive handsets has opened up new possibilities for data collection. Since 2012, the Africa region of the World Bank supports a mobile phone survey initiative called Listening to Africa (L2A). L2A collaborates with statistical agencies and offers the possibility to complement face-to-face household surveys with mobile data collection.3
The standard L2A approach starts with a face-to-face household survey that serves as a baseline. This baseline survey ensures that the randomly drawn sample is representative of the target population. During this survey, each respondent receives a simple mobile phone and when necessary, a solar charger. The respondents then receive calls from a call center every month, which conducts the mobile phone interviews. Survey questions are programmed in computer-assisted telephone interview software, allowing questions to be posed, and answers to be simultaneously recorded. The phone interviews are short so that data can be collected quickly, and respondents do not become overly fatigued. Data, once collected, are made available to the public.
The L2A approach has been introduced in several countries, including Madagascar, Malawi, Mali, Senegal, Tanzania, and Togo, and the L2A team has prepared a handbook documenting its experiences.4,5 A two-minute video explaining the L2A approach can be found on the World Bank’s website.6
When the Ebola crisis began in 2014, the World Bank team had accumulated several years of experience with mobile phone surveys. Building on the L2A model, high-frequency mobile phone interventions were designed to provide rapid monitoring of the socio-economic impacts of Ebola in Liberia and Sierra Leone.7 As the L2A approach had shown, baseline information was needed to anchor estimates in a representative dataset. Fortunately, there were recent surveys in both countries that could serve this purpose. In Liberia, the Household Income and Expenditure Survey (HIES) was being conducted as the crisis broke out, and was forced to curtail its fieldwork in August 2014. Though only about half of the sample (4075 households) were surveyed, it was nationally representative, and despite not being planned as a panel survey, had collected phone numbers and contact information for respondents. Overall, 57% of HIES households reported a mobile phone number for at least one household member. This database of phone numbers and household characteristics became the sample frame for the mobile phone survey sample. In total, five rounds of phone interviews were completed between October 2014 and March 2015. Data were collected by the Gallup Organization from their US-based call centers, as there was no suitably experienced call center on the ground in Liberia, and it was not possible to bring in international experts due to the travel ban. While using an external call center posed several challenges, including a lack of proficiency in local languages, unwillingness of respondents to speak to strangers, and a high costs of calling, the survey was able to conduct 2781 interviews with 1082 unique households over the five rounds.
In Sierra Leone, the 2014 Labour Force Survey (LFS) was also being carried out during the Ebola crisis, with fieldwork completed in July 2014. The LFS is a nationally representative survey, with a sample size of 4188 households. It was planned as a panel survey, and had therefore collected phone numbers and contact information, with 66% of LFS households reporting a mobile phone number for a least one household member. Using this database, three rounds of data collection were completed between November 2014 and May 2015. Data were collected through a call center at the national statistics bureau, Statistics Sierra Leone, supervised by Innovations for Poverty Action for the first two rounds and supervised directly by the World Bank for round three. The survey was able to reach 2111 respondents over the three rounds (Himelein et al. 2015).
3 Results from the Ebola Surveys
The Ebola surveys covered a wide range of topics, employment, agriculture, food security and prices, social assistance, remittances, migration, education, and health facility utilization. The team deliberately avoided asking questions directly related to illness within the household. Such questions were omitted for two reasons: first, to prevent non-response if households feared the authorities would come to remove ill members and, second, because the nature of the national sample was not well-suited to surveying disease incidences. The survey also included topics that were kept consistent in every round for monitoring purposes, such as those related to food security and economic activity, and some that were included in only one or two rounds based on the evolving situation. For example, the first round included questions as to whether the respondent had ever heard of Ebola and what sources of information they had on prevention. In later rounds, questions related to education were added, as schools reopened and social assistance as safety nets projects were rolled out.
Beyond the findings related to labor markets, the surveys provided important insights related to prices, food security, coping strategies, education, avoidance of healthcare facilities, and perceptions of public safety and trust in institutions. The surveys were able to monitor the usage of healthcare facilities for non-Ebola medical care. For example, the percentage of women in Sierra Leone giving birth in the previous two months in a hospital or clinic increased from 28% in November 2014 to 64% in February 2015 to 89% in May 2015. In some cases, these findings conflicted with the anecdotal evidence that had been previously guiding policy. In agriculture, farmers in both countries estimated that the production had declined, but to a lesser extent than had been feared, and with no evidence of the widescale abandonment that had been previously reported. In Sierra Leone, a delay in the arrival of seasonal rains also played a role. In education, once schools reopened, most students returned, 87% in Sierra Leone and 73% in Liberia. Of those that did not, the reason cited was monetary rather than fear of infection.
4 Implementation Challenges, Lessons Learned, and Next Steps
Although they cannot replace face-to-face household surveys in all contexts, mobile phone surveys offer substantial benefits in specific circumstances and for specific data collection needs. Advantages include the ability to collect data in volatile and high-risk environments (such as during political crises or epidemics), flexibility and responsivity to new data needs, timeliness, cost effectiveness, and utility for monitoring and impact evaluation. However, this approach remains challenging, and several lessons have been learned.
The risk of non-response and attrition applies to all panel surveys but is more likely for high-frequency mobile phone panel surveys. In the case of L2A, several strategies were undertaken to minimize these risks. Because sample selection did not consider prior ownership of a mobile phone, some households, particularly the poorest ones, had access to a mobile phone network but did not actually own mobile phones. To overcome this, mobile phones were distributed to all selected households, regardless of whether they already owned one, and respondents received training on various aspects of mobile phone ownership. In addition, the frequent power cuts in survey locations meant that phones could not be recharged, which could then lead to non-participation. To address these power cuts, small solar chargers were provided to allow households to charge their phones and receive follow-up calls.
In L2A, respondents were compensated each time they completed a phone interview, receiving a small amount of airtime credit transferred directly to their phones. This was both to compensate respondents for their participation, thereby encouraging them to stay involved, and to prevent the cancellation of phone numbers, which is a risk for those who do not ‘top up’ their phones after a certain period (usually 90 days). The lag period between the baseline survey and the first phone interview was also kept short. During the baseline survey, phone numbers were collected for all household members to increase the chances of reaching the respondent, and respondents were asked for their preferred call times. Efforts to track and trace hard-to-reach respondents also continued throughout implementation.
Another lesson learned was to keep the survey short. While households can and will participate in a mobile phone interview, the questionnaire must be kept short to minimize respondent fatigue, which can be a cause of attrition and non-response. Mobile phone-based surveys are therefore not appropriate for lengthy interviews or complex questions, such as those relating to household consumption. Mobile phone surveys also cannot substitute in-depth information that can be collected in face-to-face household surveys.
While fielding new ad hoc surveys to monitor an evolving crisis is possible (see Chapter 3), a more systematic approach is clearly preferable. If a representative mobile phone survey could be carried out on short notice, this would not only provide valuable real-time information, but could also be used to mount an effective response. The high-frequency mobile phone surveys to monitor the socio-economic impacts of Ebola in Sierra Leone and Liberia were possible because the most recent national household surveys had collected contact information. A proactive approach to crisis monitoring would start with the systematic creation (and maintenance) of databases with phone numbers and core household respondent characteristics. Another lesson from the Ebola crisis is that setting up a call center is relatively straightforward and can even be done from abroad.
Henceforth, the term Ebola is used to refer to the virus, the disease, or the epidemic outbreak.
World Bank (2016).
More information on this approach, including the instruments used, can be found on the L2A website: http://www.worldbank.org/en/programs/listening-to-africa. See also: Johannes Hoogeveen et al. (2014).
Dabalen et al. (2016).
A mobile phone survey was also conducted in Guinea but using a different methodology (World Bank (2016).
- Dabalen, Andrew, Alvin Etang, Johannes Hoogeveen, Elvis Mushi, Youdi Schipper, and Johannes von Engelhardt. (2016). Mobile Phone Panel Surveys in Developing Countries: A Practical Guide for Microdata Collection. Directions in Development. Washington, DC: World Bank. https://doi.org/10.1596/978-1-4648-0904-0. License: Creative Commons Attribution CC BY 3.0 IGO.
- Demombynes, Gabriel, Paul Gubbins, and Alessandro Romeo. (2013). “Challenges and Opportunities of Mobile Phone-Based Data Collection: Evidence from South Sudan.” Policy Research Working Paper 6321, World Bank, Washington, DC. Google Scholar
- Himelein, Kristen, Mauro Testaverde, Abubakarr Turay, and Samuel Turay. (2015). The Socio-Economic Impacts of Ebola in Sierra Leone. Washington, DC: World Bank Group. Google Scholar
- World Bank. (2014). The Socio Economic Impacts of Ebola in Liberia: Results from a High Frequency Cell Phone Survey (English). Washington, DC: World Bank Group. http://documents.worldbank.org/curated/en/428151468276303088/The-socio-economic-impacts-of-Ebola-in-Liberia-results-from-a-high-frequency-cell-phone-survey.
- World Bank. (2016). “Socioeconomic Impact of Ebola Using Mobile Phone Survey in Guinea.” World Bank Report No: ACS18659.Google Scholar
The opinions expressed in this chapter are those of the author(s) and do not necessarily reflect the views of the International Bank for Reconstruction and Development/The World Bank, its Board of Directors, or the countries they represent
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 3.0 IGO license (https://creativecommons.org/licenses/by/3.0/igo/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the International Bank for Reconstruction and Development/The World Bank, provide a link to the Creative Commons license and indicate if changes were made.
Any dispute related to the use of the works of the International Bank for Reconstruction and Development/The World Bank that cannot be settled amicably shall be submitted to arbitration pursuant to the UNCITRAL rules. The use of the International Bank for Reconstruction and Development/The World Bank's name for any purpose other than for attribution, and the use of the International Bank for Reconstruction and Development/The World Bank's logo, shall be subject to a separate written license agreement between the International Bank for Reconstruction and Development/The World Bank and the user and is not authorized as part of this CC-IGO license. Note that the link provided above includes additional terms and conditions of the license.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.