Experiences and Lessons From Implementing Cohort Event Monitoring Programmes for Antimalarials in Four African Countries: Results of a Questionnaire-Based Survey

Abstract

Introduction

Cohort event monitoring (CEM) is an intensive method of post-marketing surveillance for medicines safety. The method is based on prescription event monitoring, which began in the 1970s, and has since been adapted by WHO for monitoring the safety of medicines used in Public Health Programmes. CEM aims to capture all adverse events that occur in a defined group of patients after starting treatment with a specific medicine during the course of routine clinical practice.

Objective

The aims of this study were to describe the experiences of National Pharmacovigilance Centres (NCs) that have used CEM to monitor artemisinin-based combination therapy (ACT) for uncomplicated malaria in the African setting, to raise awareness of some of the challenges encountered during implementation and to highlight aspects of the method that require further consideration.

Method

A questionnaire-based survey was conducted to capture the experiences of NCs that have implemented CEM for active post-marketing surveillance of antimalarial medicines in sub-Saharan Africa. Six NCs were identified as having implemented CEM programmes and were invited to participate in the survey; five NCs indicated willingness to participate and were sent the questionnaire to complete.

Results

Four NCs responded to the survey—Ghana, Kenya, Nigeria and Zimbabwe—providing information on the implementation of a total of six CEM programmes. Their experiences indicate that CEM has helped to build pharmacovigilance capacity within the participating NCs and at the monitoring sites, and that healthcare providers (HCPs) are generally willing to participate in implementing the CEM method. All of the programmes took longer than expected to complete: contributing factors included a prolonged enrolment period and unexpectedly slow data entry. All of the programmes exceeded their budget by 11.1–63.2 %. Data management was identified as a challenge for all participating NCs.

Conclusions

The reported experiences of four NCs that have undertaken CEM studies on ACTs indicate that CEM has helped to build pharmacovigilance capacity within NCs and monitoring sites and that HCPs are willing to participate in CEM programmes; however, the method was found to be labour intensive and data management was identified as a challenge. Reducing the workload associated with CEM, particularly in relation to data management, and integrating the method into the routine work of HCPs and NCs should be considered for future implementation.

Introduction

The introduction of artemisinin-containing anti-malarial therapies in early 2000 for the treatment of uncomplicated malaria in endemic countries highlighted a need for studies that would yield more complete safety data in the post-authorisation period, especially under large-scale use where their safety had not been fully assessed. To meet this need, a modified version of prescription event monitoring [1–3] was proposed in 1998 [4] and was subsequently developed by the World Health Organization (WHO) as cohort event monitoring (CEM), a method of intensive post-marketing surveillance for medicines used in public health programmes. Although initially developed for monitoring artemisinin-based combination therapies (ACTs), the method has since been adapted for use in HIV/AIDS treatment programmes and is now being considered for use in tuberculosis control programmes [5–7]. CEM is intended for monitoring the safety of a new chemical entity in the early post-marketing phase, but is also suitable for monitoring older medicines with new indications [5].

CEM is a prospective, observational (non-interventional), cohort study that is undertaken early in the post-marketing phase of a new drug. The method is designed to capture all adverse events¹ that occur in a defined group of patients (the cohort) who are exposed to a specific, newly marketed medicine during the course of routine clinical practice [5]. CEM differs from its predecessor (prescription event monitoring) in that the cohort is enrolled by the healthcare provider instead of relying on prescription details supplied by pharmacies (a practice not common in resource-constrained settings).

Patients are enrolled in the cohort as they start treatment on the monitored medicine (treatment initiation); demographic information and medical information on the patient’s disease status, pregnancy status, past medical history, medication use and presenting symptoms is captured at this initial encounter. Any new medical events (change in clinical condition, new symptoms or diagnoses, or significant changes in laboratory parameters) that have occurred during a defined comparator period prior to starting the monitored medicine are also recorded at treatment initiation. Patients are then followed up after a defined interval (treatment review) to record any new adverse events that began after starting treatment with the monitored medicine, regardless of whether or not the drug was suspected to have caused the event. The information is sent to the National Pharmacovigilance Centre (NC), where each reported event is assessed for causality to determine the likelihood that the event was caused by the monitored medicine, based on the WHO Uppsala Monitoring Centre (WHO-UMC) causality assessment system [8].

By capturing all clinical events, regardless of suspicion of causality, CEM has the potential to identify previously unrecognised and unsuspected adverse drug reactions (ADRs). The cohort data provides a denominator for calculation of incidence rates and, because background health information is collected at treatment initiation, it may also be possible to identify risk factors for some ADRs.

The CEM method has been used to monitor the safety of antimalarial medicines in Ghana, Kenya, Nigeria, Tanzania and Zimbabwe [9]. Practical handbooks have been published by the WHO on how to conduct CEM in public health programmes for malaria, HIV/AIDS and tuberculosis [5–7]. Several CEM programmes have published their results from monitoring selected ACTs [10–13], and one CEM programme has recently published preliminary results from monitoring antiretrovirals [14]. Related publications identify challenges for implementing pharmacovigilance in resource-constrained settings [15] and propose strategies to complement spontaneous reporting for monitoring the safety of medicines in public health programmes [7]. This is the first paper to document country experiences of CEM implementation. In this paper, we report on the experiences of four NCs that have each implemented one or more CEM programmes for ACTs in the African setting, following the method outlined in the WHO publication ‘A Practical Handbook on the Pharmacovigilance of Antimalarial Medicines’ (the CEM handbook) [5]; we aim to raise awareness of some of the challenges encountered during the planning and implementation process and to highlight aspects of the method that require further development. The results of individual CEM programmes are beyond the scope of this paper.

Methods

A questionnaire-based survey was conducted to capture the experiences of African countries that have implemented the CEM method for active post-marketing surveillance of antimalarial medicines. The NCs of six African countries that had implemented a CEM programme were contacted to indicate their interest in sharing their experiences; five NCs agreed to participate. A questionnaire was developed in English and sent to the five NCs for feedback on the content and wording of the questions to ensure there was no ambiguity and that they could provide the required information. A final version of the questionnaire (see electronic supplementary material) was then sent out for completion by the NCs. In the event that a country had implemented more than one CEM programme, the NC was requested to fill in a separate questionnaire for each CEM programme. The questionnaire was circulated in October 2013 and responses were received between January and May 2014.

The questionnaire was divided into three sections. The first section concerned characteristics of the country, including information about the structure of the health system and pharmacovigilance in the country. The second section focused on the preparation phase, including the rationale for undertaking a CEM study, ethical approval, stakeholders and funding, development of programme tools, site selection, training and sensitisation. The third section focused on the actual implementation of the CEM study, including human resources, patient enrolment and follow-up, data management and monitoring and evaluation. Two final questions asked the respondents to consider the challenges and lessons learnt from undertaking the CEM study.

The responses from the completed questionnaires were entered into an Excel spreadsheet that was organised into the same sections as the questionnaire. Reponses from each country were compared and themes identified. These themes formed the basis for presentation of the results and subsequent discussions.

Results

Survey responses were received from the NCs of four countries—Ghana, Kenya, Nigeria and Zimbabwe; a fifth country had agreed to participate but did not send a completed questionnaire.

Profile of Participating Countries

Population data for each of the countries that responded and information on each of the NCs are presented in Table 1. All of the four participating NCs are located within the national medicines regulatory authority, as shown in Table 1, which also shows the year that each of the countries joined the WHO Programme for International Drug Monitoring and indicates the size and activity of each of the participating NCs.

Table 1 Characteristics of countries participating in the survey on cohort event monitoring (CEM) programme implementation

Cohort Event Monitoring (CEM) Programme(s) and Stakeholders

The four NCs undertook a total of six CEM programmes. All six programmes monitored ACTs, particularly artemether-lumefantrine (AL) and artesunate-amodiaquine (AA). Table 2 provides an overview of each of the CEM programmes.

Table 2 Overview of the cohort event monitoring (CEM) programmes

The NCs in all four countries coordinated implementation of CEM in collaboration with other stakeholders such as the national malaria control programme, selected healthcare institutions/sentinel monitoring sites across the country, national and international non-governmental organisations (NGOs), the WHO and marketing authorisation holders.

Development and Pre-Testing of CEM Data Collection Tools

The data collection forms used in the CEM programmes were based on a template provided in the CEM handbook [5]. Each programme adapted the template to suit their specific needs. Four of the six CEM programmes pre-tested the data collection tools by enrolling a small number of patients at each monitoring site using the forms. Nigeria did not pre-test the tools used during the pilot programme; however, because the forms used during the pilot programme were used during the scale-up (with slight modifications), the pilot served as a pre-test for the scale-up programme.

Training

All participating NCs trained the CEM site teams on the principles of pharmacovigilance and the CEM method prior to commencement of the programme. The training was either conducted centrally, where master trainers were trained and equipped to train other members of the CEM team, or through training meetings and/or on-site training for site personnel. High staff turnover at some monitoring sites and NCs necessitated frequent training for new members.

Cohort Size and Monitoring Sites

Details of the number of patients enrolled into the cohort in each CEM programme, the number, type and urban/rural distribution of monitoring sites and the human resources requirements are provided in Table 2. The criteria for site selection included regional representation, interest and willingness of the institutional contact person to participate in pharmacovigilance and CEM, active participation of the site in pharmacovigilance (measured by the number and quality of ADR reports sent to the NC), malaria burden in the locality and accessibility of the site. In addition to the public sector health facilities, two programmes engaged private sector community pharmacies for patient monitoring.

Use of Incentives

All programmes reported using incentives to encourage healthcare providers (HCPs) to participate in the programme. The reasons given for using incentives were to motivate the HCPs to collect quality data, to compensate for their time, for logistics and transport support and to ensure their commitment to the programme. The nature and quantity of incentives was determined independently by each country based on their operating environment and local context. Details of the incentives provided to HCPs are shown in Table 3.

Table 3 Planning and implementation of the cohort event monitoring (CEM) programmes

Most respondents considered that it would not be possible to undertake CEM without providing incentives for HCPs. The reasons stated were that CEM is demanding and time consuming and since many HCPs were already overworked, it is unlikely that they would take on additional work (especially work they consider to be outside their primary responsibility) without an incentive. Only one NC considered that it would be possible to undertake CEM without incentives because of the “availability of professionals who would be willing to participate” but warned that the “response rate might be low”.

Five of the CEM programmes also provided incentives for patients. The nature and quantity of the patient incentives are detailed in Table 3. Opinions on the feasibility of conducting CEM without incentives for patients were equally split between the six programmes. In Zimbabwe, where no incentive was given to patients, it was reported that patients saw the monitoring as part of their treatment and were appreciative of the treatment, especially at a time when the country was going through a financially challenging period. The NC in Ghana offered reimbursement of transportation costs by way of incentive, but reported that “over 98 % of follow-up was by telephone call” and “most patients did not take [the incentive] even when they came back to report ADR”. The NCs in Kenya and Nigeria, on the other hand, both considered that the programme would be unsuccessful without the use of incentives for patients.

Duration of Enrolment

Table 3 shows the target and actual time taken to enrol the cohort. The time taken to complete enrolment exceeded the targeted timeframe by 50–100 % in all but one of the studies; only the Nigeria Pilot Programme was able to complete enrolment within the projected timeframe.

CEM Data Management

All responding countries reported using paper-based data collection forms. All completed CEM forms were sent to the NC for centralised data entry into CemFlow [16]—an electronic data management tool developed by UMC at the request of WHO, specifically for CEM studies. Completed forms were transmitted to the NC by a variety of means including courier, hand delivery by site staff, pick up by NC staff or leveraging of existing in-country distribution channels such as those used by Ministry of Health or public health programmes to distribute commodities within the system. The time required for data entry ranged from 4 weeks to 2 years. The number of people involved in data entry and the time taken to complete the task varied across the studies. Some studies such as the Kenya CEM programme for monitoring AL used dedicated data entry clerks to enter the CEM data over a period of 4 weeks, while other programmes such as the Nigeria Pilot CEM study used temporary staff to enter data over a long period of time. The Kenya NC reported that the CEM data management tool, CemFlow, “should be made more user-friendly” and the Ghana NC commented that the “data entry process has been one of the most challenging aspects of the study”, citing concerns over the use of pharmacovigilance staff rather than data entry clerks for data entry, incomplete development of CemFlow and a lack of correlation between the data collection forms used in their programmes and the CemFlow data entry interface.

Cost of CEM

The actual cost of the CEM programmes exceeded the budgeted cost by 11.1–63.2 %. The funding sources for each programme are shown in Table 2. Two of the programmes also obtained non-monetary contributions such as insecticide-treated nets from stakeholders. The prolonged enrolment period and unexpectedly slow data entry added to the overall time and cost of the programme.

Effect of CEM on Spontaneous Reporting

The NCs of Nigeria and Zimbabwe reported a positive effect of CEM on spontaneous reporting of ADRs while Ghana and Kenya reported a reduction in the number of ADR reports received by their national spontaneous reporting programme from sites participating in CEM.

Challenges and Lessons Learnt

The key challenges and lessons learnt identified by each of the NCs in the course of implementing CEM are summarised in Table 4.

Table 4 Key challenges and lessons learnt identified by each cohort event monitoring (CEM) programme

Discussion

We report on the experiences of four African NCs that have implemented the newly described CEM method for active post-marketing surveillance of ACTs. Their experiences of undertaking a total of six CEM programmes provide useful information on the requirements and challenges of introducing safety monitoring of intensive medicines in resource-limited settings. It is important to document the experiences of these early implementations to inform further development and refinement of the method and to shed light on some of the practical issues that need to be considered when planning such a programme in this setting. The survey focused only on practical issues of implementation and did not cover issues relating to data analysis or the study results.

Overall, each of the NCs reported similar experiences. All of the CEM programmes took longer than expected to complete. A number of unforeseen delays contributed to the prolonged timeframe for implementation, including the time required to obtain individual informed consent, strike action by healthcare workers, seasonal variation in malaria cases and lower than anticipated incidence of malaria in some regions (which prolonged the patient enrolment period in some programmes), shortage of monitored medicines, insecurity in some regions, delays in disbursement of funds, attrition of trained site personnel and insufficient data entry staff leading to unexpectedly slow data entry. One programme also reported that the time taken to obtain ethical approval was longer than anticipated, contributing to a delay in starting the study.

Although it is not possible to plan for all contingencies, awareness of potential pitfalls may help to avoid some of the factors that contributed to the extended study duration in these programmes.

Site Selection

To facilitate timely enrolment, the monitoring sites should be selected to include regions with the highest prevalence of the disease for which the monitored medicine is used. Site selection should also take into consideration the representativeness of the cohort, the willingness and capacity of the HCPs to participate in the monitoring and the accessibility of the sites. Selection of sites in these CEM programmes was predominantly influenced by a need for representative geographic distribution, although malaria prevalence and willingness of HCPs to participate were also seen as important factors. Less frequently reported factors were the prior experience of ADR reporting, accessibility of the sites and health sector representation.

Two of the CEM programmes (the Nigerian Scale-Up and Zimbabwe studies) engaged private-sector health facilities (community pharmacies) for monitoring the ACTs. Up to 82 % of all malaria episodes in sub-Saharan Africa are managed outside the official health sector and the private sector accounts for 40–60 % of all antimalarial drugs distributed, with unofficial sources such as street sellers and market stalls accounting for as much as 25 % [17, 18]. Including community pharmacies in the monitoring programme may help to increase the rate of enrolment and enhance the representativeness of the cohort. In the programmes where community pharmacies were engaged, enrolment still took longer than expected, but the NCs cited a number of other problems (Table 4) that delayed their progress.

Timing of CEM Implementation

The timing of the monitoring should allow for seasonal variation in the disease prevalence, and monitoring of ACTs should be planned to coincide with the peak malaria season. The CEM programme for ACTs in Kenya reported that a number of unforeseen delays (e.g. strike action by nurses, doctors and pharmacists and the Muslim Ramadan period in some areas) extended the planned monitoring period beyond the malaria season, thereby prolonging the overall time taken to complete the enrolment. Only three of the six programmes met their enrolment target, and only one (Nigeria Pilot study) achieved their target within the anticipated time frame.

Informed Consent

The time required to inform patients about the purpose of the monitoring and to obtain their informed consent was identified as a factor in prolonging the study duration in one of the studies. The CEM Handbook warns that explaining the rationale and requirements of the monitoring programme to individual patients will be time consuming, increase complexity and add to the cost—a concern that proved true in practice. A further caveat in the handbook stated that a requirement for formal informed consent could potentially compromise the validity of the results if many patients refused to be enrolled [5]. CEM is a non-interventional, observational study; all patients who are prescribed the monitored medicine during the course of routine clinical practice and who are willing to participate are eligible for enrolment in the cohort. Consent may be required to collect their personal health information and to be contacted for a follow-up interview. For CEM programmes where obtaining informed consent is a requirement by the ethics committee, NCs need to carefully plan how to obtain full informed consent, taking into consideration the time required to explain the purpose of the study to patients and how their data will be stored and used, so that sufficient resources can be allocated.

Unforeseen challenges that were reported in relation to obtaining informed consent included socio-cultural barriers such as women requiring their husband’s permission to give consent, apprehension about signing the form, concerns about HCPs calling to enquire about treatment progress and communication barriers created by low literacy levels.

Data Management

The UMC, in collaboration with WHO, developed a data management tool, CemFlow, specifically for CEM studies. The tool was still under development at the time these CEM studies were implemented and was not fully optimised for data management. Direct data entry at the point of care was technically possible with CemFlow, but limited IT capacity at the monitoring sites made it impracticable; hence, all of the CEM programmes used paper-based data collection forms with subsequent centralised manual data entry into CemFlow.

All of the NCs reported experiencing challenges with data management. The CEM method requires data capture at each patient encounter. For ACTs, which generally involve a 3-day course of treatment, CEM requires a data collection form to be completed at the time of treatment initiation and at treatment review after a specified period of time, thereby generating at least two forms per patient. Consequently, the amount of data to be manually entered into the data management tool is very large. For example, the Nigeria Scale-Up programme, in which 10,260 patients were successfully followed-up after treatment initiation, necessitated manual data entry into CemFlow from at least 20,520 paper forms. All of the NCs reported that they had insufficient dedicated data entry clerks, and additional data entry clerks, including NC pharmacovigilance staff, were enlisted to complete the task.

It is worth noting that CEM of ACTs, with just two forms per patient, requires considerably less work than would be generated by a CEM study for a longer term therapy such as an antiretroviral medicine, in which patients would need to be followed up multiple times over a longer period (e.g. monthly for a year). NCs planning to implement CEM, especially when centralised manual data entry is unavoidable, must consider how to effectively manage the data that will be generated, including having an adequate number of staff for data entry.

The long-term solution may be the increased use of electronic health records (EHRs) that enable signal detection in longitudinal health data [19]. In the shorter term, EHRs and other digital technologies such as mobile phone applications that facilitate electronic data capture may be developed to reduce the workload associated with CEM. Access to computers, stable Internet connections and a constant electricity supply remain a challenge in many African countries [20, 21], but mobile phone technology is now widespread. Mobile phone ADR reporting apps have already been developed and are in use, for example in Kenya [22], and could be considered as a possible reporting tool for CEM studies.

Healthcare Providers’ Participation

All NCs reported an initial high level of enthusiasm by HCPs, which waned with time to a level of almost reluctance to continue. The initial enthusiasm shows that HCPs in resource-limited settings are willing to participate in pharmacovigilance activities. The reasons behind the waning interest were not solicited in the questionnaire.

There appears to be a perception among HCPs, especially in developing countries, that CEM, and by extension pharmacovigilance, falls outside their scope of practice. This perception is reflected in the response to the question “How would the (monitoring) sites best describe the additional workload associated with CEM”. All of the NCs responded that the monitoring sites considered that CEM interfered to a great extent with their routine work. These responses suggest that the HCPs involved in these CEM programmes had not fully appreciated the rationale for undertaking the CEM study and that pharmacovigilance activities should be considered an integral component of patient care. Although the number of developing countries that have joined the WHO Programme for International Drug Monitoring has increased sharply in recent years [23, 24], pharmacovigilance in many of these countries is not yet seen by HCPs as contributing to clinical decisions and improving treatment outcomes. There is a need for greater pharmacovigilance advocacy and training for HCPs to encourage their ongoing participation in future CEM studies.

The effect of diminishing returns may also have played a role in the loss of enthusiasm reported by each of the NCs. Many of the HCPs may have lost interest when the programme that was intended to be a short-term project extended beyond the expected timeframe. NCs that are planning to implement CEM need to carefully estimate the time commitment that will be required of participating HCPs, and endeavour to integrate data collection into their routine patient care activities.

Despite the waning of enthusiasm, the NCs reported positive experiences in relation to the participation of HCPs, including improved patient–HCP interaction, greater understanding of pharmacovigilance and more rational use of ACTs.

Cost of CEM

In all of the programmes, the actual expenditure on CEM exceeded the budget by 11.1–63.2 %. Factors that contributed to budget shortfalls included the unexpectedly prolonged study duration and the need to hire additional data entry clerks. A breakdown of the budget was not included in the survey questionnaire.

The use of incentives for HCPs (and, in most cases, patients) added to the cost of the CEM programmes. Although most NCs considered that implementing CEM would be difficult without the use of incentives, this is a potential target for cost reduction. Another target for cost reduction is the workload associated with patient enrolment and data entry. Electronic data capture in CEM studies would reduce the time and labour required for data processing.

Effect of CEM on Spontaneous Reporting

Two NCs reported a positive effect on spontaneous reporting of ADRs while two reported a reduction in the number of ADR reports from sites participating in CEM. The probable explanation for the observed reduction is that the same people who would have reported ADRs at the sites were engaged in CEM, thus leaving them with little time to routinely report ADRs. However, the survey responses indicate that the experience of implementing CEM helped to build pharmacovigilance capacity within the NCs and the monitoring sites, which can be expected to have a positive effect on routine pharmacovigilance activities in the long run [25].

Limitations of the Survey

The questionnaire was validated by the same people who completed the survey, but foreknowledge of the questionnaire content was thought unlikely to compromise the survey results in any way. Some of the questions were directed to the HCPs at the monitoring sites; however, it is not clear whether the NCs solicited responses for these questions from the sites or responded on their behalf. Thus, the responses may not be an accurate reflection of how CEM was perceived by the monitoring site personnel. The questionnaire was also limited in the depth of information required from respondents. It did not enable a probe into the reasons for issues such as the delay in obtaining ethical approval experienced by one of the NCs, information on individual cost items and their relative contribution to the total cost of CEM, and the waning interest of HCPs participating in the programme.

Conclusion

This survey documents the experiences of four African NCs that have implemented the CEM method for monitoring ACTs. Their experiences indicate that CEM has helped to build pharmacovigilance capacity within the NCs and monitoring sites, and HCPs are generally willing to be involved in implementing the CEM method. Pharmacovigilance advocacy and education towards integrating patient monitoring into the routine patient care activities of HCPs will improve appreciation of CEM as a complimentary tool for drug safety monitoring. Reducing the workload associated with CEM, for both the HCPs and NC staff, particularly in the area of data management, should be considered a priority for further development of the method.