Introduction

Estimates suggest that between 2010 and 2019, immunization alone has prevented over 37 million deaths globally [1, 2]. Moreso, immunization brings added value to communities by reducing families’ health expenditures and improving countries’ productivity and resilience. For example, some studies have reported that immunization returns an estimated US$16 for every US$ 1 invested, and this return can reach US$44 when broader social and economic benefits or vaccination are considered [3, 4]. But despite these benefits, national and subnational vaccination coverage rates have plateaued in several countries, particularly those in sub-Saharan Africa (SSA), where coverage rates for basically all antigens are considerably lowered compared to other regions [1].

One key contributing factor to the above phenomenon is the paucity of timely and high-quality immunization data. Indeed, in many settings, existing data systems can rarely guarantee the generation of timely and good-quality data at all health system levels - a weakness that can have dire consequences on various aspects of immunization [5, 6]. For instance, inadequate immunization data quality can lead to missed opportunities to identify under-immunized or zero-dose children, poor defaulter tracking, and inaccurate vaccine coverage estimation [6]. This, in turn, could undermine national and international investments and increase the risk of vaccine-preventable diseases (VPD) by failing to identify populations with low vaccination coverage [7].

Gavi, The Vaccine Alliance made data quality a strategic focus area in 2015, highlighting quality immunization data as a ‘critical enabler’ to achieving immunization targets [8, 9]. Yet, poor immunization data quality continues to be a pervasive problem across many Gavi-eligible countries. For example, a study assessing the validity of reported vaccination coverage in 45 Gavi-eligible countries found that 75.5% of officially reported coverage figures from administrative sources were at least 10% higher than national survey estimates [10]. Similarly, in a study evaluating the quality of immunization monitoring systems in 27 Gavi-supported countries (including Cameroon), Ronveaux et al in 2005 reported that only 60% of health units had data recording forms (immunization registers and tally sheets) available for a whole year, and that only 59% of health workers completed a vaccination card properly [11]. Similarly, a study carried out in North-West Ethiopia noted that nearly half of all surveyed health facilities did not have standardized immunization registers and tally sheets available [12]. Lack of critical immunization data collections tools as well as inadequate data recording practices invariably affects a country’s immunization coverage estimates.

In Cameroon, remarkable progress has been made to improve national immunization coverage, since the inception of Cameroon’s Expanded program on immunization [13]. However, despite making noteworthy progress, coverage rates for many antigens still fall below set targets [14, 15]. A few studies have attempted to examine the performance of different components of Cameroon’s immunization system [16,17,18,19,20,21], exploring possible barriers and facilitators to optimal immunization coverage. However, to the best of our knowledge, there is no published study evaluating the data management practices of frontline immunization staff in Cameroon. This study was, therefore, designed to assess the data management practices of immunization staff in healthcare facilities, and it unveiled several barriers to high-quality and timely immunization data in Cameroon.

Materials and methods

Study design

This was a descriptive cross-sectional facility-based study carried out as part of a national baseline assessment to identify and describe factors contributing to declining immunization coverage in Cameroon.

Study setting

The study was conducted in Cameroon, a country located in Central Africa, at the Gulf of Guinea, with a population of approximately 28 million inhabitants, and covering a total surface area of 475,440 km2 [22]. The country is divided into 10 administrative regions, namely Adamawa (AD), Centre (CE), East (ES), Far-North (EN), Littoral (LT), North (NO), North-West (NW), West (OU), South (SU) and South-West (SW) regions [23].

Cameroon’s health sector is organized into three main levels (central, intermediate, and peripheral), with each level having specific competencies, administrative, health and dialogue structures [24]. The central level is headed by the ministry of public health and focuses mainly on the development of policies, strategies, and coordination. The intermediate level is headed by the 10 regional delegations and this level provide technical support to the country’s 189 health districts. The health districts, in turn, are mainly responsible for delivering primary health care services to the Cameroonian population. Preventive services such as immunization services are incorporated into all levels of the health system [25].

Immunization performance monitoring is achieved through a hierarchical administrative data monitoring system. Focal EPI staff at health facilities aggregate data on monthly tally sheets into paper-based monthly activity report, which are then transmitted to the district data management team for entry into the District Health Information System (DHIS2). Immunization data in the DHIS2 are then aggregated at a regional level before transmission to national level for national estimates of coverages for various indicators [26].

Sampling

A minimum sample size of 145 health facilities was estimated using the following formula and parameters.

$$N = {\rm{ }}\left[ {DEFF{\rm{ }}x{\rm{ }}Z2{\rm{ }}x{\rm{ }}p\left( {1 - p} \right)} \right]{\rm{ }}/{\rm{ }}E2$$

Design effect (DEFF) of 1.5, Critical value (Z) of 1.96 for a confidence level of 95%, estimated proportion of outcome (p) = 50%, and margin of error (E) of 10%.

Multistage sampling was used after allocating the number of districts and health facilities to be selected. The number of districts was allocated in proportion to the total number of districts per region, in the ten regions. Then, the number of urban and rural districts were assigned within each region based on the region-specific breakdown and health facilities allocated across regions in proportion to the national distribution.

The districts were then randomly selected within specified region urban or rural strata in the first stage and health facilities were randomly selected within the identified rural/urban districts in the second stage. This was done while ensuring that the same number of facilities was selected within each district in a region.

Study procedures

Trainings

Trainings were carried out in all regions by members of the baseline assessment management team to provide regional supervisors and assessors with the necessary knowledge and skills to undertake the baseline assessment. Trainings lasted a minimum of two days and consisted of theoretical presentations and practical sessions on data collection, entry, and transmission processes.

Data collection

Data collection plans for targeted districts and health facilities were developed by the baseline assessment management team and the regional EPI teams prior to data collection. To prevent unproductive visits during data collection, assessors contacted the health facilities via phone to remind them of planned visits. Upon arrival in facilities, the purpose of the assessment was explained to the facility head, or their representative and the health facility focal EPI staff interviewed. Additionally, an ongoing fixed-post vaccination session was observed, and vaccination data collection tools were physically inspected.

Study tools

The tools used for this study included health facility questionnaire, (which was designed to assess immunization data collected and availability of data collection tools) and an immunization service observation guide and checklist, (which comprised of several prompts to assess data management practices of immunization staff during vaccination sessions, and the data recorded in core data collection tools). These study tools were developed in English and French, and pre-tested in four facilities in the Centre region before being used to collect data for the study.

Data management and analysis

A comprehensive database was built, pre-tested and validated by an expert data manager prior to data entry. Each assessor entered data from the filled questionnaires and observation forms into the database and transmitted the files to a secure server within three days of data collection. Data were exported and cleaned in Microsoft Excel 2016 and analyzed with Stata 13 software (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP.). Frequencies and proportions were used to summarize variables, with health facility used as the unit of analysis. Since districts and facilities were sampled proportional to national distributions, no post-stratification weights were applied.

Operational definitions

  • Data management practices: This included data collected, availability and utilization of data collection tools, data consistency across collection tools and data recording practices during vaccination sessions.

  • Vaccination staff (Used interchangeably with immunization staff, vaccine-providers, vaccinators): This included all health staff working in the immunization unit.

  • Completely filled data collection tool: All required columns of the immunization tool were filled, for one month until the time of assessmentTo examine this data on completeness for five routine immunization tools was assessed: Immunization register, tally sheets, monthly reporting form, temperature chart, stock ledger.

Administrative approval

Administrative approval was obtained from the ministry of public health and all ten regional delegations of public health prior to commencement of the baseline assessment.

Ethical considerations

This study was conducted in compliance with the Helsinki Declaration and all applicable national laws and institutional rules ethical approval was granted by the Cameroon National Ethics Committee for Human Health Research prior to study commencement.

Results

Table 1 presents the general characteristics of surveyed facilities. Overall, 265 health facilities in 68 health districts were surveyed, with slightly over half (53%) of these facilities located in rural areas. At least two staff worked in the immunization service in most (74%) of these facilities. Over two-thirds (67%) had staff who have never received training on immunization (Table 1). We also noted great regional variations in trainings, with 93% and 100% surveyed in the South region and East regions lacking a single staff who has received basic training on immunization, respectively.

Table 1 General characteristics of surveyed health facilities
Full size table

Data collected by health facilities

Table 2 summarizes the practices of healthcare workers with regards to documenting data on vaccine administration. As illustrated 92% of facilities systematically collect vaccine administration data (filling of tally sheets, immunization registers and vaccination cards) at national level. At regional level, marked variations were observed. As shown in Table 2, about a quarter of (24%) of surveyed facilities in the East and South-West Regions did not document vaccine administration data. Only 65% and 61% of health facilities collected stock levels and vaccine wastage respectively. As regards temperature monitoring, most (79%) facilities recorded vaccine storage temperatures at the time of assessment.

Table 2 Availability and utilization of data collection tools (%)
Full size table

Availability and utilization of data collection tools

At national level, majority of health facilities had core vaccination data collection tools, including tally sheets (84%), registers (87%), reporting forms (85%). Conversely, only 43% of facilities had vaccination cards, 8% had maintenance registers and 32%  had stock cards. Regarding the use of these tools, immunization registers were filled in only 63% of health facilities and tally sheets just in 19% of facilities. Similarly, only 13% of facilities had updated and fully filled monthly report forms. These findings are shown in Table 2.

Data consistency across recording tools

Discrepancies for the same data was noted across three core data collection tools assessed (tally sheets, immunization registers and monthly reporting forms). The Pentavalent 3 value from the tally sheets and monthly reporting forms was different in over half (53%) of the health facilities. This was observed across all regions and the highest percentage of health facilities with inconsistent data across sources was noted in the South (85%) and Littoral (81%) regions, as shown in Table 2.

Data recording practices during immunization sessions

Immunization registers and tally sheets were present at the vaccination site in up to 90% of facilities. Similarly, in most (85%) health facilities, immunization acts were documented in registers, tally sheets and vaccination cards as recommended, with regional variations (ranging from 73% in the East region to 100% in the North-West and South-West regions). Conversely, only 11% of health facilities adhered to the recommendation of filling the tally sheet as children were vaccinated. Up to 51% of these facilities rather filled the sheets before vaccination, and 25% did so after vaccination as illustrated in Table 3.

Table 3 Data recording practices during vaccination sessions (%)
Full size table

Discussion

The main objective of this study was to assess the data management practices of immunization staff at health facility level, in a bid to unveil potential barriers to vaccine data quality in Cameroon. To the best of our knowledge, this is the first published study examining the data management practices of vaccination staff on a national scale in Cameroon. We noted several unsatisfactory data collection and management practices in a good proportion of facilities including ineffective use of data collection tools, filling tally sheets before or after vaccination, sub-optimal collection of data on some key immunization indicators. This study also found that the unavailability of core data recording tools was not uncommon and could contribute to substandard data collection and eventually immunization data quality.

We noted various gaps when it comes to recording immunization data at facility level. In our study we found that about one in every ten (8%) of facilities did not collect data on vaccine administration; about one in three (35%) did not collect data on stock levels and about two in five (21%) did not record vaccine storage temperatures and vaccine wastage. Our findings lend support to those documented in multiple settings [7, 9, 10, 27]. As documented elsewhere, several technical, organizational and behavioral factors may explain the root causes of the observed deficits in this study, including gaps in health provider capability and motivation, poor supportive supervision, unavailability or complexity of data recording tools, lack of performance-based targets, and interrupted regular feedback [12, 28]. Deficits in capacity and capability could explain the gaps in indicators like vaccine wastage, which may be relatively more complex to compute. The observed data collection gaps could also be linked to the limited availability of some data recording tools like vaccination cards, maintenance registers and stock cards, which were lacking in 57%, 92% and 43% of surveyed facilities, respectively (Table 2).

The availability of immunization registers (87%) and tally sheets (83%) in our study is higher than what was reported by Kefiyalew et al. (available in 43–50% of facilities) in Ethiopia [12] and Ronveaux et al. [11] in 27 GAVI-supported countries, including Cameroon (available in 60% of facilities). These differences could be because Kefiyalew’s study was carried out only in one health district while Ronveaux’s study assessed availability of tools over a one-year window, unlike ours which assessed only during the study period. Difference could also reflect improvements by Cameroon’s EPI, to ensure that these critical data collections are available at facility level. Such improvements were, to a large extent, driven by governments efforts to introduce new vaccines such as IPV, combined measles vaccines, HPV etc., which are always accompanied by updating and distribution of various data collections tools. Other organizations, such as the Clinton Health Access initiative and Unicef have also supported the government to provide facilities with registers.

We also found that even when data recording tools were available and used, they were ineffectively used and incompletely filled in a good proportion of facilities, further aggravating data quality problems at facility level. This finding is comparable to that of Wallace et al. (2017) in Nigeria, who reported that only 51–55% assessed facilities stock records with sufficient data to calculate vaccine wastage rates [29]. Additionally, our study noted inconsistencies for the same data recorded across different core data recording tools. For instance, the Pentavalent 3 value from the tally sheets and monthly reporting forms was discrepant in over half (53%) of the health facilities. This finding is slightly lower compared that reported by Akerele et al. (2020) in neighboring Nigeria, who showed that Pentavalent 3 values between the health facility monthly summary form and the national health management information systems monthly summary form were discrepant in 67% assessed facilities in Kano State [27]. These gaps can be corrected by health worker training on immunization data tools and regular supportive supervisory visits, interventions which have been shown by Akerele et al. (year) as important determinants of good data consistency.

In our study we also identified some data management practices during vaccination sessions which could contribute to incomplete, or inaccurate or inconsistent data. Just over a tenth (11%) of health facilities adhered to the recommendation of filling the tally sheet as children are being vaccinated, instead of filling them before or after vaccination. This could lead to either under-reporting or over-reporting of vaccinated children, resulting into poor data quality which could impact in-country and external investments, and frustrate efforts in reaching zero-dose children and under-immunized children. This further reemphasizes the need for staff capacity building, supportive supervision, and standardization of data recording processes.

While interpreting the findings of our study, some limitations have to be considered. Our study may be biased by the Hawthorne effect because EPI personnel were aware of being observed and may have modified their behaviors during the study period. Secondly, data management practices were assessed at health facility level, thus individual level variations at health facility level could be missed.

Conclusion

This study brings to light several unsatisfactory data management practices of vaccination staff in Cameroon, including ineffective use of data collection tools, filling tally sheets before or after vaccination, sub-optimal collection of data on some key immunization indicators. These, coupled with unavailability of key data recording tools, could hinder the generation of timely, high-quality data, and ultimately affect decision making and delivery of immunization services. There is thus a crucial need to standardize data recording processes, systematically build data management competencies of health facility EPI staff in Cameroon, while building policies and systems that ensure constant availability of data recording tools.