Background

Successful malaria elimination requires rapid and accurate tracking of cases so that personnel can be promptly deployed to treat cases before onward transmission occurs. With over three million people of its 4.5 million population at risk for malaria transmission, Aceh Province, the westernmost of Indonesia’s provinces, faces a daunting challenge if it is to achieve malaria elimination by its target date of 2015 [1]. From 2005 to 2009, after the surveillance system was revitalized following the Indian Ocean tsunami of 26 Dec 2004, the passive surveillance system detected 6,481 presumptive and 2,355 confirmed malaria cases [2]. The mean prevalence of malaria as measured by 434 small scale malaria surveys conducted from 1985 to 2010 was 2.3 % for Plasmodium falciparum (range 0–43 %) and 1.4 % for Plasmodium vivax (range 0–31 %) [3, 4]. In 2009 and 2010, the Indonesia Ministry of Health and Aceh Provincial Government launched a call to eliminate malaria. Both provincial and national plans proposed that indigenous transmission of malaria be eliminated from the entire province of Aceh by the end of 2015 [5, 6]. However, a formal assessment of the capacity within the province to eliminate malaria has never been done. High quality malaria diagnostics is acknowledged as a crucial element of successful malaria elimination [7, 8], with microscopy as the gold standard playing a key role [9, 10]. Experience from malaria-free countries has demonstrated the importance of well-established, wide-spread, high quality malaria diagnosis, high capacity for active case follow-up and a robust monitoring and evaluation system [11–13]. Furthermore, extensive experience from countries striving to eliminate malaria in Africa [14–19], Asia-Pacific [1, 20–24] and Peru in South America [25, 26] have shown the necessity of access to reliable and effective diagnosis with microscopy which rests upon the basis of highly trained, well-supervised laboratory technicians who have functioning microscopes, access to re-supply of reagents, respect for positive and negative diagnosis and constant access to anti-malarials if malaria transmission is to be interrupted.

A baseline assessment aims to appraise the strength and weakness, as well as to identify opportunities and challenges of malaria microscopy diagnostics in Aceh Province as it moves toward malaria elimination. Distribution and proficiency of malaria microscopists, laboratories evaluations and observations on the logistics chain and current implementation of quality assurance for microscopy diagnostics were plotted. In overall, the work provides a comprehensive real life example of the variety of challenges faced when a large tropical province moves toward malaria elimination.

Methods

Study location

Aceh (population 4.5 million) is one of 34 provinces in the Indonesian archipelago which occupies 58,375.63 km2 on the western tip of Sumatra [27]. Aceh has 23 districts, 276 sub-districts and 6,592 villages [28] with 22 hospitals and 307 primary health centres providing health services to the population [2]. Figure 1 shows map of Aceh and the distribution of microscopists. Participants in the surveys were malaria microscopists working at primary health centres (PHCs), hospitals, government laboratories and health offices in all districts and in the provincial government.

Fig. 1
figure 1

Distribution of malaria microscopists in Aceh province, 2010 - 2011

Full size image

Assembling the database of malaria microscopists and their laboratories

Two standardized questionnaires were distributed to all PHCs, hospitals and clinics by mail and fax between October and December 2010. The first questionnaire collected personal information on microscopists, such as name, age, gender, educational level, working institution, workload and self-assessment of their diagnostic competency. The second questionnaire aimed to collate perceptions of microscopists about their working conditions, including workspace, equipment, supplies, laboratory, standard operating procedures, human resources, training, safety, collection and data reporting. Questionnaires were returned to provincial health office by mail or fax.

Evaluating the proficiency of malaria microscopists

Following data assembly, twelve assessors from the Aceh provincial health office (PHO) and provincial health laboratory (PHL) were divided into four groups and assigned to visit 23 districts to carry out on-site proficiency tests. Between May and June 2011, all malaria microscopists were invited for evaluation, with over 400 attending testing at district health offices using standardized malaria slides [29]. Each set consisted of 20 slides containing ten negatives and ten positives consisting of three P. falciparum, three P. vivax, one Plasmodium malariae, one Plasmodium ovale and two mixed infection slides. Another set comprised five slides of P. falciparum with low to high parasitaemia density for parasite counting. All participants were requested to count for a duration not to exceed ten minutes per slide [30, 31]. They were instructed to detect the presence of malaria parasites from the top left-hand part of the thick smears and move approximately five fields to the right [National Malaria Control Program, personal communications]. They also were instructed to count parasites against 200 leucocytes for parasite density estimates [32]. They were expected to complete the test within four hours and ten minutes. Six diagnostic proficiency indicators, including sensitivity, specificity, overall and species accuracy, error rate and counting agreement were then calculated. Each participant was subsequently graded as in-training, advanced, reference and expert according to the World Health Organization (WHO) guidelines [33].

Evaluating the practice of malaria diagnostic and logistic

Four districts (Aceh Besar, Aceh Jaya, Aceh Timur and Sabang) were selected for direct observation (confirmatory visits) of malaria diagnostic practice and laboratory conditions. The 64 PHCs in these districts were categorized by slide positivity rate as low (SPR < 5 %), moderate (5 % ≤ SPR < 40 %) and high (SPR ≥ 40 %). For each category, 30 % of PHCs were randomly selected for a confirmatory visit. Six evaluators from the PHO, PHL and an independent malaria quality assurance professional visited the selected PHCs between June and July 2011.

An additional four questionnaires sought information from heads of PHCs, microscopists and logistic officers. The head of PHC questionnaire consisted of 17 questions on administration, case finding, procedure of malaria slide examination and the reporting system. The microscopist questionnaire focused on administration, slide production, diagnosis and laboratory infrastructure, while a subsidiary reporting form was used to record the daily practice of microscopy diagnosis from finger prick procedure to reporting. A fourth questionnaire aimed at logistics officers asked about supply availability, planning, purchasing, storage and distribution of supplies related to malaria diagnosis.

Statistical analysis

Questionnaire data were entered into an Excel database and then analysed by STATA version 9. Descriptive statistics were calculated for personal information of microscopists and malaria laboratory elements. For each element, an adequacy score was obtained by dividing the number of respondents answering “adequate” by the total number of respondents providing a response. Elements for which ≥ 70 % of respondents responded as adequate were considered as high adequacy and elements for which the score was < 70 % were considered as low adequacy. Six diagnostic proficiency indicators, including sensitivity, specificity, diagnosis accuracy, species accuracy, error rate and parasitaemia counting, were calculated. Sensitivity and specificity were determined as the proportion of parasites detected in standardized positive smears and the proportion with no malaria parasites seen in the standardized negative smears, respectively. Diagnostic accuracy was calculated by dividing the number of slides with correct diagnosis by the total slides examined. Species accuracy was calculated as the proportion of malaria microscopists that accurately determined the species of malaria parasites. Error rate was calculated as the proportion of slides incorrectly diagnosed. Lastly, parasitaemia counting was the proportion of malaria microsopists reporting the number of parasites relative to 200 leucocytes. The density of parasites per microlitre was calculated using the formula: (number of parasites x 8,000 leucocytes)/200 leucocytes. An answer considered correct was obtained when the percentage of quantification was within 25 % of the actual density per microlitre. Kruskal-Wallis rank-test was used to detect the statistical differences between diagnostic proficiency indicators and attendance at examination sessions. Chi-square test was used to assess the proportional differences with a P-value of less than 0.05 indicating a significant difference.

Results

Characteristics of malaria microscopists in Aceh

In 23 districts of Aceh, 574 microscopists were registered. All were sent questionnaires with 463 (81 %) responding. Table 1 shows the characteristics of microscopists. More female microscopists were found compared to male (85 % vs 15 %). Mean age of microscopists was 32 years (range 20 to 50 years). Malaria microscopists had a diploma degree as a health analyst (40 %), a high school certificate (23 %), nursing or midwifery degrees (8 %), or bachelor of public health (3 %). Most worked at PHCs (69 %) or hospitals (25 %).

Table 1 Characteristics of malaria microscopists in Aceh, Indonesia, 2010
Full size table

Almost half of microscopists had been carrying out their duties for at least one year and less than 10 years, with about an additional 15 % working less than one year and 11 % for more than 10 years. Most (67 %) examine less than 40 malaria slides per day. Two thirds received at least one microscopy training during their career, with one third receiving no specific training. In terms of duration of service, 11 % of those in service less than one year had no training, 20 % of those in service 1–10 years had no training, whilst 5 % of microscopists working for more than 10 years had never received formal training. Of the 392 microscopists that self-ranked their malaria diagnostic expertise, 57.9 % rated their expertise as basic, followed by 39.5 % as intermediate skill and 2.6 % as experts.

Features of malaria laboratories in Aceh

Table 2 shows the attributes of malaria laboratories as reported by malaria microscopists in 23 districts in Aceh. Three malaria laboratories – the Provincial Health Laboratory in Banda Aceh, PHC Bandar Dua, and PHC Bandar Baru in Pidie Jaya district reported adequacy at all elements. Two districts, Gayo Lues and Simeuleu, provided no response regarding capacity of their malaria laboratories. Laboratories in six districts reported at least 20 adequate elements, including Pidie Jaya, Sabang, Banda Aceh, Pidie, Aceh Tamiang and Lhokseumawe. The majority of malaria microscopists reported high adequacies for certain components, including a clean and tidy room, appropriate storage of equipment, adequate stocks of supplies, efficient workflow, clear laboratory procedures and effective slide archiving. Microscopists in Aceh Barat, Aceh Timur and Nagan Raya districts reported low adequacy for training documentation, no regular quality checks on supplies, lack of maintenance records for equipment, no annual review on laboratory SOPs, lack of a cross-checking system and failure to limit access of unauthorized personnel to the laboratory.

Table 2 Features of malaria laboratories in Aceh, Indonesia, 2010
Full size table

Results of self-assessments and confirmatory visits were broadly similar as shown in Fig. 2, which shows the proportion of laboratories self-reporting as adequate versus results of external assessment for 17 PHCs and eight elements. No significant differences were detected in adequacy rates for any of the eight indicators, which increase a confidence that self-reporting for these elements accurately reflects reality. Results of self-assessment and field visits both reported inadequate laboratory conditions. Table 3 shows results of field visits to 17 PHCs in four districts. Out of 30 elements examined, low adequacy was found in 22 elements in Aceh Besar, 20 elements in Aceh Timur, 16 in Aceh Jaya, with Sabang the only district demonstrating low adequacy in less than half (13) of the elements examined. Assessors commonly observed unrestricted access to laboratories, lack of manuals for equipment, poor supply quality control checks, uncommonly applied annual review of working procedures, and infrequent refresher training. Most PHCs in Aceh Besar, Aceh Jaya and Aceh Timur were unlikely to have equipment maintenance records or calibration, storage and safety procedures, such as proper sharp containers, laboratory coats and handwashing soaps. Both Aceh Besar and Aceh Timur districts lacked adequate working space and clean water as shown in Fig. 2. With the exception of two PHCs in Sabang, no cross-checking systems had been implemented.

Fig. 2
figure 2

Comparison of proportion of malaria laboratory elements rated by self-assessment and direct observation in Aceh, Indonesia, 2010 – 2011

Full size image
Table 3 Characteristics of malaria diagnostic laboratories by self-assessment and direct confirmation visits at 17 PHCs in four districts in Aceh, 2011
Full size table

Proficiency of malaria microscopists

Four hundred thirty two microscopists of 574 accepted invitations to participate in proficiency testing. Nearly all microscopists (413 or 95.6 %) scored at basic or in-training level, while 10 (2.3 %) were advanced and 9 (2.1 %) were reference microscopists. Among the ten advanced microscopists, six worked at PHCs, three at district level (two at health offices, one at health laboratory) and one at a hospital. The advanced microscopists were deployed at Bireun (3), Bener Meriah (2), Aceh Tengah (1), Aceh Utara (1), Pidie Jaya (1), Sabang (1) and Simeuleu (1). Among nine reference microscopists, five worked at PHCs and four at district health offices. These reference miscroscopists were located in Aceh Tengah (3), Aceh Besar (1), Aceh Tamiang (1), Aceh Utara (1), Bireun (1), Pidie (1) and Sabang (1) (Fig. 1).

Microscopists who reported having had microscopy training performed significantly better than those who reported no training for diagnosis accuracy (70.2 vs 60.8 %, p < 0.001), sensitivity (69.1 vs 57 %, p < 0.001), species accuracy (37.2 % vs 27.7 %, p < 0.001) and parasitaemia counting (25 % vs 19.8 %, p = 0.04), but they did not perform significantly better for specificity (70.6 vs 65.3 %, p = 0.115).

The practice of malaria diagnostic at primary health centres

Sixteen heads of PHCs (eight of whom were clinicians) were interviewed during confirmatory visits; all confirmed that their PHC provided free microscopy diagnostic services. Ten of 16 heads of PHCs referred patients for laboratory examination on the basis of recent fever, history of malaria infection, or travel to malaria endemic areas. Only one respondent said that laboratory malaria diagnosis was unnecessary in areas where malaria was found rarely (1/16).

Most (14/16) heads of PHCs were satisfied with their microscopist’s performance, believing them to be competent (12/16), properly trained (11/16), properly archiving slides (12/16), supported by a cross check system (13/16), and having adequate equipment, laboratory supplies and storage space (12/16). Half (8/16) of clinicians reported receiving malaria results within less than an hour. Slightly over half (9/16) were aware of the absence of written standard operational procedures for laboratories (9/16). Clinicians in Aceh Timur complained that it takes 12–24 h to get confirmed results. In consequence, they treated patients presumptively.

Figure 3 and Table 4 show the results of field visits to assess microscopy diagnostic practices of 25 microscopists at PHCs. Microscopists in Aceh Besar and Aceh Timur were less capable than their counterparts in Aceh Jaya and Sabang. Most microscopists used non-frosted-end object slides making it difficult to record patient identifiers directly on the slides. In many cases, laboratory space and equipment were not properly cleaned, possibly increasing the number of artefacts from staining and increasing the error rate of microscopists. The quality of thick and thin smears produced was generally low, and most microscopists examined fewer than the recommended 200 fields before determining that a slide is negative. No microscopist calculated parasite density, with qualitative measures (+, ++, +++) used instead, calling into question their ability to detect low-density parasitaemia.

Fig. 3
figure 3

Less than adequate laboratories in Aceh, 2011

Full size image
Table 4 Major findings in the procedures of microscopy diagnosis obtained from supervisory visits to 17 PHCs in four districts of Aceh, 2011
Full size table

Malaria diagnostics logistic chain

Figure 4 outlines the administrative and bureaucratic processes involved in purchasing logistics for malaria diagnostics in Aceh. Microscopists work through the heads of their PHCs, who submit plans for budget review at the district. Consolidated plans are then presented to the District Parliament and District Planning Board (Bappeda, Badan Perencanaan Pembangunan Daerah) for final approval. Approval does not end the process, however, as the district purchasing committee must then open the bidding process, contact vendors, and manage procurement. When materials arrive, logistics officers inventory and store the supplies in the district warehouse. Logistics are normally delivered to the PHCs but in some cases PHC staff visited the warehouse to pick up supplies.

Fig. 4
figure 4

Scheme of logistics and purchasing system in microscopy diagnostic of malaria in Aceh province, 2010 - 2011

Full size image

Twenty two people responsible for malaria diagnostics logistics at 17 PHCs and four DHOs were interviewed to assess the effectiveness of this supply chain. Fourteen of those interviewed also currently worked as malaria microscopists. Most respondents said that they neither created a purchasing plan (15) nor read guidelines on the purchasing system (16). Two respondents reported that their PHCs planned their annual budget once a year in October.

Most respondents reported that their logistics for malaria diagnosis arrived on time (14). Most respondents reported that they had sufficient storage space for supplies (15) and said that regular inventories to prevent stock outs were done (14). More than half of the respondents reported that it might be useful to have a storage cabinet at temperature ≤30 °C for RDTs, but that it was unlikely the available space was sufficient (12). Some respondents reported that the quantities of supplies received were less than the amount requested (3).

Various frequencies of reporting on reagents and consumables consumption were reported by respondents. The majority reported monthly (11), while others reported quarterly (2), twice a year (1), once a year (1) or never (2).

Discussion

This study represents one of the few efforts to comprehensively document the readiness of a large administrative unit – in this case, the province of Aceh, Indonesia, with a population of 4.5 million people – to carry out the high quality diagnosis needed for malaria elimination. Thanks to large scale malaria control efforts including LLIN distribution, IRS, and roll-out of ACT in Aceh post-tsunami, the incidence of malaria has declined to the point where mapping and follow-up of individual cases known as “active” case surveillance for malaria elimination has become feasible [34]. However this is premised on a sufficiently robust and effective diagnostic network. Self-assessment was used to ensure wide coverage, coupled with direct observation for selected health facilities and objective assessment of the capacity of most of the microscopists in the province. Some selection bias is possible as 75 % of those sent questionnaires responded. A database for 574 microscopists providing diagnostic services in 293 PHCs, 33 hospitals, 12 district health offices, four district health laboratories, one port health office, one provincial health office and one provincial health laboratory was assembled. The objective-proficiency tests revealed that nearly two thirds of microscopists in the province graded as basic level according to the WHO standard [33]. It seems unlikely that those not tested would be of higher competency that those tested, so think it likely that the overall competence of Aceh microscopists is at a basic level.

The results of the assessment showed that microscopists in Aceh tended to overestimate their own competency. One third of surveyed microscopists (36 %) rated their competency as “intermediate” or “expert”, even though they had never attended microscopy training. Seventy-four percent of 155 self-described ‘intermediate’ microscopists tested at basic level, whereas three self-assessed ‘expert’ microscopists tested as basic, advanced, and reference, respectively. Objective assessment of microscopist’s competency using standardized slides is essential in this context, given the severe bias of self-assessments to overestimate competency and simple one-off training will not ensure diagnostic quality [35]. In contrast, the assessment of microscopists of their working environment and laboratory infrastructure closely mirrored the objective assessment of the independent team’s direct observations.

The assessment demonstrated that the microscopists in Aceh are not sufficiently skilled to detect the low-density parasitaemia associated with malaria elimination efforts; such ability is key in low transmission areas such as Aceh [36–38]. Only expert microscopists with adequate experience are likely able to detect and quantify low parasite densities [10]. However, simple one-off training will not ensure diagnostic quality [35]. For instance, in East Aceh district 25 people received standard microscopy training in 2010 organized by UNICEF and the Aceh Provincial Health Office [39]. When retested 10 months post-training, there had been significant erosion of diagnostic accuracy skills (62.4 vs 82.7 %, p < 0.001), sensitivity (73.6 vs 92 %, p < 0.001) and specificity (51.2 vs 66.6 %, p = 0.032). In the context of malaria elimination, false negative finds are particularly inimical. In addition, visits to facilities showed that microscopists were not supported by adequate infrastructure (Table 3). A study in Thailand emphasized that adequate microscope and quality control procedures were crucial [20], as well as supervisors’ training to conduct visits and evaluate the laboratories [31].

The effectiveness of a rigorous training and supervisory policy has been shown on the Thai-Myanmar border, where microscopists with objective accuracy of more than 80 % are assigned to work in PHCs [20]; those less competent are required to attend competency-based training [31]. Such intense effort is in part due to substantial funding available to reduce anti-malarial drug resistance in the Mekong region. Microscopists with marginal scores of 70–80 % accuracy undergo additional training and close supervision for six months. No such policy for competency improvement yet exists in Aceh, nor is there global interest, so that local initiatives will need to prevail.

The WHO recommends that at least one well-trained and competent malaria microscopist works in each PHC [9]. To achieve this, Aceh province will need to develop a routine accreditation programme, minimize turnover of malaria microscopists, and provide strong support for its small number of reference microscopists to become experts [40]. A small group of expert microscopists can serve as both cross-checkers and trainers. Training should be individualized, emphasizing the weakest skills for each trainee. For example, microscopists from the district of Aceh Besar require training in slide preparation, slide reading and quality assurance, while microsopists from Aceh Jaya District need more training on slide preparation. In Aceh Timur, the weakest elements were on staining and slide examination. Four districts (Aceh Besar, Aceh Jaya, Aceh Timur and Sabang) need to improve their capability to produce good malaria smears and accurate parasitaemia counts. Adequacy of laboratory facilities is an important influence on the competence of microscopists. Two studies in Africa rated laboratories by several fundamental elements (in addition to having well-trained staff), including reliable electricity, access to clean water, adequate working space, comfortable room temperature, proper venipuncture procedure and routine physician oversight [41, 42]. Using these criteria, most laboratories in Aceh (63 %) were classified as minimal, generally because of unreliable access to electricity and clean water, limited equipment and minimally trained personnel.

Logistics and procurement policy have a large impact on laboratory quality. In Aceh, the system of procurement is initiated by the microscopists, but in most cases guidelines for minimal standards are either not available or not applied. Essential elements are a procurement policy, and guidelines on equipment and reagents specifying minimum standards [31].

Sustained and high quality logistics are essential for high quality malaria microscopy [31, 43]. The responsible logistic or malaria programme officers at all levels of the health system should ensure the quantity and quality of logistics prior to transporting to PHCs [43] and ensure continuous supply [40] of both required diagnostics and treatment requirements. Logistic officers, who are mostly also the malaria microscopists, need training to manage their inventory system, i.e., to create inventory lists, monitor stocks and submit routine reports. This would ensure that they are aware of their logistics requirements and stocks in the medium term (six months) and would allow sufficient advance time in reporting to avoid lapses in supply from the DHO [31]. Such reports would enable logistic officers to predict and avoid stock-outs of microscopy diagnostic supplies [40]. Involving malaria microscopists in the purchasing plan is useful because of their hands-on experience with different supplies.

Implementation of a good External Quality Assessment (EQA) would likely increase the competency of microscopists in Aceh. At present, the cross-checking system adopts a traditional policy to send all positive slides and 5 to 10 % of negative slides to a microscopist at a higher and presumably more competent level. However, the system is barely functional, with few microscopists sending slides for cross-checking and even fewer receiving prompt and useful feedback. Part of the reason for this is the heavy administrative burden and amount of labor associated with the numerous slides submitted. In 2009, the WHO [31] recommended two alternative approaches for random selection of slides for cross-checking at PHCs: Lots Quality Assurance System (LQAS) and the Médecins Sans Frontières (MSF) system. One study in Pakistan showed that a useful LQAS-based cross-check system could be based upon sending only eight per month based on slide positivity rate (SPR) and the number of negative slides in the previous year [24]. Several countries, including Thailand, Philippines, Pakistan and Ghana [24, 30, 31, 44] have implemented this scheme for malaria and tuberculosis. In the MSF method, five negative and five weak positive slides are sent to be re-checked, based upon the assumptions that false positive slides were most likely occurring during routine examination because microscopists preferred to ‘play safe’ and report positive result for true negative slide or because of misidentification of artefacts as parasites [45, 46]. The Aceh Provincial Health Office has decided to begin implementation of LQAS for cross-checking soon.

Establishing an effective quality assurance (QA) system for microscopy diagnosis requires a comprehensive approach. Achieving and maintaining high quality performance of microscopists in various malaria endemicity settings involves multiple interventions, including dissemination of written national guidelines, modern microscopy training using computer-based modules, and supervision and feedback [31, 35, 46]. Intensive performance monitoring is likely more constructive than allowing microscopists to repeatedly attend identical standard training modules [47]. To achieve malaria elimination, Aceh province needs to immediately start implementing a comprehensive large-scale QA system and advocate to decision-makers to recruit proficient malaria microscopists.