Background

Malaria is a disease of global importance that results in 300 - 660 million cases annually and an estimated 2.2 billion people are at risk of infection [1]. Of the 2.5 million reported cases in the South East Asia, India alone contributes about 70% of the total malaria cases [2]. Currently, 80.5% of the 109 billion population of India lives in malaria risk areas [3]. Malaria presents a diagnostic challenge in most resource poor areas where malaria is endemic. In such areas malaria diagnosis is often made only on the basis of clinical symptoms although this is alarmingly inaccurate [4]. The role of the laboratory diagnosis of malaria is primarily to support clinical care [5]. The traditional method for the detection of the malaria parasite is the examination of thick and thin blood smears by microscopy. The shortcomings of microscopy for malaria diagnosis are well known [6, 7]. These diagnostic limitations affect medical care provided, as malaria is a potentially fatal disease, usually curable if diagnosed quickly [8]. The World Health Organization has recognized the urgent need for simple and cost effective diagnostic tests for malaria to overcome the deficiencies of both light microscopy and clinical diagnosis [9]. The need for a simple, sensitive diagnostic test has led to the development of rapid diagnostic tests (RDTs) among other alternative techniques. Initially the use of RDTs met stiff resistance by the malaria community because of its cost. However, a number of reports from policy makers have acknowledged that RDTs may have their place because expert microscopy in malaria-endemic countries is hard to establish and cost of RDTs has been greatly reduced [10]. Further, the recently introduced artemisinin based combination therapy (ACT) is given to patients only when the diagnosis has been confirmed parasitologically. However, providing parasitological results at all levels of health care presents a serious challenge. Expanding the use of blood slide microscopy is a possible solution but the cost and logistic challenges in remote area restricts the use of microscopy to hospitals and well-equipped laboratories. It takes great skill and years of experience to learn to accurately read a malaria slide. The use of RDTs for Plasmodium falciparum malaria is being implemented by National Vector Borne Disease Control Programme (NVBDCP) to improve diagnostic efficiency in peripheral health care settings in central India. Further, although P. falciparum causes the most severe disease, recent reports of significant morbidity and drug resistance in Plasmodium vivax infections are generating new interest in P. vivax [11, 12]. The first generation RDT tests were specific for P. falciparum [13–15], but the development of new rapid tests by including a Pan-malaria test line allowed detection of non-falciparum infections also [16, 17]. Subsequently increasing numbers of similar products have been developed [7, 18, 19].

We undertook a study on evaluation to assess the performance of commercially available malaria RDTs in comparison with microscopy and polymerase chain reaction (PCR) in an area where both P. falciparum and P. vivax are co-endemic. This would allow direct product comparisons that would assist the policy makers and programme managers in taking procurement decisions and would ultimately encourage improvement in the quality of manufacturing. Five RDTs evaluated for this purpose were selected on the basis of two main criteria i.e. tests detecting both P. falciparum and non-falciparum infections and commercial availability.

Methods

Study area

The study was conducted in 10 villages of Bajag Primary Health Centre of district Dindori (22°57' N latitude and 81°41' E longitude) between August to December, 2009. Dindori is a highly malarious district in Madhya Pradesh (Figure 1) contributing 12% of malaria in the state while its population is only 1% of state [20]. The villages of Bajag PHC are very remote, forested and inaccessible for 4-6 months during rainy season. The average annual rainfall is 1,400 mm. The inhabitants of these villages are ethnic group of Baiga primitive tribe. They are very poorly clothed and have immense faith in sorcery and witchcraft. There is no public transport system and health facilities are non-existent. Plasmodium falciparum is the predominant infection. The RDT evaluation was also carried out in 10 villages of Satanwada Primary Health Centre, District Shivpuri (25°4' N latitude and 77°44' E longitude). The inhabitants of the villages are ethnic group of Saharia primitive tribe who live in small one room hutments which are very overcrowded and unhygienic. Plasmodium vivax is the predominant infection in this area unlike Bajag PHC. The study area is hot, dry and the average annual rainfall is 875 mm.

Figure 1
figure 1

(A) Map of India showing Madhya Pradesh, (B) Shivpuri and Dindori district.

Full size image

Study design

All patients irrespective of their age and sex except pregnant women presenting at the field clinic with a clinical suspicion of malaria were included in the study after taking written informed consent. This study was approved by the ethics committee of the Regional Medical Research Centre (ICMR).

Demographic and clinical information was recorded from each patient and all five RDTs were tested simultaneously in field in parallel with peripheral blood smear for microscopic examination in the laboratory. Before the initiation of the study, a one-day workshop was organized to provide training in blood collection from finger prick, test procedure for each RDT and interpretation of the test results as per manufacturer's instructions. The results of each RDT were recorded between 15-30 minutes as per the manufacturer's instructions. 2-3 drops of finger prick blood samples were also collected in heparinised tube/filter paper for PCR to be conducted in the laboratory.

The five RDTs used in this study are - Parascreen Device (rapid test for malaria Pan/Pf), Falcivax Device (rapid test for malaria Pv/Pf), Malascan Device (rapid test for malaria Pf/Pan) (all from Zephyer Biomedicals Goa), ParaHIT Total (rapid test for Pf & Pan Malaria species) (SPAN Diagnostics Ltd, Surat) and First Response Malaria Antigen Combo Card test (pLDH/HRP2) (Premier medical corporation Mumbai). These RDTs were provided by their manufacturers for the evaluation. The detailed characteristics of each test are shown in Table 1.

Table 1 Characteristics of evaluated rapid malaria tests
Full size table

All RDTs were tested by two Research Assistants to minimize variability. The blood films were examined by an experienced microscopist in the laboratory without reference to the results of RDTs and clinical history of patient. The results of both microscopy and RDTs were matched by an independent expert. A slide was considered positive if atleast one asexual form of parasite was detected in 100 microscopic fields in thick blood film. Blood parasite density was determined from the thick films by counting the number of parasites against 200 white blood cells (WBC) and assuming that each subject had 8000 white blood cells/μl of blood. All negative slides that test positive on the RDT/PCR or all positive slides that test negative on the RDT/PCR were re-examined by another expert technician blinded to the results of microscopy, RDT/PCR and clinical status of the patients. The PCR was performed blind on coded samples by an independent Research Assistant unaware of clinical status of patients, result of RDTs and microscopic examination. Every person positive for falciparum malaria by RDT or by microscopy was treated with a combination of artesunate and sulphadoxine-pyremethamine (ACT) or with chloroquine (CQ) if RDT/microscopy showed non-falciparum infection.

For testing temperature stability of the tests, RDTs were stored at 25°C on receipt in the study sites, then allocated to separate groups for storage at 35°C & 45°C for 90 days, at 60°C for 48 hours, and at -10°C for 60 minutes before testing [21]. At the start of the study, the incubators were stabilized at the required temperature for three days before the RDTs to be tested were placed inside. RDTs were removed from storage to reach room temperature for 2 hours before testing and comparisons were made with control RDTs kept at 25°C until use and with microscopy. The malaria RDTs used in this study were from single lots of commercially available products.

Polymerase chain reaction

The DNA was isolated from the blood by using the commercially available DNA extraction kit (Bio Basic Inc) as per manufacturer protocol and also from archive blood spots by Tris-EDTA (TE) buffer method. PCR for the identification of malaria parasite was performed following the standard methods [22].

Data analysis

Results of the RDT and microscopy examination were recorded on separate forms. After double key data entry, the database was rechecked for all inconsistent entries and errors were corrected. Data were then analysed using STATA 8.2 (StataCorp, College Station Texas, USA). The figures for sensitivity, specificity, predictive values, accuracy, the area under the receiver operator characteristic curve (AUC) and the likelihood ratios were calculated using the 'diagt' command in Stata [23]. All estimated parameters are detailed with a 95% Confidence Interval (CI) unless stated otherwise.

Results

During the study period, 409 patients (age 1 to 69 years) attended the two sites 236 patients (57.7%) were screened at Dindori and 173 (42.2%) were screened at Shivpuri (mean age 15.45 ± 14.15). 37 patients (9%) were excluded as not fulfilling the study enrolment criteria due to recent anti-malarial intake. 372 patients were eligible and all these patients were recruited (mean age 15.03 ± 14.07). All recruited patients were tested by microscopy, RDT and PCR (Figure 2).

Figure 2
figure 2

Total field clinic attendance and patients recruited for malaria screening by Rapid Diagnostic Tests, Microscopy and PCR.

Full size image

A total of 190 (51.1%) were found infected by microscopy, 57 (15.3%) with P. vivax, 122 with P. falciparum (32.8%) and 11 (3%) with both P. vivax and P. falciparum. The overall sensitivity and specificity of First Response for malaria was 95.8 and 58.8%, Parascreen 93.2 and 64.3%, Malascan 90.5 and 60.4%, Falcivax 88.4 and 64.8% and by ParaHIT Total 65.8 and 76.9% respectively (Table 2). The highest sensitivity was recorded by First Response and was considered here as reference test for comparing the sensitivity and specificity of other RDTs. Parascreen was found to be 41% less sensitive than First Response (OR 0.59, 95%CI 0.24 - 1.48), though, this difference was not significant statistically (P > 0.05). Falcivax was 67% less sensitive (OR 0.33, 95%CI 0.14 - 0.78) and Malascan was 58% less sensitive (OR 0.42, 95%CI 0.18 - 1.00) than First Response and these differences were significant statistically (P < 0.05). ParaHIT Total showed lowest sensitivity (OR 0.08, 95% CI 0.04 - 0.19) and highly significant statistically (P < 0.0001). Thus, Parascreen was the first alternative to First Response. Similarly, further analysis showed that the specificity of Parascreen, Falcivax and Malascan were 1.07 to 1.26 times more when compared to First Response which is not significant (P > 0.05). However, specificity of ParaHIT Total was more than 2 times when compared to the First Response (OR 2.34, 95% CI 1.47 - 3.71), which was highly significant statistically.

Table 2 Comparative performance of Rapid Diagnostic Test kits (Parascreen, Falcivax, Malascan, First Response and ParaHIT Total) with traditional light microscopy and Polymerase Chain Reaction (PCR) for diagnosis of Malaria
Full size table

The species wise analysis revealed that the sensitivity for P. falciparum was highest (94.7%) by First Response (Table 3), while lowest for ParaHIT Total, (84.2%). The specificity for P. falciparum was highest (80.8%) by ParaHIT Total while lowest 69.5% by Malascan. The positive predictive value (PPV) for P. falciparum was highest by ParaHIT Total (70.9%) while lowest by Malascan (63.1%). The negative predictive value (NPV) was highest (96.0%) by First Response while lowest (90.2%) by ParaHIT Total. The false positive rate for P. falciparum was highest (30.5%) by Malascan (73/239) while lowest (19.2%) by ParaHIT Total (46/239). Like wise the false negative rate for P. falciparum was highest (15.8%) by ParaHIT Total (21/133) and lowest (5.3%) by First Response (7/133). When PCR was used as a reference standard the corresponding values for sensitivity, specificity, PPV, NPV and accuracy for each RDT are shown in Table 3.

Table 3 Comparative performance of Rapid Diagnostic Test kits (Parascreen, Falcivax, Malascan, First Response and ParaHIT Total) with traditional light microscopy and Polymerase Chain Reaction (PCR) for diagnosis of P.falciparum
Full size table

For non-falciparum infections i.e. P. vivax the sensitivity of the test when compared with microscopy was 84.2% by First Response, while only 15.8% by ParaHIT Total (Table 4). Specificity of the test was 100% by ParaHIT Total and 96.5% by First Response. Similarly, PPV was highest (100%) for ParaHIT Total while lowest (81.4%) for First Response. On the contrary, NPV was highest for First Response (97.1%) while lowest (86.8%) by ParaHIT Total. False positive rate for P. vivax was highest (3.5%) by First Response (11/315) and none by ParaHIT Total (0/315). On the contrary, false negative rate for P. vivax was lowest (15.8%) by First Response (9/57) and highest (84.2%) by ParaHIT Total (48/57). The values of sensitivity, specificity, PPV, NPV and accuracy using PCR as reference standard are shown in Table 4. Area under Receiver Operating Characteristic (ROC) curve (AUC) of five RDTs vs microscopy was computed for diagnosing malaria, P. falciparum and P. vivax (Figure 3) The AUC of different RDTs were significantly different for malaria (χ2 = 18.21, P < 0.001), and for P. vivax (χ2 = 108.29, P < 0.0001) but not significant for P. falciparum (χ2 = 8.47, P > 0.05).

Table 4 Comparative performance of Rapid Diagnostic Test kits (Parascreen, Falcivax, Malascan, First Response and ParaHIT Total) with traditional light microscopy and Polymerase Chain Reaction (PCR) for diagnosis of P.vivax
Full size table
Figure 3
figure 3

Area under Receiver Operating Characteristic (ROC) curve (AUC) of 5 RDTs Vs light microscopy (A) and PCR (B) as reference test.

Full size image

The analysis of RDTs sensitivity according to parasitaemia revealed that the First Response was most sensitive for diagnosis of P. falciparum (95.0%) and P. vivax (88.0%) malaria as compared to other 4 RDTs especially for levels of parasitaemia above 200 parasite/μl (Figure 4). It should be noted that in this study because of the requirement for fever in patients from a high transmission area, there were no cases in which the parasite density was ≤ 40 parasites/μl.

Figure 4
figure 4

Showing sensitivity of five RDTs according to parasitaemia

Full size image

Analysis of intensity of band and parasite density revealed that there was a weaker correlation in band intensity and parasite density for ParaHIT Total r = 0.13 (P > 0.05). Although all other four RDTs was also showing a weak positive correlation in band intensity and parasite density but it was statistically significant r = 0.17 (P < 0.025). Species wise further analysis revealed that in P. vivax all four RDTs i.e. Parascreen, Falcivax, Malascan and First Response showed very strong positive correlation in band intensity and parasite density (r = 0.50, P < 0.0001) while no statistically significant correlation was seen in P. falciparum (Figure 5).

Figure 5
figure 5

Scatter plot showing association between intensity of band (A: ParaHIT Total; B: All other 4 RDTs) and Parasite density/μl.

Full size image

Further, exposure of all RDTs to high temperature i.e. 35°C, 45° & 60°C and low temperature (-10°C) did not cause any loss of sensitivity for both P. falciparum and P. vivax except ParaHIT Total when compared with microscopy and PCR. However, there was some reduction in test line-intensity at high temperature.

Discussion

This comparative evaluation was carried out in outbreak-affected areas. From a malaria transmission perspective in both the areas, the RDTs can play a key role in rapid diagnosis and prompt treatment of malaria where resistance to CQ also necessitates the use of more expensive ACT. As RDT can be conducted immediately in the field clinic while the patient is present, the most important point for the villagers is the knowledge that they are infected with malaria parasite. On the contrary, the delay in the results of microscopic diagnosis is a serious obstacle for the operation of a malaria control programme in remote areas. Despite some inherent limitations, out of five tests evaluated, the First response was highly sensitive for the diagnosis of P. falciparum and non-falciparum especially for levels of parasitaemia above 200 parasite/μl. On the other hand its specificity was much lower than its sensitivity. Having a relatively low specificity which leads to an over-diagnosis and to an over treatment of non-malaria cases was, however, considered as less serious in such outbreak affected areas than having a low sensitivity which may lead to a potentially fatal condition being missed [24]. However, in a field setting such as ours, a negative RDT corresponds in the vast majority of cases to a non-infected individual. The high NPV allow us to confidently diagnose negative test patients as non-malaria patients [25]. Thus the risk of missing an infected individual is very small by most RDTs used in this evaluation. In Ethiopia high NPV was also recorded using Parascreen RDT in a population-based study [26] and in a health facility based study [27]. However, the sensitivity of all RDTs except First Response for non-P. falciparum infections is low (16-77%) as reported earlier from India using First Response and Falcivax RDTs [19, 28].

The performance of RDTs can be adversely affected at the temperature to which they are exposed when transported [21]. Temperatures of 35°C to 45°C are common in malaria-endemic regions and higher temperatures may be encountered during transportation. Further inadvertent freezing has also been recorded during routine shipment [29]. All types of RDTs (except ParaHIT Total) perform satisfactorily at all temperatures although we do not know whether the performance of RDTs will be equally good at low parasitaemia as recorded by some investigators [21].

A diagnostic test which is to be used in a peripheral health facility, particularly in resource poor areas, has to be simple and fast to perform by less qualified staff. Among the five RDTs tested, First Response require 20 minutes while all other RDTs require 30 minutes before classifying the RDT as negative test. Further, the First Response need only 2 drop of buffer while all other RDTs need 4 drops. Thus the First Response clearly has an advantage over other RDTs.

Conclusions

The two potential alternatives to microscopy are, PCR and RDTs. Primers exist for the reliable identification of the human malarias by PCR assays [22, 30]. However, this is largely a research tool unsuited for routine use in the field or clinical laboratory. Given the logistic and financial difficulties of the PCR in field settings, only microscopy and RDTs are viable options at present and PCR remain a future alternative to these tests when inexpensive hand held diagnostic point of care (POC) instrumentation to detect malaria is available [31]. The practical and fast nature of RDTs make them the only currently viable supplement to or replacement of microscopy based diagnosis. Thus, RDTs could play and will play an important role in malaria diagnosis in the future. However, there are reservations about how well these RDTs perform as many commercially available RDTs lack the consistency, quality control and performance capabilities as claimed by the manufacturers making their use ineffective or potentially dangerous [32]. Further, can RDTs be operated by villagers, school-teachers or forest workers, in forested inaccessible areas so that they can penetrate into areas where microscope and health facilities are non existent? This is an important question which can only be answered when more experimental next generation RDTs are available.