Background

Despite increasing efforts in prevention and treatment, malaria remains a major cause of death, especially in sub-Saharan Africa [1]. Prompt and accurate diagnosis is necessary to start adequate treatment. Microscopy, the gold standard for malaria diagnosis, is labor-intensive and requires considerable expertise [2, 3]. The use of malaria rapid diagnostic tests (RDTs) is recommended by the World Health Organization (WHO) when reliable microscopy is not available [4]. In non-endemic settings, where microscopic expertise is lacking due to low incidence, malaria RDTs are of value for the diagnosis of malaria and they provide information about the involvement of Plasmodium falciparum. In a recent external quality control session, 72.7% of 183 Belgian laboratories offering malaria diagnosis declared to use RDTs as a tool for diagnosis, and their use is recommended if performed in conjunction with microscopy [5]. RDTs are handheld cassettes detecting Plasmodium parasites by an antibody-antigen reaction. They are available in several formats, detecting one or more antigens. Targeted antigens are specific to P. falciparum (histidine-rich protein-2 (HRP-2) and P. falciparum-specific parasite lactate dehydrogenase (Pf-pLDH)), specific to Plasmodium vivax (P. vivax-specific pLDH (Pv-pLDH)) or common to the four Plasmodium species (pan-pLDH or aldolase). Most RDTs detecting P. falciparum target HRP-2, only few of them are directed to Pf-pLDH [6, 7]. However, Pf-pLDH based RDTs have advantages over the HRP-2 based RDTs, such as the rapid clearance of pLDH after successful treatment [8], the absence of the prozone effect [9] and the fact that recently observed HRP-2 gene deletions impede detection of P. falciparum by HRP-2 based RDTs [10]. For routine diagnosis of malaria at the Institute of Tropical Medicine (ITM), Antwerp, Belgium, a reference laboratory in a non-endemic setting, both a HRP-2 and a Pf-pLDH RDT are performed.

SD Bioline Malaria Ag 05FK40 (Standard Diagnostics Inc., Hagal-Dong, Korea), further referred to as SDFK40, is a one-step three-band RDT targeting Pf-pLDH and pan-pLDH. The present study was performed to evaluate the test characteristics of the SDFK40 and determine its possible role in routine diagnosis.

Methods

Study design

The study was performed in a non-endemic reference setting and consisted of two parts: a retrospective study on stored clinical samples and a prospective study in which the SDFK40 was challenged to fresh samples and run side to side to two other RDTs, OptiMAL and SDFK60, which were used as part of routine diagnostic procedure. The reference method was microscopy corrected by PCR for malaria diagnosis and Plasmodium species identification; for determination of parasite density, microscopy was the reference method. The study design was in compliance with the STARD guidelines for presentation of diagnostic studies [11].

Patients and materials

EDTA anti-coagulated blood samples were obtained in patients clinically suspected of malaria. Most patients presented at the outpatient clinic of ITM. A small part of the samples had been submitted for confirmation to ITM by other Belgian laboratories in the scope of its reference function.

For the retrospective evaluation, a selection was made out of a collection of stored whole blood samples comprising the four human Plasmodium species at different parasite densities. Mixed infections were not included. In addition, Plasmodium negative samples obtained in patients suspected of malaria were included: these samples were microscopy and PCR-negative, and showed no test lines with any of the RDTs used in routine diagnosis (OptiMAL and SDFK60). The samples had been obtained between January 2002 and April 2009 and had been stored at -70°C. Samples collected at ITM remained at room temperature (below 25°C) for a maximum of 8 hours before storing at -70°C. Samples send to ITM by other laboratories were exposed to ambient temperatures during the time of shipment which was generally less than 24 hours with a maximum of 48 hours.

The prospective part was performed between March 2009 and October 2010 and included first samples of each patient for which microscopy or one of the both routinely used RDTs (OptiMAL and SDFK60) were positive for malaria.

Reference method

All samples were assessed by an expert microscopist for diagnosis of malaria, species identification and determination of parasite density as described previously [12]. Positive or doubtful results were repeated and confirmed by a second, blinded, expert microscopist. PCR was performed on whole blood samples for which microscopy or RDT was positive: for the retrospective part real-time PCR was adapted from Rougemont et al[12, 13], for the prospective study a four-primer real-time PCR was performed [14]. For species identification, the result of microscopy corrected by PCR results was used as the reference method.

Test platforms

The SDFK40 is a lateral flow antigen detection test in a cassette format, targeting Pf-pLDH and pan-pLDH. In case of absence of the control line the test is considered invalid and has to be repeated. A unique positive Pf-pLDH line represents a P. falciparum infection whereas a unique pan-pLDH line indicates an infection with one or more of the non-falciparum species. The presence of both test lines indicates either an infection with P. falciparum or a mixed infection with P. falciparum and one or more of the non-falciparum species.

For the evaluation of the SDFK40, RDT kits from several lot numbers were used: in the retrospective study lot numbers R081005 (n = 277) and R081006 (n = 64) were used, for the prospective evaluation lot numbers BD8002 (n = 68), BD9001 (n = 11) and R081006 (n = 101) were used. All tests had been performed at last three months before the expiry date.

The OptiMAL ® pLDH (Pan, Pf) (Diamed AG Switzerland), further referred to as OptiMAL, is one of the first released three-band RDTs, targeting Pf-pLDH and pan-pLDH. The SD Bioline Ag Pf/Pan 05FK60 (Standard Diagnostics Inc.), further referred to as SDFK60, is a three-band RDT targeting HRP-2 and pan-pLDH and is in use at ITM since 2009 [12]. All RDT kits had been stored between 18°C and 24°C. Malaria diagnosis at ITM is accredited according to NBN EN ISO 15189:2007.

Test procedures

RDTs were performed according to the manufacturer's instructions except that a transfer pipette (Finnpipette, Helsinki, Finland) was used instead of the plastic transfer devices supplied in the kit. Reading of the test results was carried out at daylight assisted by an electric bulb. For the retrospective study, readings were subsequently performed by three trained observers, of whom the first was the one performing the test. Observer 1 read the test results after 20 minutes followed by observers 2 and 3 in ten additional minutes, still within the reading time mentioned by the manufacturer's instructions (20-30 minutes). The observers were blinded to each others' readings and to the results of microscopy and PCR.

To assess line intensities, a scoring system of five categories was used as described previously [12]: none (no line visible), faint (barely visible line), weak (paler than the control line), medium (equal to the control line) or strong (stronger than the control line). The test results were based on consensus agreement, i.e. an identical result read by at least two out of three observers. In case of no consensus the result of the first reader was considered.

Inter-observer agreement was evaluated by determining overall agreement and kappa values for both the results in terms of positive and negative readings as well as for line intensities. To assess test reproducibility, 15 retrospective samples representing all species at variable parasite densities were tested on five consecutive occasions.

For the prospective study, the results of the routine diagnostic work-up were considered: the laboratory technician performing microscopy also performed the RDTs. RDT results were generally read before microscopy results were available. Only the results of the first observer were considered as a second and third were not always available within the given time. Results were recorded as test line intensities.

Statistical analysis

The interpretation of test results for P. falciparum and the non-falciparum species is shown in Table 1. Samples with pure gametocytaemia were included among the P. falciparum species. Sensitivity and specificity were calculated with 95% confidence intervals (C.I.). Proportions were assessed for statistical significance using the Pearson Chi-square test, or in case of small sample size, the two-tailed Fisher's exact test. A p-value < 0.05 was considered significant.

Table 1 Interpretation of test results
Full size table

Cramer's V for categorical variables was used to assess strength of associations between parasite densities and line intensity readings. Inter-observer agreements for line intensities and positive and negative test results were expressed by kappa values for each pair of observers and by the percentage of overall agreement between the three observers.

Ease of use

The technicians performing the tests evaluated the SDFK40 kit's content and instructions for clarity, and problems and incidents during test performance were consequently recorded.

Ethical review

The study was approved by the Institutional Review Board of ITM and by the Ethical Committee of Antwerp University, Belgium.

Results

Sample collection

The retrospective study included a selection of 341 samples obtained in 341 patients. Their median age was 34 years (range 6 months - 84 years) and eight patients (2.3%) were children under the age of five years. The male-to-female ratio was 2.25:1. The travel history was known in 86.5% (295/341) of samples: 88.1% (260/295) of patients had recently returned from Sub-Saharan Africa, 8.8% from Asia and 3.1% from Latin America. Samples consisted of all four Plasmodium species at different parasite densities (Tables 2, 3 and 4). For P. falciparum samples, the median parasite density was 1,115/μl (range 0.1 - 1,000,000/μl). For P. vivax this was 969/μl (range 60 - 32,000/μl), for P. ovale 1,331/μl (range 19 - 5,930/μl) and for P. malariae 645/μl (range 0.1 - 9,900/μl). Two out of 30 PCR-confirmed P. vivax samples had originally been diagnosed by microscopy as P. ovale, whereas 5/30 P. ovale samples had originally been diagnosed as P. vivax.

Table 2 Test lines visible for the SDFK40
Full size table
Table 3 Sensitivities and specificity of the SDFK40 for the detection of P. falciparum, retrospective samples
Full size table
Table 4 Sensitivities and specificity of the SDFK40 for the detection of non-falciparum species, retrospective samples
Full size table

The panel of prospective samples (n = 181) was obtained in 180 patients, one patient had two episodes one year apart. The median age was 37 years (range 5 months - 77 years) and 3/180 (1.7%) patients were children under the age of five years. The male-to-female ratio was 1.95:1. The travel history was known in 82.2% (145/180) of samples, of which the majority (81.4%, 118/145) had been obtained in patients returned from Sub-Saharan Africa. Species and parasite densities are listed in Tables 2 and 5, median parasite density for P. falciparum samples was 3,993/μl (range 13 - 867,788/μl), for P. vivax 3,367.5/μl (range 15 - 24,563/μl) and for P. ovale 1,904/μl (range 51 - 5,310/μl). Two PCR-confirmed negative samples had originally been diagnosed by microscopy as P. falciparum with very low parasite densities (13/μl and 47/μl respectively). One PCR-confirmed P. falciparum sample had originally been concluded as negative by microscopy and was hence assigned to the category of P. falciparum with parasite density 0-100/μl. There was one mixed P. falciparum/P. malariae infection that had originally been diagnosed as P. falciparum. This sample was not considered for calculation of RDT accuracy.

Table 5 Sensitivity of the SDFK40 according to parasite density, prospective samples
Full size table

Test characteristics

No invalid test results were observed during both the retrospective and prospective studies except for a single test carried out during reproducibility testing. In two cassette blisters, discolored (pink) desiccant was found (indicating humidity saturation), they were replaced by other cassettes. Tables 2 shows the number of positive and negative test lines for all species. P. falciparum positive samples generated both visible Pf-pLDH and pan-pLDH test lines in 94.6% (105/111) and 80.6% (87/108) of the retrospective and prospective samples respectively (p = 0.0016). Twenty out of 21 prospectively tested P. falciparum samples that reacted only with the Pf-pLDH line had been assessed with tests of the same lot number (R081006). For this lot number a single Pf-pLDH line was observed in 29.9% (20/67) of positive tested P. falciparum samples, compared to 2.9% (1/34) for lot number BD8002 (p = 0.007), and none out of 7 for lot number BD9001 (p = 0.339). In the retrospective part all but one P. falciparum samples were tested with lot number R081005, in which a significant lower percentage (5.4%, 6/111) of positive samples generated a single Pf-pLDH line compared to lot number R081006 (p < 0.0002).

Tables 3 and 4 show the test characteristics for the retrospective samples. For P. falciparum, overall sensitivity was 67.9% ranging from 9.1% at parasite densities ≤ 100/μl to 77.4% and 100% at parasite densities above 100/μl and 1,000/μl respectively. False negative results only occurred at parasite densities below 1,000/μl, of which 39.2% (20/51) had parasite densities lower than 100/μl, and in 10/13 samples with pure gametocytaemia. Sensitivity for the non-falciparum species combined was 71.6%, reaching 84.8% at parasite densities >500/μl. For P. vivax, P. ovale and P. malariae overall sensitivities were 86.7%, 80.0% and 21.4% respectively, increasing to 94.7% at parasite densities >500/μl for both P. vivax and P. ovale. For P. malariae, there were not enough samples to calculate sensitivity at parasite densities >500/μl.

Table 5 shows the sensitivity of the SDFK40 when evaluated prospectively. Sensitivities were slightly higher compared to the retrospective results: overall sensitivities for P. falciparum and for the non-falciparum species combined were 78.8% and 82.1% respectively. As for the retrospective study, sensitivities were higher at increasing parasite densities. Of note, 5/93 P. falciparum samples with parasite densities >1,000/μl gave false negative results; the highest parasite density among these samples was 3,212/μl.

Species mismatch occurred in three samples, representing 0.7% of the pooled retrospective and prospective Plasmodium positive samples (n = 411): two P. malariae samples (parasite density of 62/μl and 9,900/μl) and one P. ovale sample (parasite density 113/μl) reacted not only with the pan-pLDH line, but also showed a faint Pf-pLDH line.

Compared to the OptiMAL and the SDFK60 (Table 6), the SDFK40 had a significant lower sensitivity for detection of P. falciparum (p = 0.004 and p < 0.0001 respectively). For the non-falciparum species combined, overall sensitivity of the SDFK40 tended to be higher compared to the OptiMAL and the SDFK60, though not statistically significant. Of note, the SDFK40 performed significantly better in the detection of P. ovale compared to the OptiMAL (p = 0.005) and tended to perform better than the SDFK60 (p = 0.226). The P. vivax and P. ovale samples that were not detected by the SDFK40 in the prospective part (n = 3) had low parasite densities (51/μl, 176/μl and 304/μl) and were tested with the same lot number (R081006), as well as the single P. malariae sample (parasite density 1,920/μl) that tested negative.

Table 6 Diagnostic accuracy of the different RDTs for each species, prospective samples
Full size table

The mixed P. falciparum/P. malariae infection was found in a 11-year old child returning from Cameroon and had a parasite density of 23/μl. The sample was scored as negative by the SDFK40, as a mixed infection by OptiMAL, and as a single P. falciparum infection by the SDFK60.

In the retrospective study none of the 95 Plasmodium negative samples generated positive test results (Table 2). In the prospective study, there were five false positive results among the 15 samples that were microscopy and PCR negative but positive for one or both routinely performed RDTs (Tables 2). Two of these samples tested positive for Schistosoma antibodies, one patient had been partially treated for malaria and the remaining two samples were from a patient (tested in 2009 and 2010 after two different journeys) in whom the RDT results were interpreted as false positive without a plausible explanation.

Presence and intensity of test lines

For both test lines there was a correlation between parasite density and line intensity readings, though higher for the retrospective samples (Pf-pLDH: V = 0.6622 and V = 0,4814, both p < 0.0001; pan-pLDH: V = 0.6312 and V = 0,4263, both p < 0.0001). Among the pooled retrospective and prospective P. falciparum samples, 17.8% (39/219) of the positive Pf-pLDH test lines were read as faint. For the pooled non-falciparum samples, faint test lines were observed in 22.8% (18/79) of positive pan-pLDH lines. For both Pf-pLDH and pan-pLDH test lines, medium and strong intensities occurred exclusively at parasite densities above 1,000/μl, except for one sample with pure gametocytaemia.

Reproducibility and inter-observer agreement

For both Pf-pLDH and pan-pLDH test lines overall agreement and kappa values between pairs of observers were excellent for both positive and negative readings (≥ 94.7%, kappa values ≥ 0.92) and line intensity readings (≥ 86.5%, kappa values ≥ 0.84). Test results were reproducible and all discordances in line intensity occurred within one category of difference.

Ease of use

The SDFK40 was evaluated as easy to use, with an excellent clearance of the background and clearly visible test lines. In most tests the background became blurred after 60 minutes and test lines were no longer visible. The test instructions in the package insert were clear and accompanied by well-designed pictures. The part presenting invalid test results (absence of control line) failed to demonstrate pictures of a pan line without control line as well as both test lines without control line.

Discussion

In this study, the performance of the malaria RDT SDFK40 was evaluated on a panel of stored (n = 341) and fresh (n = 180) whole blood samples, obtained in returned international travelers suspected of malaria. Sensitivity was poor for P. falciparum samples at low parasite densities, but excellent for detection of P. falciparum at high parasite densities, as well as for P. vivax and P. ovale.

The present study has its limitations. Although evaluation of a RDT in a standardized laboratory is a logical step preceding studies in field settings [15], it is clear that conditions in a reference setting such as the present one are more favorable compared to those in field settings (e.g. temperature control, training and expertise of staff). Another limitation was that, for the retrospective samples, it was impossible to explore the cause of discordant results. Furthermore stringent criteria were used. For instance, species mismatch was considered as a false negative result, although the diagnosis of malaria had been confirmed by the SDFK40: considering the non-falciparum species that reacted not only with the pan-pLDH line but also with the Pf-pLDH line as true positives would have increased the sensitivity for P. ovale to 84.6% (prospective study) and for P. malariae to 35.7% (retrospective study). In addition, samples with pure gametocytaemia were included among the P. falciparum samples [16]: assigning them to the Plasmodium negative samples would have increased overall sensitivity to 71.9% and 80.8% for the retro- and prospective samples respectively, at the expense however of a lower specificity (97.3%) in the retrospective panel. Another limitation was that, in the prospective study only the results of a single observer were considered, as a second and a third observer could not always perform reading within the given time. However, this probably has not influenced test results as the retrospective study revealed high inter-observer agreements. Finally, in the prospective study there was no Plasmodium negative control group (microscopy and RDT negative) included, precluding calculation of diagnostic specificity and predictive values.

The SDFK40 has been evaluated in two other studies: in the first round of RDT evaluation by the WHO, detection of P. falciparum and P. vivax was assessed using diluted samples at fixed parasite densities. Detection rates for P. falciparum and P. vivax were 29.1% and 50.0% at low (200/μl) parasite densities and 96.2% and 100.0% at high (2,000/μl and 5,000/μl) parasite densities respectively [6]. In a field setting in Madagascar, overall sensitivities were 89.4% for the detection of P. falciparum and 73.3% for the non-falciparum species [17]. Conform the present study, there was a higher detection rate for P. vivax (8/9 samples) compared to P. malariae (3/6 samples).

Data assessing the test characteristics of pLDH RDTs for non-falciparum infections are limited [18], but the low sensitivity for diagnosis of P. malariae and P. ovale is a well-known phenomenon among RDTs [16, 19–21]. Unlike the previous studies, the present one included samples of P. ovale. The high sensitivity of SDFK40 for detection of P. ovale is remarkable and exceeds that of other RDTs evaluated in non-endemic settings [16, 20–22] as well as that of the OptiMAL and SDFK60 run simultaneously. In the present setting at ITM, a comparable sensitivity (76.3%) for P. ovale has only been achieved for the SDFK60 [12]. Although species mismatches were rare and did not involve misidentification of P. falciparum as non-falciparum species, the cross-reaction of two P. malariae samples and one P. ovale sample with the Pf-pLDH should be taken into account and further studied.

The side-to-side comparison with the OptiMAL and the SDFK60 revealed significantly lower overall sensitivities for the detection of P. falciparum by the SDFK40. HRP-2 is known to be detected at lower parasite rates compared to pLDH [2] and this might explain for the difference in sensitivity between the SDFK40 and the SDFK60, as the latter tested positive for all P. falciparum samples at parasite density 0-100/μl, compared to only 1/11 for the SDFK40. Compared to the OptiMAL, the SDFK40 showed a lower sensitivity for P. falciparum but a higher sensitivity for the non-falciparum species, especially P. ovale. These differences may be explained by differences in affinity of antibodies [2] or by reaction conditions created by the buffer composition favoring the pan-pLDH antigen-antibody interaction over that of Pf-pLDH in the case of the SDFK40.

From the present study, it is difficult to estimate the incidence of false positive RDT results in the SDFK40: in the retrospective part, none of the Plasmodium negative samples reacted positive. These samples had also been tested negative by the routinely performed RDTs. In contrast, the prospective panel included microscopy negative samples that had given positive results with one or both of the routinely used RDTs (the SDFK60 and the OptiMAL) for diagnostic work-up, which may explain for discordance in false positive results between the prospective and retrospective panel. Among these samples, false positive reactions occurred less frequently in the SDFK40 compared to the OptiMAL and the SDFK60. For the SDFK60, persistence of HRP-2 may explain for part of the false positive results. The pLDH enzyme - expressed by viable parasites - declines rapidly after therapy but is also expressed by gametocytes [2, 23]. Prospective monitoring of diagnostic characteristics on large series will be needed to get reliable estimates of the number of false positive SDFK40 results.

Test line intensities of the SDFK40 correlated to parasite densities, but there was much overlap, precluding reliance on line intensities as an indicator of parasite density. The test line intensity of about one out of five positive test lines was scored as faint. In the present study, observers were trained to distinguish faint test lines and to interpret them as positive; in field settings however, disregarding faint test lines as negative is a well known error that might interfere with correct reading, especially at unfavorable light conditions [24–27].

Comparison of the retro- and prospective panels revealed some differences. In the prospective panel, the overall sensitivity of the SDFK40 for the diagnosis of P. falciparum was higher as compared to the retrospective panel. This may be attributed to differences in parasite density between both panels: 52.3% (90/172) of P. falciparum samples in the retrospective panel had a parasite density ≤ 1,000/μl compared to only 32.1% (44/137) in the prospective panel (Tables 3 and 5), whereas sensitivities at low (≤ 100/μl) and high (>1,000/μl) parasite densities were similar for both panels. Although the difference in parasite density appears to be the most plausible explanation, an influence of sample storage cannot be excluded. In a previous study [20], we showed evidence for decline of parasite pLDH in the case of P. ovale, but this effect was noted for samples stored for extended periods (more than 8 years) which were not included in the present study. Another difference was the presence of a single test line versus both test lines in the case of P. falciparum samples: in the retrospective panel, the majority of P. falciparum samples showed both Pf-pLDH and pan-pLDH lines, while this proportion was significantly lower in the prospective panel. The presence of a single Pf-pLDH line occurred almost exclusively among samples tested with a single lot (R081006). Lot number R081006 had been used in the retrospective study for only one P. falciparum sample, making definite comparisons impossible; nevertheless a lot-related phenomenon is highly suggestive. Lot-to-lot variability in RDTs is a well-known phenomenon and is an issue in RDT performance monitoring and quality control [6, 7, 28, 29]: according to WHO guidelines, each new lot number of a RDT used in diagnosis should be evaluated by a reference laboratory before release [30].

Compared to HRP-2 RDTs, pLDH-based RDTs are generally reported to have lower sensitivities for the detection of P. falciparum, especially at parasite densities ≤ 100/μl [2, 31, 32]. In addition, pLDH RDTs have been reported to be less resistant to heat degradation in comparison to HRP-2 RDTs [33], although the SDFK40 package insert claims heat stability up to 40°C and the heat stability testing of the WHO/FIND study confirmed resistance to temperatures up to 45°C [6]. On the other hand, pLDH RDTs have advantages over the HRP-2 based RDTs: as mentioned above, pLDH, unlike HRP-2, does not persist long after treated or past P. falciparum infections, and unlike HRP-2 based RDTs, they are not susceptible to the prozone phenomenon, i.e. the occurrence of test negative or low-intensity test lines at elevated parasite densities [9]. In addition, HRP-2 gene deletions and polymorphisms have been described, possibly limiting the use of HRP-2 based antigen tests in the affected areas [10, 34, 35]. For pLDH such genetic variability has not been described [36].

For the non-endemic setting, the low sensitivity of the SDFK40 for the detection of P. falciparum at parasite densities below 1,000/μl is of concern and precludes its use in routine diagnosis, as non-immune travelers may be symptomatic at such low parasite densities [2]. However, the particular advantage of the SDFK40 in this setting is its high sensitivity for both P. vivax and P. ovale, but it should be used in conjunction with another brand known for reliable detection of P. falciparum.

Conclusion

Challenged to a panel of clinical samples in a reference setting, the SDFK40 showed a low sensitivity for detection of P. falciparum samples at parasite densities ≤ 100/μl, precluding its use as the only diagnostic tool for malaria diagnosis. However, SDFK40 performed excellent for P. falciparum samples at high (>1,000/μl) parasite densities as well as for detection of P. vivax and P. ovale at parasite densities >500/μl.