Background

Malaria remains one of the most important infectious diseases in tropical area, affecting over 300 million people every year, mainly in sub-Saharan Africa [1,2]. As recommended by the World Health Organization (WHO), the management of suspected malaria cases relies on early diagnosis and prompt and effective treatment based on artemisinin-combined therapy (ACT) [3].

Malaria diagnosis has long been based on the microscopy examination of Giemsa-stained blood film, despite the requirement for highly qualified microscopists and reliable equipment, which are often lacking in remote areas where malaria is most prevalent [4]. For over a decade, the development of malaria rapid diagnostic tests (RDTs) has enabled reliable biological diagnostic testing in all situations where previously only clinical diagnosis was available [5]. The main advantages of the malaria RDTs are that they are easy to use, do not require electricity or complex equipment and the results are available in 15–30 minutes after finger-prick blood collection [6]. These lateral flow immunochromatographic tests are usually presented in various formats (dipstick, plastic cassette or card). They contain antibodies conjugated to colloidal gold or latex particles, which bind specifically with parasite antigens. They are generally based on the detection of the histidine-rich-2 protein (HRP-2) specific to Plasmodium falciparum but often combined with the detection of other antigens common to all species, such as lactate dehydrogenase or aldolase. These combined RDTs can detect both P. falciparum and non-P. falciparum (Plasmodium vivax, Plasmodium ovale and Plasmodium malariae) infections [7].

Over the past few years, the number of malaria RDTs and the scale of their use have rapidly increased. With over 100 commercially produced RDTs, this market has led to a proliferation in product availability against a background of relatively low capacity for regulation and quality control in many low-resourced, malaria-endemic countries. The variable quality of malaria RDTs, and consequently their diagnostic performance, have made it difficult for policy makers to determine which tests are the most suitable [5], even with the implementation of the lot-testing programme, led by WHO in partnership with the UNICEF/UNDP/World Bank/WHO Special Programme for Research and Training in Tropical Diseases (TDR) and the Foundation for Innovative New Diagnostics (FIND) [8].

At present, malaria RDTs are considered as point-of-care tests because they are mainly used by health care volunteers at community level, in remote malaria areas. Their use in field conditions allows an early diagnosis of malaria in any febrile patient and prompt treatment of confirmed cases. However, most of the studies aiming to assess the performance of malaria RDTs are conducted by research teams that are not representative of the classical end-users, who are typically unskilled in traditional laboratory techniques for diagnosing malaria. Consequently, only the ‘intrinsic’ performance of malaria RDTs is evaluated and information regarding their ‘global’ performance (including ease of use, execution, interpretation of results, and adherence to test results) when they are used by the traditional end-users is lacking.

In this context, a protocol for evaluating the global performance of malaria RDTs by end-users in malaria-endemic areas was developed. As an example, the performance of the VIKIA® Malaria Ag Pf/Pan test, previously assessed in 2013 [9], was re-evaluated by representative end-users, using well characterized and calibrated blood samples in several testing sites in Cambodia. Results obtained by the end-users were compared to the corresponding expected results. In addition, a questionnaire assessing the ease-of-use of the RDT with the support of the package insert and the quick guide was also developed.

Methods

VIKIA® Malaria Ag Pf/Pan test

The VIKIA® Malaria Ag Pf/Pan (IMACCESS/bioMérieux, Lyon, France) is a test based on immunochromatographic technology for detecting P. falciparum and other species of Plasmodium (P. vivax, P. malariae and P. ovale). This test is a ready-to-use cassette device, requiring the operator to transfer 5 μL of blood in the sample well with an appropriate device supplied in the kit, add five drops of the lysis buffer in the buffer well and read the results visually after 20 minutes (see Additional files 1 and 2; SF1 Quick guide for using the VIKIA® Malaria Ag Pf/Pan RDT and SF2 Package Insert of the VIKIA® Malaria Ag Pf/Pan RDT).

Participating communes and end-users

Seven communes in Kampot Province (Cambodia) were selected for this multisite, blinded study. Four different end-user profiles were considered as representative of the typical malaria RDTs end-users found in malaria-endemic areas: health medical centre staff, private physician, community health worker (village malaria worker (VMW)) and private pharmacist. Five individuals of each end-user profiles were enrolled in the study, as presented in Figure 1 and Table 1.

Figure 1
figure 1

Location of the selected end-users, Kampot Province, Cambodia.

Full size image
Table 1 List of selected end-users, Kampot Province, Cambodia
Full size table

Sample characteristics

Three types of sample (negative, low positive, high positive) were tested in replicates of four, corresponding to a panel of 20 aliquots, as followed: one negative sample, one low positive sample at 200 parasites of P. falciparum/μL, one low positive sample at 200 parasites of P. vivax/μL, one high positive sample at 2,000 parasites of P. falciparum/μL and one high positive sample at 2,000 parasites of P. vivax/μL.

The set of 20 coded aliquots was prepared according to the Standard Operating Procedures (SOP) developed in the Methods Manual for Laboratory QC Testing of Malaria RDTs (Version 7, June 2014) and codified as described in Table 2. Briefly, the preparation of positive parasite samples was performed in three steps: i) blood sample collection from consenting malaria patients (strong positive detection by RDT and parasite density > 2,000 parasites/μL of blood by microscopy); ii) collection of parasite-free (controlled by PCR [10]) and virus-free (controlled by serological screening for HIV, hepatitis B and hepatitis C infections) blood from the Phnom Penh blood bank and, iii) dilution of the parasite-positive bloods in low (200 parasites/μL) and high (2,000 parasites/μL) parasite densities and preparation of parasite-negative QC samples and aliquoting (50 μL) in cryotubes frozen at −80°C.

Table 2 List of the set of 20 parasite-calibrated blood aliquots used by the selected end-users, Kampot Province, Cambodia
Full size table

Frozen calibrated blood aliquots were transported at −80°C in a portable freezer running on 12 V (works with a cigarette lighter in a car). Defrosting was performed by the research team just before its use by the end-user (Figure 2).

Figure 2
figure 2

Flow chart of the use in field conditions of well-characterized and calibrated blood samples (set of 20 aliquots) by selected end-users, Kampot Province, Cambodia.

Full size image

Training

Before the start of the study, the research team explained the study objective to all end-users. Testing was performed directly by the end-users without any practical training on the VIKIA® Malaria Ag Pf/Pan kit. The only instructions for use were available within the quick guide and the package insert provided in the VIKIA® Malaria Ag Pf/Pan kit (see Additional files 1 and 2).

Quality control assessment of the calibrated aliquots

The research team performed a quality control assessment of the set of the 20 aliquots at three different times (at the start, middle and end of the study), to confirm the good quality of the aliquots distributed to the users and to verify that no degradation occurred during transport or storage that could affect the expected results.

Statistical analysis

Data were recorded and analysed using Excel software and MedCalc (MedCalc Software, Belgium). The Chi-squared test was used to compare the proportion of positive results among end-users. A P-value <0.05 was considered to indicate statistical significance.

Results

Quality assessment testing results

Three successive quality assessments were performed by the research team: firstly in Phnom Penh at Institut Pasteur in Cambodia before going into the field, secondly and thirdly, seven and 14 days later, respectively, in the field. Results are presented in Table 3.

Table 3 Results of the three quality assessments performed by the research team
Full size table

Quality assessments 2 and 3 showed a significant degradation of the quality of three out of four aliquots containing P. vivax at 200 parasites/μL. For this reason, aliquots ID 4, ID15 and ID19 were excluded from the final analysis of the end-user testing results.

End-user testing results

Among the 20 selected end-users, the results observed were consistent with the expected results, except for the low positive (200 parasites/μL) P. vivax aliquot with 35% of concordant results (Table 4).

Table 4 Results of the tests performed by selected end-users
Full size table

No significant difference was observed between the different end-users. For the low positive (200 parasites/μL) P. vivax aliquot, the proportion of positive results obtained by the VMW end-users (80%) was higher than other end-users (0-40%), but the difference was not significant (P = 0.22), and was probably due to the low number of aliquots tests (five per end-user).

Questionnaire evaluation analysis

After performing the test, all end-users were interviewed and completed a questionnaire (‘use and usability’) to evaluate ease-of-use and ease-of-reading of the VIKIA® Malaria test. Results are presented in Table 5. A total of 159 positive answers (yes, 99.4%) and only one negative answer (no, 0.6%) were recorded.

Table 5 Results of the questionnaire evaluation, Kampot, Cambodia, 2013
Full size table

Out of the 20 end-users, only one considered the test was not easy to perform with the support of the quick guide. However, with the help of the package insert, all end-users considered that the operating instructions were clear enough to perform the test without having to participate in practical training. Moreover, all end-users claimed that the package insert allowed an easy interpretation of the results.

Discussion

In endemic areas, health facility staff and community workers are in the front line to manage malaria cases, which mostly affect rural inhabitants, for achieving the laudable goal of elimination. To this end, early diagnosis of malaria by using malaria RDT and prompt treatment with effective anti-malarial drugs are the cornerstone of the management of suspect malaria cases [11]. Therefore, malaria RDTs are one of the most important tools to ensure that only malaria-infected patients receive anti-malarial drugs, limiting the unnecessary use of inappropriate treatment and thereby avoiding selection and spread of drug-resistant P. falciparum parasites, an important issue in Southeast Asia [12]. According to WHO, most of the 108 million malaria RDTs delivered in 2012 were used in the African region (78%), followed by the Southeast Asia region (16%) and Eastern Mediterranean region (3%) [11].

In this context, the availability of accurate, high performance malaria RDTs in terms of sensitivity, specificity, positive, and negative predictive values is crucial. However, most of the studies aiming to evaluate the performance of commercialized malaria RDTs are conducted by research teams, that are not representative of the traditional end-users in remote areas, whereas several studies have clearly shown that the accuracy of malaria RDTs is highly user-dependent [13-20]. Indeed, it has been demonstrated by Rennie et al. that despite their apparent simplicity of use, the accuracy of malaria RDTs also depends on the accuracy of their preparation and interpretation [21]. In most of the studies, data relating to the quality of packaging and content, ease of use and ease of interpretation of results from malaria RDTs are missing [22].

In the present study, a dedicated protocol was designed to re-evaluate the performance of the VIKIA® Malaria Ag Pf/Pan test, previously assessed in 2013 [9], by representative end-users. To this end, well-characterized and calibrated blood samples were used, along with in-depth interviews to assess ease-of-use and ease-of-reading of the VIKIA® Malaria Ag Pf/Pan test. Globally, the performance of the VIKIA® Malaria Ag Pf/Pan test performed by the 20 selected end-users was consistent with expected results and similar to those previously observed by Chou et al. [9]. No significant difference was observed between the different end-users, including health medical centre staff, private physicians, community health workers and private pharmacists. With the instructions provided in the quick guide and the package insert, most of the end-users considered that the VIKIA® Malaria Ag Pf/Pan test was easy to use and the results were easy to interpret.

The results observed with the low positive P. vivax aliquot (200 parasites/μL) with a sensitivity of 35% (95% CI: 15.4-59.2%) were similar to the sensitivity reported for P. vivax samples containing 101–500 parasitaemia/μL of blood by Chou et al. (from 36.4 to 61.9%, according to the reading time, 10–60 minutes) [9]. In addition, data from the quality control assessment performed at three different times (at the start, middle and end of the study), confirmed that the storage of low parasitaemia P. vivax samples at low temperatures had probably accelerated the deterioration of antigen activity (among the 12 low positive P. vivax aliquots, seven were positive and five were negative) [7]. The 200 parasites/μL density level was included in the protocol according to the 1999 and 2003 WHO consultations (above the 100 parasites/μL level) [23,24]. The 2010 WHO consultation on parasite detection confirmed the relevance of this level for clinical management, in high and medium/low transmission areas [25], taking into account that clinical malaria at low parasite densities occurs in a number of communities, especially in unstable low transmission areas.

The main limitations of the study were the limited number of samples tested, especially the number of low positive P. vivax aliquots and the use of frozen samples, although no other option was available. Indeed, it has been claimed by some RDT manufacturers that frozen blood samples are not optimum for RDT testing. They argue that malaria RDTs are optimized and recommended to be used with fresh finger-prick blood and that frozen blood samples could lead to aggregation, precipitation and leeching out of artefacts into the thawed blood. As all RDTs use a porous nitrocellulose membrane, the presence of artefacts clogging the membrane can alter the flow properties of the reagents and affect the completion of the test run in specified time. As a future option, WHO, TDR and FIND, along with other partners, have started the development of a generation of lot-testing samples based on the use of recombinant proteins, which could be used by community health workers to confirm quality of RDTs and significantly increase the confidence of national malaria control programmes in the results of RDTs.

Conclusions

The data presented in this study clearly demonstrate that the performance of the VIKIA® Malaria AgPf/Pan rapid test performed by traditional end-users in field conditions is similar to those obtained by a research team, and that VIKIA® Malaria AgPf/Pan can be considered as a point-of-care test. In addition, the protocol designed for this study, using well-characterized and calibrated blood samples at 2,000 and 200 parasites/μL could be used systematically in parallel to conventional evaluation studies [7] to determine the performance of malaria RDTs in field conditions.