Background

Anaemia constitutes a major global public health problem. Globally, it currently affects two billion people (32.9% of the world’s population) [1]. However it is disproportionately concentrated in developing nations where it is four times more prevalent than in developed world [2]. Anaemia also results in one million deaths annually, with three-quarters of these occurring in Africa and South East Asia [3]. Anaemia is associated with major health consequences including decreased well-being, fatigue, lethargy, impaired physical activity and work performance [4]. In children it is associated with increased mortality and decreased cognitive and physical development [5, 6]. In pregnant women it is associated with low-birth weights, increased risk of maternal and perinatal mortality [7].

Despite the fact that the burden of anaemia is highest in developing nations, standard medical testing facilities are often limited and inaccessible to most patients [8]. Thus most cases of anaemia will go undiagnosed and untreated [9]. The economic realities in developing nations makes it cost-prohibitive to setup and run standard laboratory facilities at all levels of the health care system [8, 10, 11]. The use of point-of-care (POC) haemoglobin meters is potentially economically advantageous and time-saving and may also be of tremendous help in critical care units where rapid and frequent testing are required [11, 12]. These devices are much cheaper than automated analysers and do not require sophisticated expertise training to be able to operate them [13]. As such they are usually operated by a non-laboratory staff such as a physician who is directly involved in patient care [12]. They are battery-powered and will be highly beneficial in settings with inconsistent electric power supply. They are either bench-top or hand-held thus do not require sophisticated infrastructure [14]. They require smaller blood samples and thus are valuable in conditions that require multiple blood draws such as in critical care units and neonatal/paediatric units [12]. This reduces the risk of iatrogenic anaemia. They require capillary blood samples and will be appropriate in situations where it is technically difficult and distressing to perform phlebotomy [15].

Despite the potential benefits of using POC haemoglobin meters in clinical practice, there is still a lot of concern surrounding their accuracy [16, 17]. Do they underestimate haemoglobin values and thus expose patients to needless and sometimes harmful treatment for anaemia such as blood transfusions or do they overestimate haemoglobin values and as such prevent patients with anaemia from receiving appropriate treatment for anaemia? All these concerns have slowed their adoption in clinical practice. We aimed to evaluate the technical and clinical accuracy of two POC haemoglobin meters and their agreement with a reference analyser in diagnosing anaemia in the African setting.

Methods

Study design, setting and population

This was a hospital-based cross-sectional study carried out at the Douala General hospital over a period of 4 months from 1st November 2015 to 29th February 2016.

The Douala General hospital is a tertiary care hospital located in the city of Douala, the economic capital of Cameroon. Its laboratory is accredited by the World Health Organisation (WHO) and runs a 24-h shift every day. The laboratory is equipped with two automated full blood count (FBC) analysers: a Urit-3300®analyser (Urit medical Electronics Co. Ltd., Guangxi, China) and a CELL-DYN RUBY®analyser (ABBOTT DIAGNOSTICS, Illinois, USA). The laboratory participates in a three-monthly external quality control programme with peer laboratories in Douala and France.

All in- and out-patients, both children and adults, who were prescribed a full blood count during the study period were eligible to participate. This was irrespective of their underlying conditions, indications or the presence or absence of anemia. The etiology of anaemia in study participants, if present, was beyond the scope of this study. Coagulated blood samples and those who posed difficulty getting enough capillary samples to cover the test areas on the test strips were excluded from this study.

The study was approved by the Institutional Ethics Committee for Research on Human Health, University of Douala and administrative clearance obtained from the authorities of the Douala General Hospital.

Selection of haemoglobin meters

The point-of-care haemoglobin meters were selected by contacting all hospital laboratories and distributors of health care products in order to ascertain all the brands are available in Cameroon. During this process, five meters were identified: MissionHb® (ACON Laboratories, Inc., San Diego, USA), Urit12® (URIT Medical Electronics Co.,Ltd. Guangxi, China), HemoCue® (HemoCue, Angelholm, Sweden), DHT® meter (Developing Health Technologies, Ipswich, UK) and Humameter® (HUMAN Medical Diagnostics, Wiesbaden, Germany). The last three meters are bench-top haemoglobin meters which function based on spectrophotometry with the use of microcuvettes and require expert operation. Hence, they were excluded from this study.

MissionHb® and Urit12® are both hand-held devices which use strip technology based on reflectance photometry and display haemoglobin values within 15 to 30 s of application of a drop of capillary blood.

Data collection and Haemoglobin measurement

For each participant, data was collected on age, sex and pregnancy status. Then capillary haemoglobin was measured using standard operating procedures and in compliance with the manufacturers’ prescriptions. After letting the participant’s hand to hang vertically for 20 s to allow blood flow by gravity, the side of pulp of the middle finger is cleaned with an alcohol swab and then dried. It is pricked to obtain capillary blood and the first drop of blood wiped off. The respective strips were inserted into each meter and when indicated by the meter, a drop of blood was applied to the test spot. The meters then displayed the readings within 15 to 30 s. Measurements with the haemoglobin meters were done by a final year medical student. This was done concomitantly with collection of venous samples for full blood count analysis. A full ethylenediaminetetra-acetic acid (EDTA)-anticoagulated tube of venous sample was obtained from the median cubital vein by trained health personnel. The full blood count analysis was done using the CELL-DYN RUBY® according to its standard operating procedure, with daily quality control tests on both meters and the CELL-DYN RUBY®. The reference standard results were not available to the investigator all through the tests with the haemoglobin meters.

Definition of terms

Anaemia was defined using the WHO age and sex-specific cut-off values of haemoglobin for mild, moderate and severe anaemia [18]. These cut-offs are shown on Table 1 underneath.

Table 1 Haemoglobin values for the diagnosis of anaemia
Full size table

Units of haemoglobin in g/l

Statistical analysis

Data were analyzed using STATA 14 (Stata Corporation, Texas, USA). The various haemoglobin readings were summarized as means and standard deviations. Paired sample t-test was used to compare the mean haemoglobin value of each haemoglobin meter with the reference value. The continuous agreement between haemoglobin meters-based measurements and reference value was assessed with the use of Pearson’s correlation, and Bland-Altman plots. The Clinical Laboratory Improvement Amendments (CLIA) 2014 total error allowable desirable limits for haemoglobin measurement (± 4.19%) was used to assess the clinical accuracy of the haemoglobin meters [19]. Receiver Operating Curves (ROC) comparing both meters to the reference analyser in detecting the presence of anemia were plotted and the respective areas under the curve (AUC) and confidence intervals were calculated. Cohen’s kappa statistic was used to assess for the agreement of each meter with the reference analyser in diagnosing the severity of anaemia [20].

The study was reported according to the Standards for Reporting of Diagnostic Accuracy studies (STARD) guidelines, as per the STARD 2015 checklist [21].

Results

A total of 228 participants (104 [45.6%] being men) with 8 (6.45% of women) being pregnant were included. Ages ranged from 4 to 85 years with a mean of 42.4 ± 17.3 years. The general characteristics of the study population are shown on Table 2.

Table 2 General characteristics of study participants recruited during this study
Full size table

Technical accuracy

The differences in haemoglobin measurement between each haemoglobin meter and the reference analyser ranged from −3.3 to 4 g/dl with a positive mean bias of 1 g/dl for MissionHb® (p < 0.001) and − 3.2 to 3.6 g/dl with a positive mean bias of 0.3 g/dl for Urit12® (p < 0.001). A positive correlation was observed between the two POC meters and the reference analyser; with Pearson’s correlation coefficients 0.90 (MissionHb®) and 0.89(Urit12®). This is shown on Fig. 1. No significant difference between the two correlation coefficients was present (Steiger test: z = 0.46, p = 0.66).

Fig. 1
figure 1

Scatter plot of MissionHb® versus reference haemoglobin values (Panel a) and Urit12® versus reference haemoglobin values (Panel b). X-axes: reference haemoglobin values (g/dl) Y-axes: MissionHb® or Urit12® haemoglobin values respectively

Full size image

Bland-Altman plots showing the agreement between the POC meters and the reference analyser are depicted in Fig. 2. MissionHb® meter had 7/228 (3.1%) of the difference scores out of the 95% limits of agreement (− 1.1, 3.2), with 2 values being in the positive region and 5 in the negative region. The mean bias was + 1 g/dl with a standard deviation of 1.1. The width of the 95% limits of agreement was 4.3 g/dl. Urit12® meter had 10/228 (4.4%) of the difference scores out of the 95% limits of agreement (− 2.1, 2.7), with 5 being in the positive region and 5 in the negative region. The mean difference (bias) difference was + 0.3 g/dl, with a standard deviation of 1.2. The width of the 95% limits of agreement was 4.8 g/dl.

Fig. 2
figure 2

Bland-Altman graph of agreement between MissionHb® and the reference analyser (Panel a), Urit12® and reference analyser (Panel b). X-axes: average of respective POC and reference haemoglobin values (g/dl); Y-axes: differences in respective POC and reference haemoglobin values (g/dl). The middle horizontal lines represent the mean biases (g/dl) of the respective haemoglobin meters. The outer horizontal lines represent the respective 95% limits of agreement

Full size image

Clinical accuracy

Both haemoglobin meters overestimated haemoglobin values as their measured mean haemoglobin values were significantly higher than measured by the reference analyzer (Table 3). However, only the MissionHb® meter had mean haemoglobin values above the clinically allowable range when taking into consideration the desirable limit for bias in haemoglobin measurement (Table 3).

Table 3 Clinical Significance of the different Haemoglobin meter readings based on total allowable error
Full size table

Agreement with the reference analyser in diagnosing the presence and severity of anemia

One hundred and seventy-one (75%) participants had anaemia with 62 (27.2%) being mild, 68 (29.8%) being moderate and 41 (17.9%) being severe, based on the results provided by the reference analyser. In general, both meters were excellent in diagnosing the presence of anemia as evidence by high AUC values for the ROC comparing them to the reference analyser (MissionHb® - AUC = 0.9161, Urit 12® - AUC = 0.9009) as summarised on Table 4 and Fig. 3. However, with regards to distinguishing between the degrees of severity of anaemia among the participants, both haemoglobin meters had weak agreement with the reference analyser (K = 0.39, p < 0.001 for MissionHb® and K = 0.54, p < 0.001 for Urit12®) as shown on Tables 5 and 6 below.

Table 4 Area Under the Curve (AUC) for the Receiver Operating Curves (ROC) comparing the MissionHb® and Urit 12® meters to the reference analyser
Full size table
Fig. 3
figure 3

Receiver Operating Curve (ROC) comparing the MissionHb® and Urit 12® meters to the reference analyser

Full size image
Table 5 Agreement in the diagnosis of anaemia using kappa statistics between MissionHb® and the reference analyser
Full size table
Table 6 Agreement in the diagnosis of anaemia using kappa statistics between Urit12® and the reference analyser
Full size table

Discussion

In our study we noticed a positive and significant correlation between each haemoglobin meter and the reference analyser (r = 0.90 for MissionHb® and r = 0.89 for Urit12®). However, both devices did not meet clinical accuracy requirements using total error allowable. Both meters significantly overestimated haemoglobin values and both had a weak agreement with the reference analyser in diagnosing the presence and severity of anaemia among our participants.

The positive correlation implies there was a strong direct linear relation between the measurements of each haemoglobin meter with those of the reference analyser. These results were comparable to that obtained for a bench-top haemoglobin meter-the HemoCue®(HemoCue, Angelholm, Sweden) by Lamhaut et al., in France [17]. HemoCue® is the most popular and widely studied haemoglobin meter. Although other testing systems are available, scientific literature and on-field evaluation of non-HemoCue systems are extremely limited or even lacking for some 22]. Lamhaut et al., included 44 participants undergoing potentially haemorrhagic surgery and they had a correlation of 0.85 between the HemoCue® and a reference analyser. The HemoCue® is technically more sophisticated than the devices and this similarity in results might mean that their simplicity does not compromise their technical accuracy [17]. Our results were lower than that obtained by Nkrumah et al., in their study of the HemoCue® in Ghana [22]. They had a correlation coefficient of 0.995. This difference can be explained by the fact that the HemoCue® unlike the two devices has greater calibration functionality.

We also observed good agreement between the two haemoglobin meters and the reference analyser in the measurement of haemoglobin as shown on the Bland-Altman graphs. These results were better than those obtained by Schapkaitz et al., in their study of the HemoCue® in South Africa [12]. Using a sample of 100 participants of all age groups, they had only 93% of the difference plots within the 95% limits of agreement (− 3.9, 5.5) and their limits of agreement were wider than those observed in our study. This difference can be explained by the fact that they used a smaller sample size than we did.

We also observed that the overestimated mean haemoglobin values, when compared to the reference analyser were statistically significant in both devices. However, only the MissionHb® meter had a mean haemoglobin significantly higher than the clinically allowable range. This implies that we could potentially fail to identify patients with borderline anaemia in some instances and provide adequate treatment with the use of these devices. Nevertheless, it is worth noting that both meters were overall excellent at identifying the presence of anaemia given their significantly high areas under the receiver operating curves. The ability of a haemoglobin meter to detect the presence or not of anaemia is very important in clinical settings, but its ability to determine the severity of the anaemia is even more important as the management of anaemia is entirely depending on its severity. Both meters had weak agreement with the reference analyser in determining the severity of the anaemia among our participants when we adopt a more logical interpretation of Cohen’s kappa statistic [20]. The respective levels of overestimation of haemoglobin by both meters might be an underlying reason for this finding. A weak level of agreement in diagnosing the severity of anaemia between both meters and the reference analyser, implies the severity of anaemia in several patients could be incorrectly assessed when using these meters and consequently lead to undue exposure of some patients to blood transfusions when not absolutely required. As such extreme clinical judgment would be imperative with the use of these devices.

Our study may be limited by a lack of a large representation of patients in the different ranges of anaemia (mild, moderate and severe). However, our study focused on hand-held haemoglobin meters thus adding to the extremely limited or even lacking literature of non-HemoCue systems. In addition, the results of our study may not be readily translated to all populations world-wide where the causes of anaemia may be very different from those of our study population (usually iron deficiency anemia). On the other hand, our study seems to be the first to assess the clinical implications of error in haemoglobin measurement using point-of-care haemoglobin meters.

Conclusion

In conclusion, MissionHb® and Urit12® had good technical accuracy when compared to the reference analyser. However, the MissionHb® meter failed to meet clinical accuracy requirements, and both meters overestimated haemoglobin values. Even though both meters were excellent at detecting the presence of anaemia, they both had weak agreement with the reference analyser in determining the severity of the anaemia. Our findings suggest that although the use of these devices in resource-limited settings might offer enormous advantages compared to a reference analyser, extreme caution and clinical judgment are imperative to complement their use.