Introduction

Helicobacter pylori virulent strains being predominant in several regions thus H. pylori should always be treated regardless of the presence of symptoms [1]. However, antibiotic resistance caused in Helicobacter pylori eradication failure and the resistance rates widely vary around the world [2,3,4]. Recent hospital-based study in Japan reporting 48% of clarithromycin resistance [5] while our previous study in Indonesia reported a 9% resistant rate [6]. Because of such differences, the Maastricht Consensus Report on H. pylori infection has stated that antibiotic susceptibility tests with periodic monitoring should be performed in each region to determine the most suitable therapy for a given population [2].

Given the importance of antibiotic susceptibility testing for H. pylori, it is crucial to choose a testing method that delivers high accuracy and feasible for clinical settings. According to the Clinical and Laboratory Standards Institute (CLSI), direct measurement with the agar dilution method (ADM) is the gold standard for H. pylori [7]. However, ADM requires laborious preparation and may not cost and time-effective for daily clinical practices [8]. An alternative, relatively simple method is the E-test, which uses different concentrations of antibiotics in a single strip.

In the present study, we followed CLSI guideline EP-09 to compare the measurement procedures and estimate bias [9]. These methods have been applied in other studies, but rarely for H. pylori [10,11,12,13]. Thus, a study using isolates from the Indonesian population, an area with the variable prevalence and virulence type of H. pylori [6, 14] may provide another insight of the E-test reliability.

This study aimed to examine the susceptibility of Indonesian H. pylori isolates to five antibiotics (amoxicillin, clarithromycin, metronidazole, tetracycline, and levofloxacin), using ADM as the gold standard to validate the level of agreement and reliability of the E-test.

Main text

Methods

Patients and H. pylori isolates

We analyzed a total of 72 clinical H. pylori isolates obtained from adult dyspeptic patients as part of a nationwide survey in Indonesia and reference strain ATCC26695. Data for the E-test results for the 72 strains were published in our previous study [6]. Isolate storage and all the susceptibility tests were conducted at the Department of Environment and Preventive Medicine, Oita University Faculty of Medicine, Yufu, Japan.

All the patients from whom the isolates were obtained provided written informed consent, and the study protocol was approved by the Institutional Review Board or Ethics Committee of Dr. Cipto Mangunkusumo Teaching Hospital (Jakarta, Indonesia), Dr. Soetomo Teaching Hospital (Surabaya, Indonesia), Dr. Wahidin Sudirohusodo Teaching Hospital (Makassar, Indonesia), and Oita University Faculty of Medicine (Yufu, Japan).

Antibiotic susceptibility testing by ADM and E-test

The procedure for ADM followed the protocol described by CLSI [7, 15]. Briefly, around 2 µL of bacterial suspension with 0.5 McFarland concentration was inoculated into Mueller–Hinton agar contained twofold dilutions of antibiotics. E-test (bióMeurieux, La Balme-Les-Grottes, France) procedure was reported previously, following manufacturer instructions [6]. Briefly, 100 µL of H. pylori suspension with 3 McFarland standard was inoculated into Mueller–Hinton agar plate without antibiotic and one E-test strip were applied to the center of the plate. incubation under microaerophilic conditions for 72 h. A full explanation of the method is available in the Additional file 1: Fig. S1.

Statistical analysis

The ADM is the gold standard for antibiotic susceptibility test, thus the results were used as the reference for validating the E-test method. The samples were grouped into “sensitive” and “resistant” according to the EUCAST clinical breakpoint criteria [16] and were then used to evaluate, essential agreement, Cohen’s kappa analysis, McNemar, also the major and very major error rate between the two methods. The analysis was performed using the SPSS statistical software version 23.0 (IBM Corp., Armonk, NY, USA).

To better understand the agreement of the MIC results, we followed CLSI guideline EP-09 to use the non-parametric approach proposed by Bland and Altman with Krouwer modification because the gold standard is available [17, 18]. Scatter plot and Passing–Bablok analyses were also performed using R environment ver. 3.5.1 with the mcr package [19]. Receiver operating characteristic analysis was used to evaluate the sensitivity, specificity, and area under the curve (AUC) of the E-test results relative to those of the ADM.

Results

Susceptibility results of the ADM and E-test

We initially attempted to culture 77 strains of H. pylori; however, five strains did not grow and were excluded. This growth failure number was still acceptable according to the U.S. Food and Drug Administration (FDA) recommendation for antimicrobial susceptibility testing [20]. Both the ADM and E-test were applied to the same 72 remaining strains.

In Table 1, both ADM and the E-test showed high resistance rates to metronidazole and levofloxacin. Slightly higher resistant rates were shown by the E-test for four antibiotics, but this difference was not statistically significant (McNemar test: amoxicillin, P = 0.99; metronidazole, P = 0.55; clarithromycin, P = 0.63; and levofloxacin, P = 0.99; the value for tetracycline could not be calculated because of the zero percentage result). MIC50 and MIC90 were the same for all the antibiotics except amoxicillin and tetracycline.

Table 1 Resistance rates according to the agar dilution method (ADM) and E-test
Full size table

Agreement of the susceptibility results

To determine the precision and reliability of the E-test. Differences in the “sensitive” and “resistant” interpretations were further analyzed according to the agreement percentage and Cohen’s kappa analysis (Table 1). The kappa values showed fair agreement for amoxicillin, moderate for clarithromycin, and substantial agreement for metronidazole and levofloxacin prevalence by ADM. The kappa value for tetracycline could not be evaluated because of the zero value for prevalence.

Measurement of errors

Figure 1 shows the major errors and very major errors, as described in previous studies [13, 21, 22]. For all the antibiotics, the very major error rate was lower than the major error rate. Despite its low resistance rate according to both the ADM and E-test (4.2 and 5.6%, respectively), amoxicillin was associated with a very major error rate of 2.78% and a major error rate of 4.17%. Metronidazole also had a high very major error and major error rates (5.56 and 9.72%, respectively).

Fig. 1
figure 1

The major error and very major error rates in categorical agreement. A very major error was defined as when a strain that was resistant according to ADM was assessed as “sensitive” by the E-test; this may result in the patient not receiving effective antibiotics, allowing the infection to continue. A major error was defined as when the strain was sensitive according to ADM but assessed as “resistant” by the E-test. True sensitive is sensitive percentage minus very major error rate while the true resistant is resistant percentage minus major error rate

Full size image

Analysis of agreement and systematic bias

The modified Bland–Altman difference plots [17, 18] to assess the limit of agreement and any pattern of bias are shown in Fig. 2. For all the antibiotics except tetracycline, the median difference in measured MIC was zero, indicating there was no constant bias between the E-test and ADM. At low MIC values, the differences in the plots tended to clump around the median value, indicating precision between the methods. The error tended to increase at higher values of MIC for all the antibiotics, shown by the greater difference values. Spearman correlation analysis confirmed this correlation was significant for amoxicillin (r = 0.53, P ≤ 0.001), clarithromycin (r = 0.56, P ≤ 0.001), and tetracycline (r = 0.76, P ≤ 0.001).

Fig. 2
figure 2

Bland–Altman difference plots comparing the minimum inhibitory concentration (MIC) results of the agar dilution method and E-test for the five antibiotics. The plots use the modification of Krouwer [17, 18]. The plots confirm that the measurements by both methods were equal for all the antibiotics, except for tetracycline. For tetracycline, the median difference was 0.061 mg/L

Full size image

As recommended by the CLSI, Passing–Bablok regression analysis was used to estimate the analytical method agreement and detecting the presence of any bias. It can determine whether the E-test procedure and agar dilution method can be used interchangeably. In the Passing–Bablok regression plots (Additional file 1: Fig. S1), the confidence intervals for the intercepts for all the antibiotics except amoxicillin and tetracycline included zero, indicating there were no significant proportional differences between the values measured by the two methods. The confidence intervals for the slopes included the value 1 (indicating no significant difference between the measurements) for all the antibiotics except amoxicillin and tetracycline.

Accuracy of the E-test

The sensitivity, specificity, and AUC of the E-test for distinguishing the strains resistant to each antibiotic were calculated. For amoxicillin resistance, the E-test showed low sensitivity but high specificity (33.3 and 95.7%), with a good AUC value (0.783). For clarithromycin, the sensitivity was also low (66.7%) but the specificity and AUC were high (95.7 and 0.937%, respectively). For metronidazole and levofloxacin, the E-test showed quite high sensitivity (86.7 and 88.9%, respectively), specificity (83.3 and 94.4%), and AUCs (0.887 and 0.919). It was not possible to make these calculations for tetracycline because ADM showed the resistance rate was zero.

Discussion

This is the first study to evaluate the validity of the E-test as an alternative method for the detection of resistant strains of H. pylori in Indonesia. E-test is preferable in clinical practice due to the lower cost and less time consuming [8]. Several previous studies have validated the E-test for European populations [21,22,23], Brazilian children [13] and American [24].

In this study, we found that the resistance rate obtained from E-test showed a slightly higher discrepancy compared to ADM, although this was statistically not significant according to McNemar analysis; similar to previous studies [23, 24]. For levofloxacin, metronidazole, and clarithromycin, the essential agreement was in the acceptable range according to the FDA (> 90%) with low major error and very major error rates [20, 25]. Agreement for MIC values with Bland–Altman and Passing-Bablok also showed that both methods can be used interchangeably with good sensitivity and specificity [26, 27].

However, consistent with other studies, metronidazole showed a low essential agreement and high very major error rate, but metronidazole resistance is common and is not a first-line treatment so its use is limited [2, 6, 22, 28]. The emergence of heteroresistance to metronidazole within a mixed colony may have played an important role in this result [24]. For amoxicillin, Cohen’s kappa analysis only showed a fair agreement. Moreover, a non-significant correlation for Passing-Bablok and very low sensitivity might indicate the lower reliability of the E-test for measuring amoxicillin resistance, compared to other antibiotics. However, it also probably due to a very low prevalence of amoxicillin resistant thus limit the kappa analysis in estimating values, known as the “kappa paradox” [29]. Hence, amoxicillin is included in the first line regiment and the non-concordance that occurred may affect the treatment choice and patient’s outcome [2].

The reason for the discrepancy observed in the E-test results was still unknown but is probably the result of several factors. The procedure for storing E-test strips may affect the drug concentration, and incubation in a microaerophilic environment may influence the activity of the antibiotic, especially for macrolides [8, 23]. The large bacterial inoculum size was tested by an E-test strip while in ADM, small number of bacteria were inoculated on agar plates containing antibiotics. The difference in bacteria/antibiotics ratio may have resulted in higher growth capabilities of the strain, resulting in a higher measured resistance rate in the E-test than ADM. Indeed, it has previously been demonstrated that inoculum concentration has an impact on the discrepancies observed in microdilution and ADM [30].

In general, the E-test may overestimate the rates of resistance to antibiotics, but it may be applicable because it showed good agreement with ADM results for levofloxacin, metronidazole, clarithromycin, and tetracycline. However, the disagreement from amoxicillin may require further confirmation.

Limitations

One of the limitation of the present study was that no tests were performed in other laboratory centers to check the reproducibility of the methods. The isolates used in this study were collected in 2015 and stored in − 80 °C. Even though the freeze storage effect on antibiotic sensitivity in H. pylori was varied between the studies [31,32,33], it may explain the discrepancy of MIC found in this study. The zero resistance rate for tetracycline also affected the analysis. The number of study samples met the recommendation of CLSI; however, a greater number may have improved the precision of the comparison. Nevertheless, the findings of this study provide the data that E-test was a simple but reliable diagnostic tool for antibiotic-resistant detection in H. pylori.