Introduction

Exhaled nitric oxide (NO) originates primarily in bronchial epithelial cells. NO is an excellent surrogate marker of inhaled corticosteroid (ICS)-sensitive local type-2 cytokine-driven inflammatory mechanisms, which are of central importance in allergic airway inflammation [1–4].

The detection of NO in exhaled air was first reported in 1991 [5], and was later shown to be elevated in exhaled air from patients with asthma [6]. Measuring the fraction of exhaled NO (FeNO) offers the only simple, non-invasive and objective method of assessing local airway inflammation in asthma [1, 7, 8]. FeNO is now an established biomarker for airway inflammation and is strongly recommended by the American Thoracic Society (ATS) for the diagnosis and management of asthma [9].

FeNO is easily measured in a clinical setting using handheld or stationary analyzers [10]. The ideal analyzer is portable and easy to use with good reproducibility and reliability regardless of patient baseline FeNO, age, or operator expertise [11]. NIOX VERO® and NIOX MINO® are two such handheld electrochemical analyzers, which enable rapid and cost-effective assessment of airway inflammation in asthma at the point of care. Furthermore, the testing is easy to perform and requires minimal training for the operator to use. The assessment cannot be influenced by patient effort or variations in the clinician’s technique. NIOX MINO® and NIOX VERO® do not require calibration at the clinic, while built-in controls and external quality control procedures warrant reliability of measured values. Both devices can be used by patients aged 4 years and older in the European Union (EU) and patients aged 7 years and older in the United States (US). NIOX VERO® contains significant improvements learned from years of making and using electrochemical sensors for the measurement of exhaled nitric oxide. When compared with NIOX MINO®, enhancements to NIOX VERO® include its use without being connected to a power supply and improved visibility of the display screen. In addition, NIOX VERO® has an ergonomic handle to aid portability.

In this paper, we describe the outcomes of four studies performed to validate the NIOX VERO® analyzer.

Methods

Agreement between NIOX MINO® and NIOX VERO® was investigated in two studies, AER-045 and AER-048 (AER-045/048). Repeatability of FeNO measured with NIOX VERO® using different operators was also evaluated across two studies, TV-014 and TV-018 (TV-014/018). AER-045 and AER-048 share the same methodology, as do TV-014 and TV-018 with the exception that AER-048 and TV-018 also aimed to enrol a proportion of patients with higher FeNO (>40 ppb). Since the second studies are extensions of the first two, only the pooled data from each pair of studies are presented in this article.

The studies were approved by the local Institutional Review Board and were conducted in compliance with Good Clinical Practice and the Declaration of Helsinki. All subjects provided written consent, or legal guardians provided written assent, prior to any study procedures being performed. In each study, subjects attended the clinic once for study-specific assessments.

Study Design

AER-045/048 were multi-center, open-label, randomized, single-visit, controlled studies to compare FeNO measured with NIOX VERO® and NIOX MINO®. TV-014/018 comprised multi-center, randomized, single-visit, point-of-care studies to assess inter-operator variability of NIOX VERO®. Further details are given in the online supplement.

Participants

Subjects

Males and females were enrolled aged 7 years or over with physician-diagnosed asthma [or were under evaluation for asthma (AER-045)] and able to perform FeNO measurements using the study analyzer(s). Subjects who did not refrain from taking food, drink, nicotine or strenuous exercise within 1 h of the FeNO assessments or who had any other pulmonary disease or acute respiratory illness or sinusitis were excluded. For AER-045, subjects were excluded with severely impaired lung function (predicted FEV1 <50%), diagnosed lung cancer or other active pulmonary infectious disease, including tuberculosis.

For AER-048 only, subjects were included if they had risk factors for elevated FeNO (>50 ppb) or they had current respiratory symptoms such as cough, wheeze, and/or dyspnea or a history of non-compliance with prescribed asthma treatment. In addition, subjects were only included in the AER-048 if they had a FeNO of >40 ppb on the study day.

Operators

Operators were male and female physicians, nurses, respiratory therapists or laboratory technicians already familiar with NIOX MINO® (TV-014/018).

Devices

NIOX VERO® (Aerocrine, Sweden) was under development at the time of the study in the US and was subsequently cleared by the FDA in November 2014. The analyzer was used according to the Investigator User Manual [12]. NIOX MINO® (Aerocrine) is CE marked and cleared for clinical use in the US by the FDA, and was used according to the User Manual [13].

Assessments

In all studies, FeNO measurements were obtained using a standard technique—single breath online, 10-s exhalation in accordance with current recommendations [14]. If a valid FeNO assessment was not achieved after 10 attempts, the measurement was considered unsuccessful and the subject was discharged from the study. Data from all valid FeNO measurements for all subjects were analyzed. In AER-045/048, subjects attempted a total of two valid exhalations (maximum of six exhalations) at a fixed flow rate (50 mL/s) with each device. The order in which the devices were used was randomized. An evaluable subject successfully completed four valid FeNO measurements, two with each device, within approximately 30 min.

Adverse events (AEs) and serious injuries were assessed from the time of informed consent/assent and during the study visit for each subject (not for operators). Further details are provided in the online supplement.

Statistical Methods

Statistical Analyses

Two-sided tests were used for all analyses; 95% confidence intervals (CI) were calculated and a p value ≤0.05 was considered statistically significant. Statistical analyses were processed using SAS v.9.2.

AER-045/048

Agreement between the first valid FeNO measurements from the two devices was confirmed if the lower limit of the one-sided 95% CI of the proportion of patients within the tolerance limits was ≥0.90. Differences of <10 ppb for absolute values <50 ppb and of <20% for those >50 ppb were considered within tolerance limits. A clinically significant change in FeNO is considered to be >10 ppb (or 20%) [9], so device differences within these limits were considered clinically non-significant. Normal approximation was used to calculate the CI when the observed proportion (p) and 1 − p was >5/n [15], otherwise the exact CI was calculated.

Agreement was also analyzed between first valid FeNO measurements from the two devices and evaluation of the mean differences between mean values from each device using Bland–Altman plots. Intra-individual values and mean values of FeNO measurements from each device were evaluated using 95% limits of agreement, Weighted Deming Regression analysis, and scatter plots with regression lines.

Repeatability was assessed by analyzing the differences between mean intra-subject standard deviation (SD) of duplicate FeNO measurements from each device using the Wilcoxon Signed Rank Test.

TV-014/018

Repeatability was measured using NIOX VERO® when three consecutive valid FeNO measurements were obtained from one subject by three different operators using the same device; assessment was based on NCCLS EP5-A2 guidance [16]. Intra-subject variation was obtained for each subject and the coefficient of variance (CV) was calculated. The intra-subject variance, CV and upper one-sided 95% CI were generated for each measure. Summaries were provided for subjects with mean FeNO <50 ppb and those with mean FeNO ≥50 ppb. In TV-018, enrollment was stratified for subjects with FeNO >40 ppb based on their first FeNO value.

Details of additional analyses and any changes in the planned analyses are provided in the online supplement.

Analysis Populations

In AER-045/048, efficacy analyses were performed using subjects with ≥1 valid measurement using NIOX MINO® and NIOX VERO® (Efficacy Subjects population) and the All Subjects population was used for all other analyses. In TV-014/018, subjects completing three FeNO measurements were included in the Efficacy Subjects population.

Sample Size

Ninety evaluable subjects were planned to give >80% power to demonstrate that the lower limit of a 90% CI was ≥0.90. This was based on the hypothesis that the true proportion within tolerance limits was 0.97 (AER-045), with an extra 20–30 subjects planned and stratified by baseline FeNO level to give 10 subjects with FeNO >40 and <50 ppb, and 10–20 subjects with FeNO ≥50 ppb (AER-048).

Between 80 and 90 subjects were planned, based on the expected precision for the estimated intra-subject variance for subjects with valid FeNO values to obtain approximately 20 subjects with FeNO values ≥50 ppb (TV-014); a minimum sample size of 20 was planned given the one-sided upper bound of the SD was 1.18 in order to establish relative variance estimates for subjects with FeNO >40 ppb (TV-018).

Approximately 40 operators were planned with the number of operators per site based on the number of planned subjects. Each set of 3 operators was allowed to make only one assessment together, no pair of operators conducted more than three assessments together, and no operator was allowed to make more than eight assessments.

Results

Participants

Subject demographics are presented in Table 1. Of the 112 subjects enrolled, 8 (7.1%) did not complete the AER045/048 studies: 4 (3.6%) did not complete the two approved exhalations using NIOX VERO® within the maximum number of attempts allowed; 1 (0.9%) experienced NIOX VERO® instrument failure; and 3 (2.7%) had a FeNO concentration <5 ppb. All subjects completed the TV-014/018 studies.

Table 1 Demographics, pooled for studies AER-045 and AER-048 and for studies TV-014 and TV-018
Full size table

In TV-014/018, 54 operators were employed, all female, with a mean (SD) age of 48.5 (13.85) years with the largest proportion trained as registered nurses (46.3%), or as licensed practical nurses/associated nursing degree (20.4%) (Supplementary Table 1).

No AEs, serious injuries, issues or problems were reported with use of either device.

Further details are provided in the online supplement.

Device Comparisons

Agreement

The primary objective of the study, showing that the 95% CI for the proportion of subjections within the tolerance limits was above 0.90, was not met when comparing NIOX MINO® and NIOX VERO® (Table 2). The mean overall observed paired difference (SD) between the devices in relation to the mean first valid FeNO was −4.6 (6.45) ppb equating to a mean percent difference of −13.2% (33.95). The mean observed paired difference in average results (NIOX VERO® – NIOX MINO®) was −4.6 ppb (95% CI −5.825 to −3.377; p < 0.0001) (Fig. 1). The slope of the Weighted Deming Regression Analysis plot of first valid FeNO values was 0.842 (95% CI 0.757, 0.927) with a y-intercept of −0.472 ppb (95% CI −1.999, 1.055) (Fig. 2).

Table 2 Summary of the number and proportion of subjects within the tolerance limits for first valid and mean FeNO measurement; pooled data for studies AER-045 and AER-048 (Efficacy Subjects)
Full size table
Fig. 1
figure 1

Bland Altman Plot results for the first valid FeNO measurement on each device (NIOX VERO® and NIOX MINO® showing pooled study data from studies AER-045 and AER-048 (Efficacy Subjects)

Full size image
Fig. 2
figure 2

Observed results for first valid FeNO measurement on both devices (NIOX VERO®and NIOX MINO®) for pooled study results from study AER-045 and AER-048 (Efficacy Subjects)

Full size image

Evaluation of the mean differences in mean FeNO values obtained from each device gave a mean observed difference in average results (NIOX VERO® – NIOX MINO®) of −4.5 ppb (95% CI −5.635 to −3.449; p < 0.0001). Similar results were seen with the mean of the average FeNO measurement on each device: the Weighted Deming Regression analysis slope was 0.929 (95% CI 0.854, 1.033) and y-intercept was −2.418 ppb (95% CI −3.861, −0.974). FeNO measurements using NIOX VERO® were slightly lower than those using NIOX MINO® by approximately 3 ppb, which remained within the technical specifications of the devices (±5 ppb).

Reproducibility

Intra-subject repeatability was demonstrated for both devices, with significantly better repeatability achieved with NIOX VERO® than NIOX MINO® (median 0.25 vs, 1.00, respectively; p = 0.0112) (Table 3).

Table 3 Intra-subject variance for efficacy subjects with two valid FeNO measurements on each device to show repeatability (Efficacy Subjects)
Full size table

Inter-operator Repeatability

Inter-operator repeatability was achieved using the NIOX VERO®. Pooled mean intra-subject variance was 6.61 ± 17.954 ppb (upper 95% CI 9.41) with an estimated SD of 2.57 (upper 95% CI 3.07). The CV was 0.066 ± 0.054 (upper 95% CI 0.074) (Fig. 3a, b; Supplementary Table 2).

Fig. 3
figure 3

a Individual FeNO measurements versus subject mean FeNO for pooled study data from studies TV-014 and TV-018; b individual FeNO measurements: subject mean FeNO for pooled study data from studies TV-014 and TV-018 (Efficacy Subjects). Both figures provide evidence for inter-operator repeatability using NIOX VERO®

Full size image

There was no observed operator-order effect on the FeNO results. Post hoc repeated analysis of variance (ANOVA) showed mean FeNO values were similar between all three operators, but, due to the large sample size, small but statistically significant differences were noted (p < 0.05) (Table 4; Supplementary Fig. 1a–c).

Table 4 FeNO results by operator order and site by mean subject FeNO value for pooled study data from studies TV-014 and TV-018 (Efficacy Subjects)
Full size table

Post hoc Weighted Deming Regression on the three pairs of observations gave an estimated bias <2% for all pairs (estimated percent bias close to 1%; estimated average bias between −1.00 and 1.20% depending on the pairs assessed).

Categorical FeNO values were consistent and repeatable for all subjects. For FeNO <50 ppb, mean intra-subject variance was 1.37 ± 2.136 (upper 95% CI 1.77), with an estimated SD of 1.17 (upper 95% CI 1.33) and CV of 0.072 ± 0.058 (upper 95% CI 0.082). For FeNO ≥50 ppb, mean intra-subject variance was 21.97 ± 30.976 (upper 95% CI 31.64), with an estimated SD of 4.69 (upper 95% CI 5.62) and CV of 0.048 ± 0.036 (upper 95% CI 0.060).

Discussion

In our studies, FeNO values were obtained using two electrochemical exhaled NO analyzers, NIOX MINO® and NIOX VERO®. The subject population included adults and children with asthma and was representative of asthma patients in the US and EU [17]. These were the first studies to compare NIOX VERO® with NIOX MINO® and showed a clinically acceptable agreement between the devices with a significant improvement in reproducibility with NIOX VERO® compared to the reference device. While there was a statistically significant difference between the two analyzers, the differences were small and are unlikely to be of clinical significance in the care of patients. The same agreement was seen when comparing the first valid measurement and the mean of two measurements. Although FeNO measurements using the NIOX VERO® were slightly lower than those using the NIOX MINO® (by approximately 3 ppb) they were highly correlated and the difference was within the technical specification of the instrument (±5 ppb).

To date, NIOX MINO® has been compared with three chemiluminescence NO analyzers, the NIOX®, Ecomedics and Sievers NOA280i [18–30]. All these studies showed agreement between the FeNO values obtained with the devices being studied, with a clinically acceptable level of agreement shown with NIOX [19–22, 25, 26] and Ecomedics [29]. NICE have also reviewed evidence from studies comparing NIOX MINO® and NIOX VERO® with chemiluminescence analyzers to establish whether these devices could be considered equivalent [31]. Some differences were observed, and equivalence seemed poorer under certain circumstances, such as in patients with higher FeNO levels. However, there was generally a good correlation seen for NIOX VERO® and NIOX MINO® with other chemiluminescence analyzers so that these devices are considered broadly equivalent [31].

Intra-subject repeatability was acceptable for both devices with significantly better repeatability shown for NIOX VERO® than NIOX MINO®. FeNO measurements using NIOX VERO® were reproducible, consistent and well within the technical specifications of the device. Our findings correlate with previous observations with NIOX MINO®; other studies in adults and children have reported excellent reproducibility when using this device [11, 21, 22, 25, 29, 30, 32, 33]. However, poor reproducibility was seen in one study of NIOX MINO® specifically in children and teenagers, but only when comparing absolute FeNO values—reproducibility was acceptable when FeNO values were categorized as low, normal, intermediate or high [19].

Our findings showed no observable pattern of a training effect or order effect on FeNO results when measured three times using NIOX VERO® by three different operators. These results support the feasibility of using NIOX VERO® at the point of care and are consistent with those seen using NIOX MINO®. Gill et al. [11] showed close agreement between replicate operator measures for NIOX MINO®; mean within-subject SD in 17 subjects with FeNO <30 ppb was 1.15 (95% CI 0.67–1.53) which was below the set limit of 3 ppb. In the three subjects with FeNO >30 ppb, the mean within-subject coefficient of variation was 2.4 and below the hypothesis limit of 10 [11].

How did higher FeNO affect our results? In TV-014/018, increased variability between measurements was seen as the average subject FeNO value increased, but the CV appeared to remain relatively constant across the range of measured values. Furthermore, values were consistently within 20% of their overall mean and all pairs were within 10 ppb for values with an overall mean <50 ppb. With regards to statistical considerations, there was no evidence of bias across the three operator assessments as confirmed by Weighted Deming Regression and ANOVA. Variability did increase in time and a few pairs of high values differed by >20%, although in these cases it would not have affected the ≥50 ppb assessment, and the CV was similar among the data cuts considered. Further, across all studies, NIOX MINO® and NIOX VERO® were shown to be safe in subjects with high FeNO.

Study Strengths and Limitations

These were one-visit studies that were simply designed in order to evaluate technical validation of the devices. More visits over time would have demonstrated how NIOX VERO® can measure changes due to both inter-operator and biological variability.

Conclusion

Our findings have shown that the NIOX MINO® and new NIOX VERO® analyzers are clinically equivalent when used in an asthma population representative of that seen in the US and EU. Furthermore, we have shown that FeNO measurements taken using the NIOX VERO® analyzer are reproducible, consistent and well within the technical specifications of the device, showing no observable pattern of a training effect or operator-order effect on FeNO results. Thus, our findings validate the portable NIOX VERO®, enabling its clinical use in both children and adults.