Background

Recently we reported the validation of the “Allergy-Control-SCORE© (ACS)” which includes three categories: lung, nose and eyes [1]. Now we report the validation of the score for eyes and nose, only. With this new score we suggest an approach which covers a symptom score, and a medication score, for eyes and nose symptoms and use of symptomatic allergy medication to a combined symptom-medication score (SMS), the Rhino-Conjunctivitis Allergy-Control-Score (RC-ACS©). Such a SMS is recommended to measure the primary outcome of clinical trials on respiratory allergies [2], and the use is proposed by international regulatory agencies, e.g. European Medicines Agency (EMA) [3].

With this validation we also report the outcome of the validation of the Eye-Allergy-Control-Score (E-ACS©) and the Nose-Allergy-Control-Score (N-ACS©). The Allergy-Control-Score (ACS©). was used for several years in different clinical trials [4–6]. In these studies the score was used under the synonym, symptom medication score. The E-ACS© and the N-ACS© as well as the RC-ACS© are parts of the Allergy-Control-SCORE©. The RC-ACS© covers drugs used in clinical trials and observational studies.

The concept of validation of the RC-ACS© is based on measuring 1) convergent reliability and 2) retest reliability in allergic patients as well as 3) discrimination capacity in healthy controls and patients with respiratory allergies. Therefore a prospective study was designed which included patients suffering from allergic rhinitis, conjunctivitis or both and healthy controls. The convergent reliability was assessed to determine the degree to which the scores of an instrument show a relationship to scores of similar instruments. For this purpose the correlation to the following two instruments and two clinically important disease related health economic measures was assessed: 1) Global Assessment of Severity of Allergy by use of a Rating Scale; 2) Quality of Life (RQLQ); 3) number of medical consultations due to the allergy within the last 12 months; 4) the number of non-productive days due to allergic rhino-conjunctivitis and/or asthma within the last 12 months. Retest reliability is the extent to which scores for patients who have not changed are the same for repeated measurement over time. The retest reliability was determined by correlating the SMS values of the first week with the values of the second week. The discrimination capacity reflects the degree to which the scores of an instrument can discriminate between different patient groups. Discrimination capacity was assessed by comparing the average SMS values of the allergic patients (first week) and the control group, respectively.

The validation of a Rhino-Conjunctivitis Allergy-Control-Score (RC-ACS©) is an important and topical issue in allergy clinical research. It is a relevant instrument to assess rhino-conjunctivitis severity in clinical trials and observational studies. With the validation a formal aspect for reliable use of such an instrument is fulfilled.

This Rhino-Conjunctivitis Allergy-Control-Score (RC-ACS©) introduces the concept of “control of disease (rhino-conjunctivitis)” as this will be the aim when using any therapeutic intervention. The presented score balances symptoms and use of medication and it also considers the influence of treatment on allergic symptoms. Therefore, for calculating the combined symptom medication score, each medication will be assessed e.g. according to the use, the specific effects and the way of administration.

Methods

Study population

Patients (age 19 to 65 years) were recruited from the outpatient clinic department of “Dermatologikum Hamburg”, Germany, between 21 June and 17 August 2008. Inclusion criteria were: 1) atopic sensitization (SPT positivity to at least one of the following allergens: grass, rye, mugwort pollen, house dust mites) (Allergopharma J. Ganzer KG, Reinbek, Germany); 2) current clinical manifestations of allergic rhino-conjunctivitis and/or asthma due to exposure to one of the four allergens listed above; 3) expected natural exposure to the relevant allergens during the study period. Controls were non-atopic volunteers with a negative history for IgE-mediated allergies. The following exclusion criteria were applied to patients and controls: 1) current use of systemic or nasal corticosteroids, inhaled corticosteroids (>400μg budesonide or >500μg beclomethasone dipropionate per day); 2) long-term prophylactic use of anti-allergic medication with constant dose; 3) current treatment with specific immunotherapy; 4) food allergy; 5) clinically-relevant rhinitis/rhino-conjunctival or respiratory symptoms related to other unidentified causes; 6) vasomotor, drug-induced or other kinds of non-allergic rhinitis/rhino-conjunctivitis; 7) febrile infections or inflammation of the respiratory tract; 8) irreversible secondary alterations of the upper and lower airways (e.g. emphysema, bronchiectasis etc.). The study protocol was discussed with the local Ethics Committee before it was commenced. The committee advised that formal approval was not required, because the study was observational and no changes in treatment were involved. However, written informed consent was received from all patients involved before they were included into the study.

Study design

The study was designed as prospective, observational and controlled study. Patients and healthy controls completed a questionnaire on demographic and clinical parameters at recruitment. Scores of individual symptoms and individual medications were documented daily during the pollen season over a period of 2 weeks in patients and 1 week in healthy controls, respectively. On each day, patients and controls were also asked to complete a self-administered questionnaire for a “Global Assessment of Severity of Allergy”. This was performed through a Visual Rating Scale (rating scale ranging from 1 = no symptoms at all to 10 = very severe symptoms), which is similar to the one described by Bousquet and colleagues [7]. At the end of the first week, patients completed a validated questionnaire to rate their quality of life (RQLQ©) [8]. Participants started at different times during the season, so that both study groups included subjects that were exposed to high or low pollen counts. In addition, patients were asked how many days they were incapable of working due to their allergic rhino-conjunctivitis and/or asthma within the last 12 months (number of non-productive day).

RC-ACS©

Symptom score

The characteristics of the Rhino-Conjunctivitis Allergy-Control-SCORE© as listed in Table 1 were elaborated according to the GA(2)LEN taskforce Guidelines [9]. The symptom score is recorded using diaries in which subjects documented daily the severity of various allergy symptoms scaled according to the EMA guidelines [3] on a scale ranging from 0 to 3: 0 = absent (no sign/symptom evident); 1 = mild (sign/symptom clearly present, but minimal awareness; easily tolerated); 2 = moderate (definite awareness of sign/symptom that is bothersome, but tolerable); 3 = severe (sign/symptom that is hard to tolerate; causes interference with activities of daily living and/or sleeping). For each day, the sum of the values of the seven allergy symptoms is calculated. These include ocular (itching, tear flow, redness), and nasal (sneezing, itching, running, blockage) symptoms.

Table 1 Characteristics of the Rhino-conjunctivitis allergy control score (RC-ACS)©, according to the GA2LEN recommendations[8]
Full size table

Medication score

Patients also have to document the allergy medication needed. All allergy medications for treating related symptoms are scored for each patient and each available day. Categories of medication taken into consideration include nasal and ocular anti-histamines and glucocorticoids, nasal decongestants, nasal cromoglycate acid and salts, systemic antihistamines, glucocorticoids and their combinations, leukotriene receptor antagonists. Drugs not foreseen by international Guidelines for treating allergic rhino-conjunctivitis are not included (e.g. anti-IgE). The total number of “score points” (SP) for symptoms on one day is 21 (i.e. each of the 7 symptoms scored with a maximum of 3). The maximum SP that can be achieved by intake of medication is also set to 21 SP, subdivided into the two sub-scores for nose (max. 12 SP) and eyes (max. 9 SP). Each drug is scored considering pharmacological action (according to the corresponding ATC code), expected impact on symptoms, route of administration, the dose taken and duration of effect. Each medication score is balanced for the respective weight on symptoms and within the maximum score of each organ system. Thus scoring of medication cannot yield a higher value than symptoms at the respective organ. An example for the scoring of medications is given in Table 2. In this example a patient had a combined intake of glucocorticoid-containing nasal spray, antihistaminic eye drops and nasal spray as well as systemic antihistamine. The most potent drugs are scored first; in this case the local glucocorticoids are scored. In case the maximum score of the corresponding subscore is not reached, the score for topical or systemic antihistamines is added until the maximum score points are reached (nose: 12 SP eyes: 9 SP). In case both systemic antihistamines and local antihistamines are given, the systemic antihistamine e.g. loratadine is scored first. The sum of SP of topical, systemic antihistamines and combination-drugs containing antihistamine cannot exceed 7 SP for the subscore nose and 5 SP for the subscore eyes, respectively.

Table 2 Example for the calculation of the medication score (combination of topical and systemic drugs)
Full size table

Symptom-medication-score

The RC-ACS© is obtained by adding the daily medication score to the daily symptom score leading to a range of 0 to 42 SP. Similarly, the N-ACS© as well as the E-ACS© is calculated. The daily E-ACS© and N-ACS© range from 0 to 18 SP and 0 to 24 SP, respectively.

Pollen counts

Pollen counts (grasses, rye and mugwort) were derived from the European pollen information database (European Aeroallergen Network, Vienna, Austria) between 21 June and 17 August 2008 for the pollen traps in Lübeck and Reinbek. Pollen exposition was assessed using the 4-point scale (“None”, “Weak”, “Moderate”, “Strong”) according to the definition of the German Meteorological Service) for each week of assessment (Table 3).

Table 3 Assessment of pollen counts according to the definition of the German Meteorological Service
Full size table

Statistics

Descriptive analysis

Background and demographic characteristics of subjects are summarized for both groups. Continuous variables are displayed by sample size, mean, median, standard deviation and range. Discrete variables are shown with frequencies and percentages. Missing SMS values were replaced by linear interpolation if at most 25% of the values were missing. Regarding all other parameters, the last-observation-carried-forward (LOCF) method was applied. Data management and statistical analysis were performed using the statistical analysis program SPSS Version 15.0 (SPSS Inc., Chicago, USA).

Validation of the RC-ACS©

The dataset was analyzed to measure the convergent reliability, discrimination capacity, and retest reliability, as follows.

Convergent reliability

Convergent reliability is the degree to which the scores of an instrument show a relationship to scores of similar instruments. The convergent reliability of the RC-ACS© and the total N-ACS© as well as the total E-ACS© was tested by correlating the average SMS value of week 1 with the following four parameters: 1) Global Assessment of Severity of Allergy (Rating Scale 1–10); 2) Quality of Life (RQLQ); 3) number of medical consultations due to the allergy within the last 12 months; 4) the number of non-productive days due to allergic rhino-conjunctivitis and/or asthma within the last 12 months. Spearman’s rank correlations were calculated for each of the criteria 1 to 4. A significant positive correlation (p < 0.05, two-tailed) was considered as evidence for convergent validity. This corresponds in this study to a medium effect size (r ≥ 0.30) which can be regarded as a considerable correlation [10].

Discrimination capacity

The discrimination capacity is the degree to which the scores of an instrument can discriminate between different patient groups.

Discrimination capacity of the RC-ACS© and the total N-ACS© as well as the total E-ACS© was assessed by comparing the average SMS values of the allergic patients (first week) and the control group, respectively. Discrimination capacity was assumed to be good if the SMS value in the allergy group was significantly higher (p < 0.05, two-tailed) than the SMS value of the control group. Significance testing was performed with the Wilcoxon Rank Sum test. Sensitivity and specificity were analyzed by a ROC-curve.

Retest reliability

Retest Reliability is the extent to which scores for patients who have not changed are the same for repeated measurement over time. The retest reliability was determined for the patient group only, by correlating the SMS values of the first week with the values of the second week. Comparisons were conducted using Spearman’s rank correlations.

Results

Socio-demographic and clinical data

A total of 122 adults (82 allergic patients and 40 healthy controls) were screened for inclusion into the study. Of the 82 patients, 81 fulfilled the in/exclusion criteria and were included in the study and 80 completed the entire study. All 40 control subjects completed the study. The socio-demographic data of both groups are shown in Table 4 and the data on medical history of allergic diseases are given in Table 5 for the patient group.

Table 4 Socio-demographic data of the patient and control group
Full size table
Table 5 Medical history data of the patient group
Full size table

Severity of symptoms in allergic patients vs. healthy controls

The severity of symptoms in allergic patients vs. healthy controls is shown in Table 6. Clearly, patients had higher values than healthy controls. Controls had values close to the lowest possible values. Since healthy controls were non-allergic based on their medical history, symptom rating of the control patients is due to other factors than allergic symptoms.

Table 6 Severity of allergy in allergic patients vs. healthy controls
Full size table

Discrimination capacity of the RC-ACS©

The patient group showed a significantly higher mean RC-ACS© in comparison to the control group (Median: 3.7 range: 0; 14.1 vs. Median: 0 range: 0; 2.9; p < 0.001, two-sided Wilcoxon Rank Sum test) (Figure 1A and B). The same was seen also for the E-ACS© (patient group: Median: 0.9; range: 0; 5.1 vs. control group: Median: 0; range: 0; 1.0; p < 0.001, two-sided Wilcoxon Rank Sum test) and the N-ACS© (patient group: Median: 3.0; range: 0; 9.0 vs. control group: Median: 0; range: 0; 1.9; p < 0.001, two-sided Wilcoxon Rank Sum test). Thus, RC-ACS©, E-ACS© and N-ACS© separated excellently between patient and control group. The best possible SMS-cut-off-value with a sensitivity of 93.8% and a specificity of 92.5% was 0.786 (Figure 1C). In other words, 93.8% of the allergic patients had an RC-ACS© of 0.786 or higher and thus could be assigned correctly to the allergy group. The same analysis has been repeated separately for E-ACS© and N -ACS© and provided similar results (data not shown).

Figure 1
figure 1

Discrimination capacity of the SMS. Frequency distribution of the SMS among 81 allergic patients (A) and 40 healthy control subjects (B). Panel (C) shows a ROC curve of the discrimination power of patients vs. controls; the area under the curve is 0.9755; the best discrimination point is 0.786, corresponding to a sensitivity of 93.8% and a specificity of 92.5%.

Full size image

Convergent reliability of the RC-ACS©

A statistically significant (p < 0.0001, two-tailed) positive correlation was observed between the RC-ACS© and the Global Assessment of the Severity of Allergy, the Quality of Life, and the number of medical consultations due to the allergic disease, but not with the number of non-productive days due to allergic rhino-conjunctivitis and/or asthma within the last 12 months (Table 7). However, only seven patients (8.75%) had lost at least one productive day due to allergy.

Table 7 Correlation of the RC-ACS © , E-ACS © and N-ACS © with further assessment tools for the severity of allergy
Full size table

Re-test reproducibility of RC-ACS© over time

In the patient group, the average RC-ACS© values of week 1 and week 2 were almost identical (week 1 = 4.3 ± 3.0; week 2 = 4.0 ± 3.1 (Table 8)). The RC-ACS© values of the first and second week correlated significantly (r = 0.8134, p < 0.0001, two-tailed). The reproducibility of the E-ACS© and N-ACS© (Table 8) was comparable (E-ACS©, week 1: 1.2 ± 1.3 vs. week 2: 1.2 ± 1.3; r = 0.7716, p < 0.0001; N-ACS©, week 1: 3.0 ± 2.1 vs. week 2: 2.9 ± 2.2; r = 0.7990, p < 0.0001).

Table 8 Re-test reliability of RC-ACS © , E-ACS © and N-ACS ©
Full size table

Pollen counts

The pollen counts of Poaceae in the two aerobiology stations were monitored during an eight-week period (Figure 2). Patients recorded their symptoms in a diary during this period.

Figure 2
figure 2

Pollen exposure during the study period. Pollen counts in two sites representative for this region during the whole study period.

Full size image

Discussion

Major findings

This study evaluated the validity of the RC-ACS©, a symptom-medication score which assesses severity of nasal and ocular allergy by considering symptoms and intake of anti-allergic medication. RC-ACS© is reliable, reproducible, and feasible because: 1) the convergent reliability analysis showed highly significant correlations with Global Assessment of Allergy Severity, Quality of Life, and the number of medical consultations due to allergy within the past year; 2) it discriminated significantly between patient and control groups (p < 0.001); 3) it showed good retest reliability; 4) it had an excellent sensitivity (93.8%) and specificity of (92.5%) in discriminating between patients and healthy controls. Thus, the RC-ACS© is considered as a reliable and valid instrument evaluating severity of symptoms of nasal and ocular allergies. The same holds also true for the elements of the RC-ACS© i.e. the N-ACS© and the E-ACS©.

Peculiarities of the RC-ACS©

RC- ACS© is based on scoring nose and ocular symptoms and use of symptomatic medications. It includes a full list of relevant drugs, i.e. without any limitation in drugs. Each score can be used separately or combined. Thus, RC-ACS© can be used in daily practice and real-life situations. Of note is the fact that, as with the ACS©, the RC-ACS© balances impact of symptoms and drugs. Although with this RC-ACS© a recommendation of the EMA guideline [3] can be met, it has to be considered that rhino-conjunctivitis can precede asthma [11] and often patients have both rhino-conjunctivitis and asthma [12]. Thus it is important to collect also lung symptoms in order to make assessments on development of asthma.

The combination of symptom and medication score

The RC-ACS© and the subscores E-ACS© and N-ACS© combine scoring of symptoms and medication by summing up both. For the symptom score and medication score separately, the retest reliability and discrimination power were excellent (data not shown). In contrast to the combined use of “Average Rhinoconjunctivitis Total Symptom Score” (ARTSS) and “Average Rescue Medication Score (ARMS)” [13] which has been specifically designed considering WAO recommendations [2], the RC-ACS© weights rescue medication, and balances rescue medication and symptoms. The advantage of the RC-ACS© is the individual scoring of each medication according to their ACT class, based on efficacy, mode of action, mode of administration, and duration of action.

Other instruments

A visual analog scale as used by Bousquet and colleagues [7] seems to be an easy to use instrument to assess efficacy of e.g. specific immunotherapy. However, nowadays a visual analog scale as used by Bousquet and colleagues [7] will no longer be accepted by health authorities to achieve approval for new medications especially if used for specific immunotherapy because the EMA guideline [3] states that “…the primary endpoint has to reflect both, symptom severity as well as the intake of rescue medication”. The present article does not claim to have a comprehensive discussion of all available instruments. For this purpose the authors would like to refer to a recently published review on such instruments by Pfaar and colleagues [14].

Cross-cultural validation

RC- ACS© is available in German, English, Polish and other languages. Translation to other languages may be possible. The Medication Score can be used worldwide because use of ATC codes guarantees that even country-specific therapies can be coded.

Conclusions

This study shows that the RC-ACS© is a valid and reliable diagnostic tool, for assessing, and monitoring allergy severity. It considers both, symptoms and allergy medication. The structure is robust enough for using it in clinical trials and daily clinical practice. Therefore, with this validated RC-ACS© there is a tool available which focuses only on rhino conjunctivitis. However, it has to be noted that the full picture of an allergic patient cannot be assessed without collecting data on lung function. With this paper we want to take the opportunity for opening a discussion on validation of such instruments and how it may be performed in the future and how it can be improved.