Introduction

Rheumatoid arthritis (RA) is an autoimmune disease characterised by pain, morning stiffness, joint swelling, deformity and functional impairments. Patients with RA have an increased risk of mortality and several adverse outcomes such as infections and cancer compared with those who do not have RA [1–4]. Several studies, however, suggest that complications in RA patients may not be attributable to the disease itself, but to the use of disease-modifying anti-rheumatic drugs (DMARD). For instance, tumour necrosis factor (TNF) α blocking agents have an association with specific types of infections and may be related to an excess risk of lymphomas and neurological complications [5–9]. Conventional DMARDs may also increase the incidence of lymphoma [10, 11].

In studies that seek to determine the relationship between drug therapy and adverse events, disease severity is an important confounder. That is, disease severity is known to increase the risk of many adverse events and is probably associated with a higher likelihood of receiving more immunomodulating DMARDs. Failure to adjust for such confounding by indication can create false associations between the exposure and study outcome [12].

Health care utilisation ('claims') data are routinely collected for insurance and are commonly used in health services research [13, 14]. Because adverse outcomes of RA are relatively rare, health care utilisation databases are an ideal source of information for studies of the relationship between DMARDs and adverse events such as cancer and infections. Thus, the development of an RA disease severity measure from claims merits high priority. We believe that health care claims data contain information such as physician visits, surgeries and laboratory tests that correlate with RA disease severity. Thus, to develop a claims-based severity index, we first created an RA medical records-based index of severity (RARBIS) from ratings by a Delphi panel on potential markers of RA severity commonly found in medical charts [15]. We then assessed the performance of the RARBIS in a cohort of Veteran's Administration (VA) patients and showed that the RARBIS correlated moderately well with RA treatment intensity and thus exhibited construct validity [16]. Next, we established the convergent validity of the RARBIS against a widely-used and accepted RA clinical measure, the Disease Activity Score (DAS28) [17]. The goal of the present study was to develop a claims-based severity index (claims-based index for RA severity [CIRAS]) using the previously validated RARBIS, not the DAS28. If validated as a measure of RA disease severity, the CIRAS may serve as a potentially important tool in adjusting for RA severity in pharmacoepidemiology studies of RA treatment and complications using health care utilisation data.

Materials and methods

Study population and data source

The study population consisted of 120 patients from the New England region of the VA Health System who had at least two recorded visits with a diagnosis of RA (International Classification of Disease-9-CM 714.0), at least two outpatient visits from hospitals within the New England VA Health System from July 1999 to June 2001 and had sufficient evidence of RA from their medical record. The VA maintains a comprehensive electronic medical records database containing information on demographic characteristics, surgical history, prescriptions, laboratory results, discharge summaries, radiology reports and progress notes. A review of the VA electronic medical records of the study population was conducted to obtain information on individual components of the RARBIS. The current study was approved by the VA Health System Human Subjects Committee.

RA records-based index of severity

A records-based index of severity was developed based on ratings from a Delphi panel of six New England board certified rheumatologists of potential indicators of RA severity [15]. The potential indicators were divided into the following categories: radiological and laboratory results; surgeries; extra-articular manifestations; clinical and functional status; and medications (see Table 1). Indicators that were ranked by the panel as having strong or very strong associations with RA severity and are typically found in medical charts were incorporated into the RARBIS. Sub-scales and individual components of the RARBIS were weighted according to how strongly they were regarded by the panel as being correlated with disease severity. Because we wanted to develop an administrative-based severity score that could be used to study drug-outcome relationships, we created the RARBIS with the option to exclude the medication sub-scale.

Table 1 Rheumatoid arthritis medical records-based index of severity
Full size table

Data on clinical status indicators (number of flares, physician global rating, functional and ambulatory status, presence of swollen joints, receipt of intra-articular and intramuscular injections, and hours of morning stiffness) and medication use from the VA medical records visit notes were collected for the chart review study period, 30 June 2000 to 30 June 2001. Data on medication use were derived from pharmacy records. We obtained information on surgical history (C1–C2 fusion and joint surgery), laboratory values (rheumatoid factor, erythrocyte sedimentation rate [ESR], C-reactive protein [CRP] and platelet counts), extra-articular manifestations (subcutaneous nodules and vasculitis) and X-rays (C1–C2 subluxation, erosions) from all available data in the medical record.

Potential health care utilisation data indicators of RA severity

We extracted the following information from the VA databases: rehabilitation visits (physical and occupational therapy), rheumatology visits, plain radiographs (hand, wrist, foot, ankle and cervical spine), extra-articular manifestations (pulmonary, soft tissue nodules, Felty's syndrome and Sjogren's syndrome), number of inflammatory marker (CRP and ESR) tests, number of platelet counts and chemistry panels ordered, rheumatoid factor testing, joint surgery (hand, wrist, knee, foot, ankle, elbow, cervical spine and shoulder) and DMARD use. The administrative study data period included both the one-year (1 July 1999 to 29 June 2000) and two-year (1 July 1 1998 to 29 June 2000) period before the one-year chart review study period.

Each physical therapy and occupational therapy visit was counted as a rehabilitation visit. Tests for CRP and ESR were aggregated into one category. Tests performed on the same day counted as separate tests. The number of hand, wrist, foot, ankle and cervical spine radiographs were also added together into one category. Three methods were used to count the number of prescriptions in a given year. First, we counted the total number of prescriptions (including repeat prescriptions) for the following 10 medications: auranofin, aurothioglucose, azathioprine, cyclosporine, etanercept (Enbrel, Amgen), hydroxychloroquine, infliximab (Remicade, Centocor), leflunomide, methotrexate and sulfasalazine (adalimumab, abatacept and rituximab were not yet available for RA). For the second method, prescriptions for each DMARD were counted once and added to obtain the total number of different DMARDs. For the third method, synthetic DMARDs and biological DMARDs were counted separately. Prescription for each type of DMARD was counted only once and then added together to obtain the total number of different synthetic DMARDs and biological DMARDs.

Statistical analyses

For each patient, scores were calculated for the RARBIS with and without the medication sub-scale using data from the medical chart review. Using Spearman non-parametric tests, the correlations between the RARBIS and various forms of administrative data variables were then analysed. Data taken from one year before the chart review and from two years before the chart review were examined.

We then built linear regression models with the RARBIS as the dependent variable and the administrative data variables as the independent variables using SAS (Cary NC) automated procedures and the forward, backward and stepwise selection methods to select the best model. Administrative data variables were entered into the model in the form that gave the highest Spearman correlation with the RARBIS. The inclusion criterion for model selection was p < 0.2.

We added the regression parameters based on each patient's covariate values using PROC Score (SAS, Cary NC) to calculate claims-based severity scores (with and without the medication variables) for each patient in the study cohort. Finally, we examined the correlation between the CIRAS and the RARBIS using the non-parametric Spearman correlation coefficient.

Results

Characteristics of the study population are summarised in Table 2. The study cohort was predominantly male with a mean age of 71 years. During the chart review study period, most had no functional limitations (78%) and did not require a device or wheelchair for ambulatory purposes (66%). About one-half of the population had swollen joints, morning stiffness that lasted less than one hour, but did not have an arthritis flare. The mean score for the RARBIS with medications was 4.4 (range 0 to 11) and without medications was 3.0 (range 0 to 8).

Table 2 Patient characteristics based on information from the medical records review
Full size table

Table 3 provides the unadjusted Spearman correlations for the claims-based RA severity variables and the RARBIS with and without the medication sub-scale using data from one year before the chart review study period. The variables for rheumatology visits, inflammatory markers and other laboratory markers yielded the highest correlation with the RARBIS. In our analysis using administrative data from one year before the chart review period, the highest correlation between the RARBIS and the medication variable were obtained using the medication variable created from the sum of all DMARD prescriptions in method one. For both the RARBIS with and without medication scale, having data from two years before the chart review period did not substantially increase the Spearman correlation coefficients and, in some cases, even decreased the value of the coefficients (data not shown).

Table 3 Unadjusted Spearman correlations with the rheumatoid arthritis records-based index of severity (RARBIS) with and without medication sub-scale
Full size table

Table 4 presents the adjusted correlations between the claims-based RA severity variables and the RARBIS with and without the medication sub-scale with data from one year before the chart review study period. The forward selection models yielded the highest model R2 for both the RARBIS with the medication sub-scale (R2 = 0.31) and the RARBIS without the medication sub-scale (R2 = 0.26). Using two years of data resulted in lower model R2s (data not shown).

Table 4 Adjusted correlations between claims-based variables and rheumatoid arthritis records-based index of severity (RARBIS) with and without medication sub-scale
Full size table

Table 5 includes the means and ranges for the CIRAS scores and the Spearman correlation coefficients between the CIRAS and the RARBIS. The CIRAS score with the highest correlation with the RARBIS included the following components: orders for inflammatory markers, rehabilitation visits, age and gender, rheumatoid factor, presence of Felty's syndrome, number of platelet counts and chemistry panels ordered, and rheumatology visits. Figure 1 is a graphic representation of this CIRAS score in tertiles versus the median and interquartile range for the RARBIS with medication sub-scale. Table 6 presents the suggested scoring method for the CIRAS.

Table 5 Claims-based index of rheumatoid arthritis severity (CIRAS) score (mean, range) and Spearman correlation of CIRAS score with rheumatoid arthritis records-based index of severity (RARBIS)
Full size table
Table 6 Suggested scoring method for claims-based index of rheumatoid arthritis severity (CIRAS)
Full size table
Figure 1
figure 1

This plot illustrates the median and interquartile range for the tertiles of the claims-based index of rheumatoid arthritis severity (CIRAS). These values are plotted against the records-based index of rheumatoid arthritis severity (RARBIS). The CIRAS exhibits a moderate linear correlation with the RARBIS.

Full size image

Discussion

We developed a claims-based RA severity index (CIRAS) that demonstrated moderate correlation with a previously validated medical records-based index, the RARBIS. The RARBIS has been previously shown to have good construct validity and moderate convergent validity with the DAS28 [16, 17]. Because health care utilisation databases are a valuable source of data for studying health outcomes, other investigators have also used medical records-based indices to create indices for administrative databases. For instance, Deyo and colleagues adapted the Charlson Comorbidity Index, a well-validated index designed for medical records, so that International Classification of Diseases, Ninth Revision codes could be used to calculate the Charlson Comorbidity Index from administrative data [18]. Components of the administrative-based index we developed for RA include orders for inflammatory markers, number of platelet counts and chemistry panels ordered, rheumatoid factor, rehabilitation visits, age and gender, presence of Felty's syndrome and number of rheumatology visits. If the CIRAS is found to be valid in other populations, then it might be used to partially adjust for an important confounder, disease severity, in claims-based epidemiology studies. In our analysis, we used data taken from one and two years before the chart review study period. However, using two years of data resulted in lower R2 and Spearman correlation values. Including another year of older data might have caused a dilution effect. Additionally, to compute scores on the CIRAS, we used weights from the regression models with the RARBIS. Other methods of weighting could have been chosen, for example, assigning a value of one to administrative variables that had significant correlations with the RARBIS. However, we believe that the method we selected, using beta coefficients as weights, better captures the relationship between the CIRAS and the RARBIS.

Because administrative data are collected primarily for reimbursement purposes, some question the use of claims data for clinical research regarding disease severity [19]. However, administrative data are gaining increasing acceptance in health care research, because they represent typical populations, contain large cohorts of patients with given conditions and are readily available. We also demonstrate in the present study that indicators of RA severity from claims data are moderately well related to clinical indicators of RA severity. Thus, it is possible to capture RA disease severity to some degree in claims data. Other proxies for severity of illness measures using claims data such as the diagnosis related group, the all patients refined diagnosis related group and the International Classification of Diseases Ninth Revision-Based Illness Severity Score have been developed [20]. Unlike the CIRAS, these other measures are not specific to RA.

The present study has important limitations. Our data source for this study was the New England VA Health system. The VA's population is mostly older men. Older male patients with RA may not represent typical RA patients. This highlights the need to consider these findings as preliminary and requiring replication in other settings. Additionally, data from the VA might be gathered differently from other health care systems, again highlighting the preliminary nature of our findings. However, because the VA contains rich data from both medical record and health care utilisation databases, it is a unique and ideal data source for our analysis. Additionally, the RARBIS, which we used to create the CIRAS, was developed using standard nominal group technique methods, followed by assessing its convergent validity with the DAS28. However, the DAS28 is a measure of disease activity not disease severity. While disease activity is an important component of disease severity, it is not the same. Currently, there is no standard RA disease severity measure.

In our cohort of 120 VA patients, the CIRAS showed moderate correlations with a validated medical records-based index and can be used for improved adjustment of RA disease severity in claims data studies. We do not believe that the value of the CIRAS will be limited to the VA population. We plan on assessing its validity in other populations, such as Medicare patients, and will examine its ability to adjust for confounding and predictive validity for outcomes known to be associated with severe RA, such as future joint surgeries, higher medical care costs and use of combination DMARDs. Additionally, we will explore whether different variations of the CIRAS should be used depending on the study outcome of interest. Ultimately, the CIRAS may be an important methodological tool for researchers studying RA treatment and complications using health care utilisation data, but further tests need to be conducted in other populations.

Conclusion

We developed a claims-based severity index (CIRAS) from a previously validated medical records-based index (RARBIS). The CIRAS can potentially be used for improved adjustment of RA severity in studies of RA medication use and adverse outcomes using claims data, but future studies should examine its validity in other populations.