Introduction

Health indicators are used as a basis to evaluate the quality of health care. In osteoporosis, quality indicators among women aged 65 or more years include risk assessment by dual-energy X-ray absorptiometry (DXA) and/or pharmacotherapy within 6 months following fragility fracture. Since 2004, the National Center for Quality Assurance in the USA has included DXA testing and/or treatment within 6 months of fracture as a measure of the quality of osteoporosis management [1, 2]. In 2007, the province of Ontario, Canada began funding osteoporosis coordinators in fracture clinics to help improve osteoporosis pharmacotherapy post-fragility fracture—a program modeled after a successful single-site project [3, 4]. Better understanding of the accuracy of healthcare utilization (medical and pharmacy claims) data to identify DXA testing, osteoporosis diagnosis, and osteoporosis pharmacotherapy will clarify the benefits of using these data to track the quality of osteoporosis management. In Ontario, pharmacy claims are only available for residents aged 65 or more years. Exposure to osteoporosis pharmacotherapy before age 65 is not available, and thus, relying on pharmacy claims may underestimate prior treatment exposure. In addition, to our knowledge, the validity of claims data to identify DXA testing has not previously been examined.

To get a better understanding of the accuracy of healthcare utilization data in Ontario, we linked data from community-dwelling women aged over 65 years who completed a standardized telephone interview regarding osteoporosis management, and their DXA test results when available, to their healthcare utilization records. We hypothesized that agreement between self-report of drug use and pharmacy claims would be good, little measurement error would be found when using medical claims data to identify DXA testing, and thus collectively, results would support the validity of healthcare utilization data to examine quality indicators of osteoporosis management.

Methods

Subjects

Between May 2003 and May 2004, we collected detailed information regarding osteoporosis management and fracture risk from 871 community-dwelling women aged 65 to 90 years who resided within two regions of Ontario, Canada [5–9]. The study sample was randomly selected from a list of 14,163 participants who completed a short baseline questionnaire between 1995 and 1997 [6]. We recruited participants by standardized telephone interview (participation rate = 84%, response rate = 72%). Respondents were similar to non-respondents in terms of fracture history, osteoporosis diagnosis, and osteoporosis treatment [9], as determined by self-reported data collected at baseline [10].

Data sources and measures

Study questionnaire (self-report of drug use and DXA testing)

As part of the standardized telephone interview completed in 2003/2004, we asked participants if they had ever had a bone density test and recorded information regarding osteoporosis pharmacotherapy (bisphosphonates, calcitonin, and raloxifene) and the use of other agents that may impact bone density (estrogen therapy, glucocorticoids, and thyroid medication) as current, past, or never. Question wording is included in the “Appendix.”

DXA confirmation and DXA—documented osteoporosis

DXA results were sought from participants who reported having had a DXA test and who completed a signed release of information form. For these patients, physicians were contacted to confirm that a DXA was completed and to obtain a copy of the most recent DXA report. We previously reported that the positive predictive value for self-report of having had a DXA was 93% when using physician responses as the gold standard [5]. Among those with a DXA report, we categorized bone mineral density according to the lowest T-score at the lumbar spine (L1-4 or L2-4) or hip (femoral neck or total hip) as normal (T-score ≥ −1), osteopenic (−1 < T-score > −2.5), or osteoporotic (T-score ≤ −2.5) [11].

Healthcare utilization data—medical claims

In Canada, physician and hospital services are funded through publicly financed comprehensive universal health insurance. In Ontario, claims for physician services are documented in the Ontario Health Insurance Plan (OHIP) Claims History Database. Information about inpatient services are captured in the Canadian Institutes of Health Information Discharge Abstract Database, and information about emergency department services are documented in the National Ambulatory Care Reporting System. Prior to April 1, 2002, hospital and emergency department records were coded using the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM). Since then, they have been coded using ICD-10-Canada (CA). July 1991 is the earliest date for which individual level data are available.

DXA tests were identified using OHIP claim codes: J654, J655, J656, J688, J854, J855, J856, J888, X145, X146, X149, X152, X153, X155, and X157. These include codes for dual-photon absorptiometry, which predates DXA technology and was used prior to April 1998 [12]. We considered claims back to July 1991 when individual level claims data were first recorded in Ontario.

Osteoporosis diagnosis was identified by any OHIP diagnosis code of 733 or any hospitalization or emergency department visit code of ICD-9-CM = 733.0 or ICD-10-CA = M80, M81, or M82. We considered diagnosis within 1 year pre- and post-DXA, as well as within 1 to 5 years before questionnaire completion.

Healthcare utilization data—pharmacy claims

Residents of Ontario qualify for provincial drug coverage on the first day of the month following their 65th birthday [13]. Ontario Drug Benefits claims data were used to identify use of bisphosphonates (alendronate, etidronate, and risedronate), calcitonin, estrogen therapy, raloxifene, oral steroids, and thyroid medication using a 1-year lookback period from date of questionnaire completion. “Current users” were those whose questionnaire completion date fell within a period of drug treatment—defined by the prescription dispensing date, number of days of medication supplied, and a 50% grace period to allow for a missed or reduced dose. “Past use” was identified by dispensing within the lookback period, without theoretical overlap with questionnaire date. “Never use” was coded when there were no relevant pharmacy claims within the lookback period.

In a sensitivity analysis, we considered a lookback period of 180 days as this time frame was examined previously [14]. We also considered a lookback period of 5 years restricted to the subgroup aged 70 or more years to permit a longer period of time to define “never” use based on pharmacy claims. Non-osteoporosis formulations (daily or IV etidronate, 40 mg alendronate, 30 mg risedronate, and 50/100 IU nasal calcitonin or injection calcitonin) were documented separately. We did not consider teriparatide or zoledronic acid because these were not available during the study period.

Data linkage and eligibility

Study participants were linked to provincial healthcare utilization databases using probabilistic matching based on name, date of birth, and residential postal code [15]. While deterministic linkage using a common unique identifier, such as health insurance number, would have been preferable, we did not collect this detail from participants during the survey. Participants successfully linked to claims data were eligible for the current study. We then restricted inclusion to those aged 66 or more years at the time of questionnaire completion to ensure a minimum of 1 year of pharmacy claims data prior to questionnaire completion. All analyses were performed at the Institute for Clinical Evaluative Sciences.

This study was approved by the Research Ethics Board of Sunnybrook Health Sciences Centre.

Statistical analysis

Descriptive statistics were used to summarize sociodemographic characteristics of participants and drug use within the year prior to questionnaire completion. Agreement between self-report of drug use and pharmacy claims was examined using kappa statistics for current versus past/never use and ever versus never use. Quadratic weighted kappa statistics were calculated for ordinal values of never, past, or current use. Kappa statistic values below 0.61 indicate from no to fair agreement, between 0.61 and 0.80 indicate good agreement, between 0.81 and 0.92 indicate very good agreement, and between 0.93 and 1.00 indicate excellent agreement [16].

The validity of medical claims data to identify DXA testing was examined by calculating sensitivity and specificity with corresponding 95% exact binomial confidence intervals. Sensitivity was calculated as the proportion of physician-confirmed DXA tests identified in medical claims data. We estimated the specificity of DXA testing as the proportion of participants reporting not to have had a DXA test that were “correctly” classified as such in medical claims data. Given that DXA testing among women aged 65 or more years is considered a quality indicator of osteoporosis management, we defined a minimum sensitivity and specificity of 90% to be appropriate.

Sensitivity and specificity of claims data to identify DXA-documented osteoporosis was determined among the subgroup with DXA results. Osteoporosis (T-score ≤ −2.5) on the DXA report was used as the gold standard diagnosis.

Results

Characteristics of study participants

Eight hundred and sixty-seven of 871 questionnaires (99.5%) were successfully linked to healthcare utilization data, and 858 of these subjects (99.0%) were eligible—aged 66 or more years (mean age = 75 years, SD = 6.0, median = 75, range 66 to 90). The sample included primarily Caucasian (96%), native English-speaking (82%), non-smokers (91%), with at least some high school education (78%; Table 1). About half of the subjects resided in the Metropolitan area of Toronto (population density of 5,418/km2), one third in small towns or rural areas (population density of 33/km2), and the remaining 20% in a small city (population density of 1,086/km2).

Table 1 Characteristics of study participants, N = 858
Full size table

Agreement between self-report and pharmacy claims-based treatment

Agreement between self-report and claims-based osteoporosis pharmacotherapy categorized as current, past, or never was very good (quadratic weighted κ = 0.81, 95% CI = 0.76, 0.86). The breakdown by agent is summarized in Table 2. We found no claims for non-osteoporosis formulations of bisphosphonates (200 mg or 400 mg daily, or intravenous etidronate, and 40 mg alendronate or 30 mg risedronate) or calcitonin (50 or 100 IU nasal or intravenous) within the year preceding questionnaire completion. One fifth (n = 187) had an eligible oral bisphosphonate, and fewer than ten participants had prescription claims for nasal calcitonin or raloxifene. Agreement between self-report and pharmacy claims was particularly high for current use of cyclical etidronate (κ = 0.86, 95% CI = 0.80, 0.92) and thyroid medication (κ = 0.92, 95% CI = 0.88, 0.95). Agreement was particularly poor for ever use of estrogen therapy (κ = 0.33, 95% CI = 0.28, 0.39) and oral steroids (κ = 0.35, 95% CI = 0.25, 0.46). Results were similar based on a 180-day lookback period instead of a 365-day lookback period, or using a 5-year lookback period, and restricting to ages 70 or more years (data not shown). However, applying the 5-year lookback improved the agreement between ever use of estrogen therapy (from κ = 0.33 to κ = 0.45) and oral steroids (from κ = 0.35 to κ = 0.47).

Table 2 Agreement between self-report and claims-based drug use history, N = 858
Full size table

Validity of claims data to identify DXA testing

Physicians confirmed the presence of a DXA test in 379 women. The sensitivity of claims data to identify these 379 confirmed DXA tests was 98% (95% CI = 95.9, 99.1; Table 3). Using self-report of DXA testing as the gold standard, the estimated specificity of a reimbursement claim for DXA testing was 93% (95%CI = 89.8, 95.4).

Table 3 Proportion of women with a dual-energy X-ray absorptiometry (DXA) test identified in claims data among those reporting to have had a DXA test, by length of claims lookback period, N = 501
Full size table

Validity of claims data to identify DXA-documented osteoporosis

Of the 379 confirmed DXA tests, we obtained 359 complete DXA reports, and 114 (32%) had DXA-documented osteoporosis. The sensitivity for identifying DXA-documented osteoporosis was highest when including osteoporosis pharmacotherapy claims and/or diagnostic codes (80% when considering the year after DXA date), while specificity was highest (over 92%) when we required both an osteoporosis diagnosis and a pharmacy claim (Table 4). When we used a definition of any osteoporosis diagnosis and/or pharmacotherapy within the year following DXA testing, sensitivity was 80% (95% CI = 71.3, 86.8), and specificity was 72% (95% CI = 66.2, 77.8). This was similar to results using a 365-day lookback in the pharmacy claims and a 5-year lookback for osteoporosis diagnoses in medical claims: sensitivity = 82% (95% CI = 74.5, 88.7) and specificity = 66% (95% CI = 59.8, 71.7)—data not shown in table.

Table 4 Ability of claims data to identify patients with dual-energy X-ray absorptiometry (DXA)-documented osteoporosis among those having had a DXA test, N = 359
Full size table

Discussion

Payers of healthcare rely on quality indicators to assess the performance of their healthcare system, to identify areas for improvement, and to assess the ability of targeted interventions to improve outcomes. We found healthcare utilization data to be very good at identifying DXA testing with sensitivity of 98% and specificity of 93%. We also identified very good agreement between self-report and claims-based osteoporosis pharmacotherapy (κ = 0.81) despite only having pharmacy data since age 65 years and applying a 1-year lookback period. Our data therefore support the use of healthcare utilization data to measure DXA testing and osteoporosis pharmacotherapy among women aged over 65 years. Our results are also similar to those reported among a younger cohort of chronic glucocorticoid users enrolled in a managed care program with kappa statistics for agreement between self-report and claims data of 0.80 (versus 0.81 in our study) for alendronate and 0.78 (versus 0.79 in our study) for risedronate [14].

Although we identified very good agreement between self-report and claims data for osteoporosis pharmacotherapy, we found that the ability of claims data to identify past use of estrogen or oral steroids was poor, and both exposures have implications for bone health. These results are not surprising since estrogen therapy is commonly prescribed at the time of menopause, and oral steroids may be prescribed for a number of conditions that are not specific to those aged over 65 years. Nonetheless, agreement between claims data and self-report of thyroid medication use that is intended for chronic use was very good.

Our results also identify the importance of pharmacy claims data to help identify DXA-documented osteoporosis, as relying on medical diagnosis claims alone identified only 43% of women with DXA T-score ≤ −2.5. The combination of medical diagnosis claims and pharmacy claims proved to be a good proxy for DXA-documented osteoporosis, with a sensitivity of 80% and specificity of 72%. Our results therefore suggest that healthcare utilization data may provide a reasonable method to identify those most likely to have DXA-document osteoporosis. Although we had DXA results for only 359 of the 501 women (72%) reporting to have had a DXA, the prevalence of osteoporosis is similar to prior age-stratified prevalence in North American women [17–19]. We thus believe little bias was introduced by only having data for a subset of women who reported having been tested by DXA.

We report the ability of healthcare utilization data to identify DXA-documented osteoporosis but cannot comment on the ability of these data to identify asymptomatic, untreated osteoporosis. Nonetheless, among a subgroup having been tested by DXA, healthcare utilization data may provide a reasonable method to identify those most likely to have DXA-documented osteoporosis. A recent study from Manitoba, Canada similarly found that including osteoporosis pharmacotherapy as well as osteoporosis diagnosis improved the ability of healthcare utilization data to identify DXA-documented osteoporosis. This study included all patients aged 50 or more years who had DXA and recommends the use of age, fracture diagnoses, and persistence with osteoporosis pharmacotherapy to improve the identification of patients with DXA-documented osteoporosis [20]. However, the ability of these more comprehensive algorithms to identify DXA-documented osteoporosis had similar discriminatory performance to that using osteoporosis diagnosis or pharmacotherapy in our study, given our underlying prevalence of osteoporosis of 32%. Our results therefore suggest that among women aged over 65 years, identification of DXA-documented osteoporosis based on medical diagnosis codes and osteoporosis pharmacotherapy may be as robust as more complex algorithms that also consider fracture history and age.

In addition to those already mentioned, several other study limitations are worth noting. First, we studied women in a single province of Canada that uses provincial-specific claim codes for outpatient physician services (OHIP claims). However, given that the OHIP diagnostic code for osteoporosis (733) is essentially the same as the ICD-9-CM code of 733.0, we believe that our results will generalize to other jurisdictions that use ICD-9-CM codes in the outpatient setting. Similarly, although we used provincial-specific procedural codes to identify DXA testing, our results are expected to generalize to other jurisdictions that operate on a fee-for-service basis. Second, our results are most applicable to use of bisphosphonates, as we had few exposures to nasal calcitonin or raloxifene and no exposure to teriparatide or zoledronic acid. Finally, by using only the most recent DXA test to define DXA-document osteoporosis, we may have misclassified some patients whose BMD improved with therapy yet had been classified as osteoporotic on a prior DXA.

Despite limitations, our study has many strengths. We studied a broad sample of older women residing within different regions of Ontario, and the prevalence of osteoporosis in our study is consistent with age-stratified estimates for North American women [17–19]. We therefore believe that our study results are highly representative of the ability of claims data to identify quality indicators of osteoporosis management among older women in Ontario, and that our results may generalize to other jurisdictions that use healthcare administrative claims for billing purposes. In conclusion, healthcare utilization data may be useful as quality indicators of the assessment of DXA testing and osteoporosis pharmacotherapy (care processes), with minimal measurement error in women over 65 years of age. However, medical and pharmacy claims do not provide a good means for identifying women with underlying osteoporosis.