The validity of decision rules for selecting women with primary osteoporosis for bone mineral density testing
- First Online:
- Cite this article as:
- Cadarette, S.M., McIsaac, W.J., Hawker, G.A. et al. Osteoporos Int (2004) 15: 361. doi:10.1007/s00198-003-1552-7
- 213 Views
The purpose of this study was to determine the validity of the Osteoporosis Risk Assessment Instrument (ORAI), Osteoporosis Self-Assessment Tool (OST) chart and equation, and a criterion based on body weight for identifying women with asymptomatic primary osteoporosis. Prospective recruitment and chart abstractions from family practices of three University affiliated hospitals were completed for women aged 45 years or more with baseline bone mineral density (BMD) testing results by dual energy X-ray absorptiometry. Those taking bone active medication other than hormone therapy, with prior fragility fracture or with risk factors for secondary osteoporosis were excluded. Women were categorized as being normal, osteopenic or osteoporotic by lowest BMD T-score at either the femoral neck or lumbar spine (L1–L4). Sensitivity, specificity and area under the receiver operating characteristic (ROC) curve to identify those with osteoporosis were determined for each decision rule. The positive predictive value (PPV) for detecting osteoporosis after using a second cut point to convert each decision rule into a risk index (low, moderate or high risk) was also determined. The sensitivity of the decision rules to identify women with osteoporosis ranged from 92% to 95% and specificity from 35% to 46%. The area under the ROC curves were significantly better for the ORAI (0.80), OST chart (0.82) and OST equation (0.82) compared with the body weight criterion (0.73). PPV for detecting osteoporosis ranged from 30% to 58% among women deemed at high risk. These data confirm the validity of the ORAI, the OST chart and the OST equation as screening tools for BMD testing. Further evidence is required to confirm the validity of the body weight criterion.
Bone mineral density (BMD) assessment by dual-energy X-ray absorptiometry (DXA) is the gold-standard for identifying asymptomatic individuals with osteoporosis [1, 2, 3, 4, 5]. A recent review of the clinical applications of bone densitometry suggests that clinicians need tools to identify patients most likely to benefit from testing . There are three steps involved in developing and testing tools to aid clinical decision making: 1) development, 2) validation in several cohorts , and 3) impact assessment . Several tools, or decision rules, based on simple clinical risk factors have been developed to aid in the identification of women with osteoporosis by targeting BMD testing to those at highest risk [9, 10, 11, 12, 13, 14]. While these rules currently use a single cut point for deciding whether to test or not, it has recently been suggested that two cut points be used to stratify the likelihood of osteoporosis as low, moderate or high [11, 12, 15, 16]. As a risk index, women at low risk would not require a BMD test, those with moderate risk would be recommended for BMD testing and those at high risk could be treated to prevent fracture without the need for BMD testing .
The primary purpose of this study was to determine the validity of four decision rules: the Osteoporosis Risk Assessment Instrument (ORAI), the Osteoporosis Self-Assessment Tool (OST) versions 1 (chart) and 2 (equation) and the body weight criterion for identifying asymptomatic women at high risk for primary osteoporosis, and therefore appropriate as clinical decision aids for BMD testing. The benefit of a second cut point to convert each decision rule into a risk index (low, moderate or high risk for osteoporosis) was also evaluated.
Materials and methods
Two groups of women were studied. Women aged 45 or more years presenting for BMD testing between 11 November 1999 and 25 May 2000 at an ambulatory care centre affiliated with the University of Toronto were recruited prospectively. Women taking bone active medications other than hormone replacement, with a prior fragility fracture, or with major risk factors for secondary osteoporosis (e.g. menopause before age 45, malabsorptive syndromes, hyperthyroidism, long-term glucocorticoid use) were excluded. Given that it was not feasible to recruit women prospectively at other sites, the records of a second group of women attending two family practice clinics affiliated with the University of Toronto were reviewed retrospectively. Women aged 45 years and older with a baseline DXA report since January 1997 were eligible. Information in the patient’s chart was used to exclude women taking bone active medications other than hormone therapy or with major risk factors for secondary osteoporosis. While hormone therapy is recognized as a first-line treatment for postmenopausal women with low bone mass and women with vasomotor symptoms in Canada, it is a second-line treatment for women with osteoporosis without vasomotor symptoms . As this study focuses on identifying women with osteoporosis, we included women being treated with hormone therapy. Sensitivity analyses were used to compare results when excluding women treated with hormone therapy. Ethical approval was obtained from the respective hospital ethics review boards.
Calculating decision rule scores
Decision rules for bone mineral density testing, calculation, selection cut point and cut points to form risk indices. ORAI Osteoporosis Risk Assessment Instrument, OST Osteoporosis Self-Assessment Tool
Recommend BMD testing ifa
Points are given for:
Age (years): 15 if 75+, 9 if 65–74, 5 if 55–64
Moderate: 9 to 17
Weight (kg): 9 if <60, 3 if 60–69.9
Estrogen: 2 if not currently taking
OST version 1 (chart)=a–b
a=(0.2×weight in kg); truncate to yield an integer
Moderate: -3 to 1
b=(0.2×age in years); truncate to yield an integer
OST version 2 (equation)=0.2×(weight in kg−age in years); truncate to yield integer
Body weight criterion 
Body weight in kg
Moderate: 57 to <70
Decision rule thresholds for recommending BMD testing
Previously validated cut points for Caucasian women (ORAI >8, OST <2 and body weight <70 kg) were used to determine whether a given women would be recommended to undergo BMD testing for a particular decision rule [9, 10, 15, 17]. Each decision rule was then converted into a risk index by using a second cut point to differentiate between moderate and high risk for osteoporosis among those who would have been selected for testing. The second cut point to determine high risk for osteoporosis was chosen as previously evaluated among primarily Caucasian cohorts  (Table 1). Given that the body weight criterion has not been previously evaluated using two cut points, a value <57 kg was used to identify high risk. This value was chosen as it is listed as a risk factor for osteoporosis in the Canadian clinical practice guidelines  and is similar to the body weight criteria (<57.6 kg, or 127 lb) reported as an indication for BMD testing by the National Osteoporosis Foundation .
BMD testing by DXA was used as the gold standard for diagnosing osteoporosis. Bone mass was categorized as being normal (T-score ≥−1.0 SD), osteopenic (–1.0 >T-score >−2.5 SD) or osteoporotic (T-score ≤−2.5 SD) based on lowest BMD t-score at the femoral neck and lumbar spine (L1–L4) . BMD was categorized based on the single site where details for only one site were available.
Demographic and other characteristics of the study population were calculated as means and standard deviations, or proportions as applicable. The sensitivity, specificity and the area under the receiver operating characteristic (ROC) curve of each decision rule for selecting women with osteoporosis by BMD testing were determined. In addition, the positive predictive value for identifying osteoporosis for each risk index of low, moderate and high risk was calculated. Analyses were repeated evaluating BMD outcomes at the lumbar spine and femoral neck separately, as well as when excluding women currently taking hormone therapy. Exact binomial 95% confidence intervals (CI) were calculated.
Summary of descriptive characteristics of the study cohort, n=644. BMD bone mineral density, lowest value at the femoral neck or lumbar spine (L1–L4). Normal: T-score ≥−1.0SD, 0steopenia: −1.0 >T-score >−2.5 SD. Osteoporosis: T-score ≤−2.5 SD
Manufacturer of dual-energy X-ray absorptiometry machine
Current estrogen usea
Overall, the OST chart would recommend slightly fewer women (60%, 95%CI=56.5–64.2) for BMD testing compared with the other decision rules (66–70%), but the difference was only statistically significant when compared with the body weight criterion (70%, 95%CI=66.3–73.5). The largest difference was in the number of women with normal BMD who would have been recommended to undergo BMD testing. The body weight criterion selected significantly more women with normal BMD (54%, 95%CI=48.0–60.7) versus OST chart (32%, 95%CI=26.5–38.5) or OST equation (39%, 95%CI=32.6–45.1). Although the ORAI selected 10% fewer women with normal BMD compared with the body weight criterion, this difference did not reach statistical significance (44%, 95%CI=37.7–50.4).
Sensitivity and specificity of each decision rule for selecting women with osteoporosis, n=644. ORAI Osteoporosis Risk Assessment Instrument, select women with score >8, OST chart Osteoporosis Self-Assessment Tool version 1 (original), select women with score <2, OST equation Osteoporosis Self-Assessment Tool version 2 (modified), select women with score <2, body weight criterion, select women who weigh <70 kg, osteoporosis bone mineral density T-score ≤−2.5 SD by lowest value at the femoral neck or lumbar spine (L1–L4)
Body weight criterion
Proportion of women with low, moderate or high risk for osteoporosis by each decision rule and corresponding positive predictive values. ORAI Osteoporosis Risk Assessment Instrument, OST chart Osteoporosis Self-Assessment Tool version 1 (original), OST equation Osteoporosis Self-Assessment Tool Version 2 (modified), body weight criterion weight in kg, PPV proportion of sample in the category with osteoporosis, osteoporosis bone mineral density T-score ≤−2.5 SD by lowest value at the femoral neck or lumbar spine (L1–L4)
Distribution of study sample
Positive predictive value (PPV)
9 to 17
−3 to 1
−3 to 1
Body weight criterion
57 to <70
At present, no universally accepted policy for screening to identify patients with osteoporosis exists. While mass screening for osteoporosis is not recommended [2, 4, 20], using DXA to screen high risk groups is essential to facilitate osteoporosis diagnosis and allow prophylactic treatment for the prevention of further bone degeneration and fracture [3, 4, 18, 21, 22]. Although there are many published guidelines for managing osteoporosis that provide lists of indications for the diagnostic use of bone densitometry [3, 4, 18, 21, 22, 23], implementing such guidelines in practice is difficult [24, 25]. For example, a study of family physicians in Ontario, Canada found that they were most interested in evidence-based strategies for managing osteoporosis that are easy to apply in practice . The clinical decision rules evaluated in this study may be used to assist physicians in making decision about BMD testing.
While previous studies have evaluated the validity of decision rules for BMD testing, further validation in different populations is important to get a sense of the generalizability of these approaches . This study has demonstrated the validity of the ORAI, the OST chart and the OST equation as a means of assisting physicians in making decisions regarding BMD testing. These three tools selected over 90% of women with osteoporosis, yet less than half of the women with normal BMD. These data confirm prior findings in cohorts of primarily Caucasian women [9, 15, 17]. Although the results suggest that body weight alone may identify just as many women with osteoporosis, other research has found that the sensitivity of the body weight criterion was significantly less [9, 17]. Consistent with findings in this study, the body weight criterion has also previously been found to select significantly more women with normal BMD for testing . Therefore, further evidence is required to confirm the benefit of body weight alone as a decision aid for BMD testing.
A number of clinical practice guidelines [3, 18, 22] and the US Preventive Services Task Force  recommend BMD testing for women aged 65 or more years. The ORAI is consistent with, and provided some of the evidence for this recommendation [2, 3, 9]. The OST decision rules, however, provide more specific criteria based on body weight. In the current study, the OST recommended 90% (chart) to 97% (equation) of women aged 65 or more years for testing, while each identified 100% of women aged 65 or more years with osteoporosis. While both versions of the OST performed similar, the simplicity of the OST chart in not requiring any calculations may be more appealing to clinicians. However, as electronic patient records become common practice, the OST equation may become useful as an automatic clinical reminder for BMD testing .
The OST equations have also been validated using race-specific cut points among Asian women, and sex-specific cut points for depicting risk for osteoporosis among men [27, 28]. Although the ORAI has been validated using a cut point of 15 or more among Asian cohorts and its sensitivity for identifying those with osteoporosis was similar to the OST equation (cut point ≤–1) among Japanese women (89–91% ORAI versus 87–88% OST equation), its sensitivity was only 84% versus 91% in another study of Asian women [11, 16]. These data suggest that the OST chart and equation may be better applied with race-specific cut points to identify risk for osteoporosis among Asian women, as well as with sex-specific cut points to identify risk for osteoporosis among men.
The OST chart can also be simplified further by replacing the numerical values with colors to indicate gradient of risk for osteoporosis. For example, all squares with values less than –3 could be colored red, those with values between 1 and –3 yellow, and those with values greater than 1 green. Similar to a traffic light, green would indicate low risk for osteoporosis (4%), yellow caution (17%), and red indicating high risk with a 58% likelihood of osteoporosis. Although a similar color-coded legend could be used with the OST equation or the ORAI, a mathematical calculation is still required to determine a woman’s score.
Converting the decision rules into risk indices of low, moderate and high risk may be useful to clinicians. Our data validate the cut points for the ORAI and both versions of the OST previously determined among primarily Caucasian women . Although it has been suggested that treating those in the high risk group without BMD testing may be beneficial , our data indicate that 6–20% of those in the high risk group will have normal BMD. This raises the question about the potential usefulness of having a high-risk category and the willingness of women to undergo long-term treatment without confirmatory BMD testing. Rather, these categories may serve as a means of educating patients regarding their risk for osteoporosis.
The limitations of this study stem from the sample and methods of data collection. Given that this study is based on data from women already selected for BMD testing, study results may not be generalizable to all women aged 45 or more years. While the prevalence of osteoporosis in our sample of women aged 50 or more years of 18.4% (16.5% ages 45–90), is similar to that reported by the Canadian Osteoporosis Multicentre Study among women aged 50 or more years , we would expect our sample to have a slightly lower prevalence owing to the fact that we excluded women with known prior fragility fracture and those identified as being at high risk for secondary osteoporosis. Therefore, the sample of women under study may be more likely to be at risk for osteoporosis based on unexamined factors. For example, they may be more likely to have a family history of osteoporosis that prompted the BMD test. Similarly, vertebral fractures commonly go undiagnosed , and thus women presenting for BMD testing may have been more likely to have a prior vertebral fracture. However, if there was any suspicion of vertebral fractures in the patient’s chart, they were excluded from the study. While the underlying prevalence of osteoporosis may impact the PPV of the study (here by overestimating the PPV), results presented for the sensitivity and specificity would not be compromised . Due to the similar lack of clinical information, we were also unable to evaluate the validity of other decision rules for BMD testing, such as the Simple Calculated Osteoporosis Risk Estimation (SCORE) , the Osteoporosis Index or Risk (OSIRIS)  or the Age, BOdy size, No Estrogen (ABONE) . However, the decision rules included in this study are generally simpler to calculate, and may thus be more easily applied in practice .
BMD measurement is important to diagnose osteoporosis and thus intervene to prevent further bone loss and the pain/morbidity associated with fractures. Although mass screening is not recommended, case finding of women at high risk for osteoporosis is an important strategy to try and reduce osteoporotic fractures in the population. The decision rules in this study are meant to assist physicians in identifying asymptomatic women likely to have primary osteoporosis before sustaining a fracture. Women with a prior fragility fracture are at high risk for osteoporosis and recurrent fracture and should be referred for BMD testing to facilitate treatment decisions [3, 4, 18, 21, 22]. Similarly, women with major risk factors for secondary osteoporosis should discuss bone health and BMD testing independent of these decision rules.
In conclusion, this study provides evidence for the validity of the ORAI, OST chart and OST equation as useful clinical aids to assist physicians in making decisions about which women to refer for BMD testing for the purpose of diagnosing osteoporosis. Each rule identifies over 90% of women with primary osteoporosis while limiting BMD testing among those with normal BMD. However, the relatively high proportion of women with normal BMD in the high-risk categories may limit the usefulness of these rules as guides to treatment without confirmatory BMD testing. The practical application of these decision rules and risk indices in facilitating discussions of osteoporosis and in selecting women at high risk for osteoporosis for BMD testing should be explored further, including the potential benefit as well as harms, such as labeling women at high risk for osteoporosis . A direct comparison with usual practice is also needed to determine if decision rule approaches provide more optimal use of BMD testing .
Ms. Cadarette is supported by a Doctoral Research Award from the Canadian Institutes of Health Research (CIHR) in partnership with the Ontario Ministry of Health and Long-Term Care (OMHLTC), and an Ontario Women’s Health Scholars Doctoral Prize by the Ontario Women’s Health Council. Dr. Jaglal is a Career Scientist of the OMHLTC and Dr. Hawker is a CIHR Scientist.