In 2014, an estimated 136,830 Americans will be diagnosed with colorectal cancer (CRC),1 and an estimated 50,310 will die of the disease.1 Appropriately applied, routine screening can reduce both CRC incidence and mortality.2 Thus, many health organizations, including the Veterans Health Administration (VHA), have invested heavily in programs to increase CRC screening. Clinical reminder systems are now commonly used to alert providers at the point of care when patients are due for screening. CRC screening performance measures are now included in most quality measurement systems. In the VHA, CRC screening rates are used as one measure of the quality of care provided by physicians, clinics and facilities, and can affect clinician and administrator compensation plans. As a result, the VHA CRC screening rate of 80 %3 is well above US general population rates.4

One potential unintended consequence of the emphasis on screening promotion is the inappropriate use of screening tests. There are at least three reasons a CRC screening test could be clinically inappropriate. First, the patient may not be due for screening. The US Preventive Services Task Force recommends screening with fecal occult blood tests (FOBTs) annually, with colonoscopy every 10 years, or with flexible sigmoidoscopy every 5 years.5 Second, the patient may be unlikely to live long enough to realize a screening benefit. Decision analyses and a meta-analysis of randomized controlled trial (RCT) data suggest that average-risk individuals with a life expectancy of less than 10 years are unlikely to benefit from CRC screening.6,7 Third, the patient may receive the wrong test. Generally, patients with CRC-related symptoms or with a personal or family history of CRC should undergo colonoscopy (not FOBT or other tests).8 Screening patients who are not yet due or who are of limited life expectancy can strain gastroenterology and laboratory resources and expose patients to unnecessary inconvenience, stress, and medical risk. Serious complications are estimated to occur in 25 per 10,000 colonoscopy procedures,9 and deaths attributable to colonoscopy occur in three per 10,000 procedures.10 These forms of inappropriate screening can result in harm even when FOBTs are used because, if the FOBT is positive, the patient may undergo a diagnostic colonoscopy. Utilizing FOBTs when a colonoscopy is indicated may delay or decrease the likelihood of undergoing a needed colonoscopy, potentially decreasing the benefits of screening.

Previous studies of inappropriate CRC screening11–14 assessed only one facility, one screening mode, or one category of inappropriate screening. The current study quantifies the extent of potentially inappropriate screening for both FOBT and colonoscopy in the entire VHA system (130 medical facilities), using a measure that distinguishes between three types of inappropriate screening (not due, limited life expectancy, and wrong test), and examines the association between this measure and patient and facility attributes.

METHODS

Sample Identification

We searched VHA databases containing extracted medical record information on all VHA patients to identify data on all colonoscopies and outpatient fecal occult blood tests (FOBTs) performed or paid for by VHA between 1 October 2009 and 31 December 2011 on patients aged 50 years and older. We did not include flexible sigmoidoscopy, since this mode represents fewer than 2 % of CRC testing procedures. Among patients with two or more CRC tests, the first was used (referred to hereafter as the index test).

Dependent Measures

To identify potentially inappropriate tests, we extracted patient age, diagnosis codes associated with gastrointestinal (GI) or life-limiting conditions, and dates of all colonoscopies, FOBTs, flexible sigmoidoscopies and barium enemas occurring within 10 years of the index test. We also obtained the FOBT result.

Cases were classified as “potentially inappropriate—not due” if there was a completed FOBT within the prior 10 months of the index test date, colonoscopy within 9.5 years, or sigmoidoscopy or barium enema within 4.5 years. These time intervals have been used in prior research15 and are slightly shorter than intervals recommended in national guidelines.5 This allows for early screening that may occur due to scheduling convenience.

For each patient, we calculated a Charlson–Deyo comorbidity score,16,17 an estimate of disease burden that predicts CRC18 and all-cause mortality.19 Patients were classified as “potentially inappropriate—limited life expectancy” if their Charlson-Deyo Comorbidity score was four or greater, if there was documentation of life expectancy less than 1 year, or if over the age of 85 years. The Charlson-Deyo cutoff of four has been used to define severe life-limiting comorbidities in prior cancer screening studies.20,21 The age criterion is based on the US Preventive Services Task Force recommendation against screening patients over the age of 85 years.22

Cases were classified as “potentially inappropriate—wrong test” if the index test was an FOBT and the patient had documentation of a recent gastrointestinal condition that was considered an indication for colonoscopy (see Appendix 1), a personal/family history of CRC, or a prior positive FOBT and no follow-up colonoscopy.

Independent Measures

Patient variables examined included: gender, age, race, ethnicity, and marital status. These variables have been associated with CRC screening in prior research.23–25

We hypothesized that facility CRC screening rate (percent of patients screened), dependence on FOBT versus screening colonoscopy, and complexity (a standardized score composed of workload volume, patient risk level, number of complex clinical services offered, and amount of teaching and research activity that represents complexity of services and activities)26 would correlate with potentially inappropriate screening, because these variables are likely to be associated with the number and types of processes in place to facilitate screening. To test these hypotheses, we calculated the proportion of index tests that were FOBTs, and obtained 2010 CRC screening rates and complexity scores (derived from 2009 to 2010 data). To compare relatively low, moderate, and high scores, facilities were split into three equal groups on these measures.

All VHA primary care clinics utilize a CRC screening clinical reminder (CR) that is programmed locally. We hypothesized that attributes of these CRs would be associated with potentially inappropriate screening because these attributes affect which staff members discuss CRC screening with patients, the information that is discussed, and the ability of providers to exclude patients from future screening. To obtain CR attributes, we contacted a computer application programmer at each facility and obtained screenshots and programming language of CRs from 104 of 130 facilities. These materials were coded to capture key CR attributes.

Identification of Screening Colonoscopies

To distinguish screening from non-screening colonoscopies, three investigators manually reviewed charts of a random set of 754 colonoscopy cases. Cases in which providers entered the term “screening” or “Sx” as a procedure indication and did not list accompanying symptoms or additional indications were classified as screening (391 cases). All other indications were classified as non-screening (351 cases). We excluded 12 cases because indication was undocumented. A random 41 cases reviewed by two reviewers resulted in two disagreements (Agreement = 0.95, Cohen’s kappa = 0.90). Disagreements were resolved through discussion. A two-step algorithm was then developed to classify all colonoscopies as screening or non-screening. In Step 1, we classified as non-screening all chart-reviewed colonoscopies with a CRC-related diagnosis code entered between 0 and 365 days of the procedure. This set of codes (Appendix 2, Step 1) has been used previously to identify non-screening colonoscopies.27,28 In Step 2, we developed a Random Forest model to classify cases not classified as non-screening in Step 1. (See Appendix 2, Step 2 for a list of variables used.) The Random Forest method uses regression trees and random selection of predictor variables and cases to avoid high-dimensionality. This two-step classification process correctly classified 88 % of chart-reviewed screening cases and 100 % of chart-reviewed non-screening cases. We then applied this classification process to colonoscopy cases in the full data set. Our final sample included 901,292 FOBT and 134,335 colonoscopy cases classified as screening. We excluded 272,352 colonoscopies classified as non-screening.

Analyses

We report the percent of tests classified as potentially inappropriate by screening mode (colonoscopy, FOBT) and by the reason for designation (not due, limited life expectancy, or wrong test). The percent of potentially inappropriate tests is further broken out by patient and facility characteristics in Appendices 3 and 4. Among potentially inappropriate FOBT cases, we report the percent of cases with a positive test result.

We used hierarchical logistic regression models to assess the relationship between each independent variable and potentially inappropriate screening, adjusting for clustering of patients within facilities. Models were created to assess independent variables by each mode (FOBT or colonoscopy) and by each reason (any reason, not due, limited life expectancy, wrong test). For each dependent variable, we created a single model, entering all patient characteristics. Because many of the facility characteristics were highly correlated, we then estimated the effect of each facility characteristic, controlling for patient characteristics. A dummy code was created for each independent variable to represent missing data. (For parsimony, odds ratios for missing data categories are not reported.) All statistical analyses were performed using SAS 9.2 (SAS Institute Inc, Cary, North Carolina).

RESULTS

Sample Characteristics

Characteristics of patients and facilities are reported in Tables 1 and 2. FOBT and screening colonoscopy patients were similar, although FOBT patients were older (MeanFOBT = 64.7 years, MeanColonoscopy = 61.0 years). Patients were primarily white (80.3 %) and male (95.1 %). Facility CRC screening rates were high across all sites, ranging from 71 to 93 %. FOBT was the predominant screening mode. Only a small number of sites had CRs that included questions about the patient’s family CRC history (8.7 %). All other CR characteristics were present in at least 18 % of facilities.

Table 1. Patient Characteristics by Mode of Screening (Percent or Mean and Standard Deviation of Patient Cohort)
Full size table
Table 2. Facility Characteristics (Percent of Facilities and Number of Facilities for which Data Was Obtained
Full size table

Rates of Potentially Inappropriate Screening

In all, 24.5 % of cases were classified as potentially inappropriate (26.1 % of FOBTs and 14.2 % of colonoscopies—see Table 3). Patients could be categorized as potentially inappropriate for multiple reasons. Among both FOBT and colonoscopy groups, “patient not yet due” was the most common reason for being designated as potentially inappropriate (13.9 and 10.4 % respectively). Time since prior colonoscopy was the predominant reason cases were not due. Sensitivity analysis using a more restrictive definition of “not due” (9 months for FOBT, 9 years for colonoscopy, and 4 years for sigmoidoscopy/barium enema) reduced the number of “not due” cases by 1 %. Limited life expectancy accounted for 7.3 % of potentially inappropriate tests. A total of 11.0 % of FOBTs were classified as potentially inappropriate because a colonoscopy appeared to be indicated instead of a FOBT.

Table 3. Percent of Screened Patients Who Were Classified as Having Received Potentially Inappropriate Screening by Screening Mode and Reason Test Was Potentially Inappropriate (Patients Could Be Classified as Potentially Inappropriate for Multiple Reasons)
Full size table

FOBT positivity rates were 13 % among “not due” patients, 16 % among “limited life expectancy” patients, and 19 % among patients indicated for colonoscopy.

Characteristics Associated with Potentially Inappropriate Screening

Associations between potentially inappropriate screening and patient and facility characteristics are reported in Tables 4 and 5. Each additional 10 years of patient age was associated with an 83 % increased likelihood of potentially inappropriate screening among FOBT patients and 60 % increased likelihood among colonoscopy patients (all p < 0.001). This relationship between age and potentially inappropriate screening was consistent across all reasons. Potentially inappropriate FOBTs were less common among males than females (OR = 0.75, p < 0.001), primarily because males were less likely to complete an FOBT before due (OR = 0.77, p < 0.001) or to receive the wrong test (OR = 0.68, p < 0.001). Male colonoscopy recipients, however, were more likely than females to be of limited life expectancy (OR = 1.49, p < 0.001). African Americans were more likely than Whites to complete a potentially inappropriate test (ORFOBT = 1.47, ORcolonoscopy = 1.20, all p < 0.001), with limited life expectancy contributing most to this association (ORFOBT = 1.72, ORcolonoscopy = 1.69, all p < 0.001). Hispanic and unmarried statuses were significantly positively associated with potentially inappropriate screening for any reason; however, these effects were small.

Table 4. Association Between Patient Characteristics and Receipt of a Potentially Inappropriate FOBT or Screening Colonoscopy by Reason Type*
Full size table
Table 5. Association Between Facility Characteristics and Receipt of a Potentially Inappropriate FOBT or Screening Colonoscopy by Reason Type, Controlling for Patient Characteristics*
Full size table

Compared to low-complexity facilities, screening colonoscopies conducted at high-complexity facilities were more likely to be classified as “potentially inappropriate–limited life expectancy” (OR = 1.37, p < 0.001). Facilities with CRC screening rates in the upper third were more likely than those scoring in the lower third to conduct colonoscopies before patients were due for screening (OR = 1.34, p = 0.03). Facilities in the top third on our measure of relative use of FOBT were less likely to conduct potentially inappropriate FOBTs (OR = 0.78, p = 0.003), but more likely to conduct potentially inappropriate colonoscopies (OR = 2.20, p < 0.001) compared to facilities scoring in the bottom third. This relationship between FOBT use and potentially inappropriate screening was consistent across all reasons. Facilities that assigned primary care providers partial or full responsibility for completing the CR were less likely to conduct potentially inappropriate screening colonoscopies for any reason than facilities that assigned sole responsibility for completing the CR to intake nurses (OR = 0.73 to 0.74, p’s = 0.04), but no other CR attributes were significantly associated with overall rates of potentially inappropriate screening.

DISCUSSION

In a cohort of over one million VHA patients receiving CRC screening, we categorized 26 % of FOBT cases and 14 % of screening colonoscopy cases as potentially inappropriate. These estimates amount to approximately 104,600 potentially inappropriate FOBTs and 8,500 potentially inappropriate screening colonoscopies annually in the VHA. In addition, our results suggest that approximately 7,200 patients who are not yet due for screening and 5,000 patients with limited life expectancies have positive FOBTs, which could lead to over 12,000 additional colonoscopies annually.

All inappropriate screening tests waste screening resources. Additional negative effects may occur, depending on the screening mode, result of the test, and the reason the test was inappropriate. For example, FOBTs that are conducted before a patient is due compress the time interval between screening tests and therefore increase the lifetime risk of a false positive test. This in turn exposes patients to additional risk of complications from unnecessary diagnostic colonoscopies. Patients with limited life expectancies that complete screening tests expose themselves to the risk of being diagnosed and treated for cancer even when that cancer may have never caused morbidity. FOBTs that are conducted when a colonoscopy is indicated not only waste lab and primary care staff resources, but also increase the risk of delayed or missed cancer diagnoses.

Rates of potentially inappropriate screening in our sample are comparable, and in some cases lower, than those found in other studies. A VHA study examining FOBT overuse in 24 facilities between 2004 and 2009 reported that 21 % of FOBTs were completed too soon after a prior screening test15 (compared to 14 % observed in > 100 VHA facilities between 2009 and 2011 in our study). A large-scale review of screening colonoscopies conducted in 11 countries classified 14 % of procedures as inappropriate,29 which is equivalent to the proportion we report, but two US studies examining the appropriateness of colonoscopy referrals (both screening and non-screening) classified 19 to 28 % as inappropriate.30,31 Two single-site VHA reviews of patients with a positive FOBT in 1999 and 2004 found that 35 to 41 % of these patients should either have received a colonoscopy instead of FOBT or should not have been screened at all11,12 (compared to 26 % classified as potentially inappropriate for any reason in our more recent nationally representative VHA sample). Another VHA study using data collected from four facilities between 2001 and 2002 found that 41 % of patients with a life expectancy of 5 years or less completed a CRC screening test21 (compared to 7 % classified as limited life expectancy in our nationally representative study using more recent data and a more conservative definition of limited life expectancy). Differences in estimates across studies may be due to temporal and regional variation in inappropriate screening rates, or variation in methodology and definitions of inappropriate screening.

Healthcare systems have adopted a variety of strategies to promote screening in recent years. These strategies may powerfully influence rates of inappropriate screening either positively or negatively. Our study adds to the current research by identifying facility processes that correlate with potentially inappropriate screening and by comparing facilities that rely primarily on FOBT to those that use a more even mix of FOBT and colonoscopy. These analyses offer insight into how programs might be refined to minimize inappropriate screening. At the facilities where the highest proportion of screening was done with FOBT, the proportion of potentially inappropriate FOBTs was relatively small, but the proportion of potentially inappropriate screening colonoscopies was relatively large. Thus, healthcare systems may benefit from examining their processes to safeguard against inappropriate use of screening modes that are not the focus of their screening program. We also found that facilities where the primary care provider was assigned partial or full responsibility for completing the CRC clinical reminder were less likely to conduct potentially inappropriate colonoscopies, but were no less likely to conduct inappropriate FOBTs. We suspect that the reminder helps providers maintain appropriate intervals between colonoscopies and facilitates the identification of patients who are of limited life expectancy. Providers who do not interact with the reminder may use less effective processes to manage colonoscopy screening decisions.

Tests on older patients were more likely to be classified as potentially inappropriate across all reasons. As patients age, additional efforts to triage screening candidates may therefore be needed.

Our study has several limitations. First, our findings may not generalize to healthcare systems with screening rates that are lower than those of the VHA. It is noteworthy, however, that the Centers for Disease Control and Prevention have set a 2014 goal of achieving a nation-wide screening rate of 80 %.32 Our findings may become increasingly relevant as provider organizations strive to meet this goal. Second, our classification of tests as potentially inappropriate is subject to error. A definitive determination of appropriateness requires a thorough review of each patient’s individual circumstances. Third, there was some imprecision in the algorithm used to identify and exclude non-screening colonoscopies. This algorithm had high specificity but somewhat lower sensitivity, resulting in the exclusion of some screening colonoscopies. Nonetheless, the reliability of our algorithm was substantially higher than that reported in prior work using administrative data.27,28 Fourth, procedures and diagnoses obtained from non-VHA providers are not represented in our data set. Fifth, because we selected only one index test per patient, our results do not allow for the possibility that some patients may have been inappropriately screened multiple times. Because of these last three limitations, the rates of potentially inappropriate screening we report may be somewhat underestimated.

In spite of these limitations, our research demonstrates that, within the largest integrated healthcare system in the US, a substantial number of CRC screening tests may be inappropriate. Inappropriate screening may be partially addressed by establishing organizational processes that enforce appropriate screening intervals, triage patients with limited life expectancies, and discourage the use of FOBT to investigate gastroenterological symptoms.