Methods to Estimate the Comparative Effectiveness of Clinical Strategies that Administer the Same Intervention at Different Times
 759 Downloads
 2 Citations
Abstract
Clinical guidelines that rely on observational data due to the absence of data from randomized trials benefit when the observational data or its analysis emulates trial data or its analysis. In this paper, we review a methodology for emulating trials that compare the effects of different timing strategies, that is, strategies that vary the frequency of delivery of a medical intervention or procedure. We review trial emulation for comparing (i) single applications of the procedure at different times, (ii) fixed schedules of application, and (iii) schedules adapted to the evolving clinical characteristics of the patients. For illustration, we describe an application in which we estimate the effect of surveillance colonoscopies in patients who had an adenoma detected during the Norwegian Colorectal Cancer Prevention (NORCCAP) trial.
Keywords
Causal inference Dynamic strategies Inverse probability weighting Colorectal cancer Surveillance ColonoscopyIntroduction
Clinical decisions are increasingly reliant on guidelines, but clinical guidelines are only as good as the available evidence on the comparative effectiveness of interventions [1•]. Ideally, such evidence would come from randomized controlled trials. When a randomized trial is not available, it may be possible to emulate it using observational data [2•]. This approach requires appropriate confounding adjustment, avoidance of selection bias in the definition of the groups to be compared, and formulation of a research question that is relevant for decision makers.
Prior explicit attempts to emulate trials using observational data have studied, for example, postmenopausal hormone therapy [3], statins [4••], epoetin [5••], and antiretroviral therapy [6••]. Here, we review the emulation of trials to compare strategies that differ in the timing of the intervention of interest. As an example, we will consider postpolypectomy surveillance by colonoscopy. During this procedure, adenomas (benign tumors of the colon [7]) are detected and removed. Most adenomas will not develop into colorectal cancer, but most cancers arise from adenomas [8]. In patients with removed adenomas, surveillance colonoscopies are recommended to detect and remove future adenomas before they become malignant. The optimal interval between colonoscopies is not known. Current guidelines both in the USA [9] and the EU [10] are mostly based on expert opinion due to the scarcity of available evidence.
Besides reviewing a methodology to emulate trials for the comparison of strategies that administer the same intervention at different times, we also review a classification of these strategies. First, we consider point interventions to study the effectiveness of a single application of the treatment. Second, we consider sustained interventions to study the effectiveness of a fixed treatment schedule (e.g., colonoscopy at 3 years after the initial procedure). Third, we consider sustained interventions to study the effectiveness of a personalized schedule of treatment (e.g., colonoscopy every year if the most recent procedure detected large adenomas, otherwise every 3 years). To fix ideas, we review the methodology in the context of its implementation to a cohort of Norwegian individuals. We start by describing this cohort.
Data
The Norwegian Colorectal Cancer Prevention (NORCCAP) screening study was a randomized clinical trial of onceonly sigmoidoscopy screening versus no sigmoidoscopy, conducted in Oslo and Telemark counties in Norway between 1999 and 2001. Our analysis includes participants in the sigmoidoscopy arm in whom at least one adenoma was detected (n = 2190). As part of the trial, endoscopies were conducted in these individuals until the bowel was free from adenomas. We excluded patients with history of serious gastrointestinal disease, known genetic predisposition to colorectal cancer, and cancer detected as a result of screening in NORCCAP.
In addition to the available data (age, sex, county, smoking, family history of colorectal cancer, and findings at NORCCAP colonoscopies), we conducted a manual chart review at all hospitals in Oslo and Telemark—guided by claims data from the governmental singlepayer agency HELFO—to collect data on the date, findings (e.g., size and type of adenomas) and indication of all subsequent colonoscopies and sigmoidoscopies. Of the postscreening endoscopies, 64 % were for surveillance purposes (3 % sigmoidoscopies and 61 % colonoscopies); 30 % were clinically indicated because of symptoms (27 % colonoscopies, 3 % sigmoidoscopies); and 6 % were due to a recent incomplete endoscopy (4 % colonoscopies, 2 % sigmoidoscopies).
Our outcome of interest was incidence of colorectal cancer. For many surveillance interventions, the use of cancer incidence as an outcome is questionable because of potential lead time bias: [11] cancer cases will be detected earlier in patients with more intensive surveillance, which will make surveillance appear less beneficial. In this case, however, the use of the outcome cancer incidence is justified because most of the beneficial effect of surveillance colonoscopy seems to be due to removing adenomas before they become malignant [12], with only a small component of the effect due to earlier detection of prevalent cancer. Death from colorectal cancer could not be studied as an outcome because there were too few cases.
Characteristics of 2190 eligible individuals from the intervention arm of the NORCCAP trial
Number of men  1322 (60 %) 
Average (SD) age at first eligibility, years  57.2 (3.8) 
Median (IQR) duration of followup, months  134 (126–143) 
Incident cases of colorectal cancer  21 
Detected at surveillance colonoscopy  1 
Deaths  187 
From colorectal cancer  5 
Number of colonoscopies during followup  819 
Number of sigmoidoscopies  75 
Number of people with at least one colonoscopy after first eligibility  577 
Number of people whose first followup colonoscopy was for surveillance  395 
Median (IQR) time to first followup colonoscopy, months  68 (51–91) 
Number of colonoscopies per individual  
0  1613 (74 %) 
1  389 (18 %) 
2  140 (6 %) 
3+  48 (2 %) 
Three Hypothetical Randomized Trials
The design of any trial is determined by the causal question of interest, which in turn is determined by the population, the strategies being compared, and the outcome of interest to the decision makers [13]. For surveillance tests, the strategies are defined by the timing of the test. Some strategies involve a point intervention at baseline, whereas other strategies involve interventions that are sustained over time according to either a fixed schedule (e.g., do not perform a colonoscopy for 5 years after baseline, then perform a colonoscopy at the end of year 5) or a schedule that depends on each individual’s timeevolving clinical characteristics (i.e., schedule the time of every colonoscopy according to the findings at the previous colonoscopy). We refer to sustained strategies with a fixed schedule as static and to those with a subjectspecific schedule as dynamic.
Trial type #1: point interventions assigned at a fixed time after first eligibility
Individuals who survived 36 months since first eligibility are randomized to either (1) immediate surveillance colonoscopy or (2) no surveillance colonoscopy. Additional eligibility criteria are no colorectal cancer, colonoscopy, or sigmoidoscopy during the 36 months before randomization. Individuals who reach age 70 or develop any invasive noncolorectal cancer before baseline also become ineligible (other comorbidities might be added to the exclusion criteria). For each individual, followup starts at the time of randomization, i.e., baseline is 36 months after first eligibility.
More generally, one can consider trials in which baseline is month z, where z ranges between 36 and 84. The effect estimates from these trials will only apply to survivors without symptoms or cancer by z months after first eligibility. These trials will help determine the effect of undergoing a colonoscopy among the survivors, but it does not directly inform the decision of when to undergo the colonoscopy. The next trial does so.
Trial type #2: sustained static strategies assigned at first eligibility
Baseline is the time of first eligibility. Individuals are randomized to either (1) surveillance colonoscopy 36 months after baseline or (2) surveillance colonoscopy 84 months after baseline. Individuals in both arms who reach age 70 or develop malignancies other than colorectal cancer may have surveillance colonoscopies at any time as determined by their physician. More generally, one can consider additional arms in which 36 is replaced by any value of x between 36 and 84. We could also consider similar trials in which baseline is any month after first eligibility. For example, one could consider a trial in which individuals who have survived 36 months after first eligibility are randomized to either (1) immediate surveillance colonoscopy or (2) surveillance colonoscopy at month 84 after first eligibility (48 months after baseline at 36 months). We will only consider trials with baseline at first eligibility.
Both trial types #1 and #2 compare fixed surveillance schedules, but they address different questions. Trial #1 helps individuals who have survived z months after adenoma removal decide whether they should undergo a surveillance colonoscopy at that time. Trial #2 helps individuals who just had their adenomas removed decide how long they should wait before having a surveillance colonoscopy (if they plan to have only one surveillance colonoscopy). Neither trial type considers strategies that assign different surveillance schedules to different individuals (i.e., dynamic strategies). The next trial type does so.
Trial type #3: sustained dynamic strategies assigned at first eligibility

First surveillance colonoscopy at 36 months if the adenomas detected at baseline sigmoidoscopy were low risk (1 or 2 small adenomas without villous features) and 12 months earlier (at month 24) otherwise.

Followup surveillance colonoscopy 36 months after the previous colonoscopy (surveillance or clinical) if lowrisk adenomas were detected, 12 months earlier (24 months after the previous colonoscopy) if highrisk adenomas (more than two, or large, or containing villous features) were detected, and 12 months later (48 months) if no adenomas were detected.
More generally, one can consider additional arms in which 36 is replaced by x with x ranging from 36 to 84, or trials in which the time until the next surveillance colonoscopy is obtained by adding or subtracting y (rather than 12) months.
Emulating the Design of the Hypothetical Trials
In this section, we review how to emulate the design of each of the above hypothetical trials by setting up a database with the same structure as that of the trial. In the next section, we review how to mimic the analysis of the hypothetical trials.
Trial type #1: point intervention assigned at a fixed time after first eligibility
We emulated 49 “trials,” one starting at each month z between months 36 and 84 after first eligibility. For the “trial” starting in month z, we identified the individuals who met the eligibility criteria at baseline, i.e., all individuals with adenomas detected and removed at first eligibility who were alive and had not yet had a postscreening colonoscopy/sigmoidoscopy or been diagnosed with colorectal cancer by z months of followup. For each trial, individuals were classified into the colonoscopy arm if they received a colonoscopy during month z and into the control arm otherwise.
We identified 2028 eligible individuals. On average, each participated in 45 trials, of which at most 1 was in the colonoscopy arm. The number of eligible individuals who received a colonoscopy at baseline ranged between 0 (in several trials) and 16 (in trial z = 61). See Appendix Table 3 for details. Unfortunately, all trials had zero cancers among the exposed, which means the data from NORCCAP cannot be used for a meaningful emulation of trial type #1.
Trial type #1 has the advantage of being easy to emulate and analyze when sufficient observational data are available. This approach has been used in observational studies to estimate the observational analog of the intentiontotreat effect of statin therapy [4••] and postmenopausal hormone therapy [3]. Here, we will not consider this trial type further.
Trial type #2: sustained static strategies assigned at first eligibility
We emulated a randomized trial with 49 arms, in which the participants were assigned at first eligibility to colonoscopy at a randomly assigned time ranging from month 36 to 84 after first eligibility. Classifying the 2190 eligible individuals into a single arm is not possible because, at baseline, each individual’s data are consistent with all 49 arms. To overcome this problem we created an expanded dataset with 49 clones of each individual, and assigned each of them to a different arm [14]. The 2190 eligible subjects contributed 107,309 clones to this trial. See Appendix Table 4 for details.
The clones in the expanded dataset were censored at the time their data deviated from the strategy to which they were assigned. For example, in arm 84, 12.9 % of participants were censored for having a surveillance colonoscopy too early (before month 84), 73.5 % of participants were censored for failing to have a surveillance colonoscopy in time (in month 84), and 0.5 % were censored for having a sigmoidoscopy. Those who received a colonoscopy for clinical reasons or developed malignancies other than colorectal cancer were subsequently considered “immune” from censoring.
Trial type #3: sustained dynamic strategies assigned at first eligibility
We emulated a trial with 49 arms, one for each value of x in the dynamic strategies defined above. The 2190 individuals were classified into the arm that was consistent with their observed data. Like in the previous trial, individuals cannot be assigned to a single arm at baseline, so we created an expanded dataset with 49 clones of each individual and assigned each of them to a different arm. The clones were censored at the time they deviated from the strategy to which they were assigned. For example, in arm 84, 11.3 % of participants were censored for having a surveillance colonoscopy too early, 79.7 % of participants for failing to have a surveillance colonoscopy in time, and 1.3 % for having a sigmoidoscopy. The 2190 eligible subjects contributed 107,309 clones to this trial. See Appendix Table 5 for details.
Emulating the Design of Hypothetical Trials with a Grace Period
So far, we have implicitly assumed that it is possible to administer a colonoscopy at a precisely specified time point, e.g., month 36. However, in many clinical settings, this may not be feasible. We may therefore be more interested in emulating trials with a grace period, that is, a window of m months during which the patient may undergo colonoscopy. For example, in trial type #2, patients would be assigned to interventions of the form “surveillance colonoscopy between x and x + m months after baseline.” Trials with a grace period more accurately reflect clinical practice in which administrative delays and patient availability may prevent an immediate intervention.
Strategies with a grace period are emulated using “clones” as described above, but with different criteria for censoring. Suppose we use a grace period of m = 6 months. An individual who received a surveillance colonoscopy in month 40 now has data consistent with arm 36 because subjects assigned to this arm are allowed to have a colonoscopy at any time between months 36 and 42. Therefore, his clones assigned to arms 36 to 40 will not be censored whereas his clones assigned to arm 41 will be censored because he received a surveillance colonoscopy before the assigned time.
The addition of a grace period requires us to specify the distribution of the interventions during the grace period. For example, we might ask whether most colonoscopies are performed during the first 2 months of the grace period or whether they are more equally distributed during the grace period. In our application, we will specify a uniform distribution of colonoscopies during the grace period [14].
In both trials #2 and #3 with a 6month grace period, each of the 2190 eligible individuals in the original dataset contributed 49 clones, for a total of 107,310 clones to the expanded dataset. In trial #2, the average censoring time ranged between 41.9 months for x = 36 to 89.1 months for x = 84. In arm 84, 12.9 % of participants were censored for having a surveillance colonoscopy too early (before month 84), 71.5 % of participants were censored at month 90 for failing to have a surveillance colonoscopy in time, 0.1 % were censored after month 90 for having a second surveillance colonoscopy, and 0.6 % were censored for having a sigmoidoscopy. Across the 49 arms, there were 381 incident cases of colorectal cancer in the clones, which occurred in 12 unique individuals.
In trial #3, the average censoring time ranged from 34.2 months for x = 36 to 78.1 months and for x = 84. For arm 84, 11.3 % of participants were censored for having a surveillance colonoscopy too early, 77.6 % for failing to have a surveillance colonoscopy in time, and 1.4 % for having a sigmoidoscopy. In total, there were 254 incident cases of colorectal cancer in 13 unique individuals. See Appendix Tables 4 and 5 for details.
Emulating the Analysis of the Hypothetical Trials
After reviewing how to create observational databases with the same structure as hypothetical randomized trials, we review how to use those databases to estimate the cumulative incidence curves (or their complement, the survival curves) that would have been observed under each strategy if all individuals had fully adhered to their original arm assignment. In a slight abuse of notation, we index the strategies by the variable x, which was defined in the previous sections. For example, in trial #2, x = 78 corresponds to the strategy “surveillance colonoscopy between 78 and 78 + 6 months after baseline.”
In a true randomized trial with many arms x, we could estimate these curves nonparametrically (KaplanMeier curves) or parametrically by fitting a pooled logistic model of the form \( logit\; \Pr \left({Y}_{t+1}=0\Big{Y}_t={D}_t=0,\;x\right)={\alpha}_{0,t}+{\alpha}_1f(x)+{\alpha}_2f(x)\times t \), where t denotes time (in months), Y _{ t } is an indicator of colorectal cancer by t, D _{ t } is an indicator of death by t, \( {\alpha}_{0,t} \) is a timevarying intercept (estimated, for example, via restricted cubic splines for time with knots at 30, 60, 90, and 120 months), \( f(x) \) is a function of x (for example, a second degree polynomial), and \( f(x)\times t \) is a product term to allow the hazard ratio to vary during the followup. For example, for the first 36 months of followup, the hazard is known to be identical under all strategies, but it may change after that if colonoscopy has a nonnull effect on colorectal cancer incidence.
We would then calculate the predicted values for each value of x and compute their product in order to estimate the survival curves. Pointwise 95 % confidence intervals for the curves can be obtained via a nonparametric bootstrap. In our emulated trials, however, the above logistic model needs to be adjusted by both baseline and postbaseline (timevarying) confounders. The procedure then needs to be modified as we now describe.
Adjustment for Covariates
In both trials # 2 and #3, we need to adjust for covariates that jointly predict surveillance colonoscopy A _{ t } (and therefore censoring) and subsequent outcome. Some of these variables are fixed at the baseline of each trial; others vary during the followup. Let L _{0} represent the vector of baseline covariates, which include age at baseline, sex, family history of colorectal cancer, history of smoking, and findings at NORCCAP colonoscopies (number of adenomas, size, histology, and presence of villous elements). Let L _{ t } represent the vector of timevarying covariates, which include an indicator for incident noncolorectal malignancies, and a vector of the findings from the most recent colonoscopy (number of adenomas, size of largest adenoma, histological grade, and presence of villous elements).
To adjust for L _{0}, one could fit the pooled logistic model \( logit \Pr \left({Y}_{t+1}=0\Big{Y}_t=0,x,{L}_0\right)={\alpha}_{0,t}+{\alpha}_1f(x)+{\alpha}_2f(x)\times t+{\alpha}_3{L}_0 \) to the expanded dataset of each trial separately. To obtain the survival curves under each strategy x, one would then calculate the predicted values for each value of x, standardized them by L _{0}, and compute their product. However, the timevarying covariates L _{ t } cannot be added to the logistic model because these variables may be affected by prior treatment [10, 11] (a colonoscopy may change the findings at future colonoscopies, for example by removing adenomas; see Appendix). We therefore need to use IP weighting to adjust for L _{ t }.
The subjectspecific, timevarying IP weights are \( {W}_t=\prod_{j=0}^t\frac{1}{f\left({A}_j{\overline{A}}_{j1},\;{\overline{L}}_j,\;{Y}_j={D}_j=0\right)} \). Informally, the denominator of the weights is each subject’s conditional probability of having, at each time t, his or her own surveillance colonoscopy history. We use overbars to denote history, i.e., \( {\overline{L}}_t= \) (L _{0}, L _{1}, L _{2}, …, L _{ t }).
The factors in the denominator of the weights were set to 1 in months following age 70, a nonsurveillance colonoscopy, or the diagnosis of malignancies other than colorectal cancer because the individual has a probability 1 of remaining uncensored during those months. The factors in the denominator were also set to 1 during the first 9 months after a colonoscopy is received, because no surveillance colonoscopies were performed during this period (only colonoscopies due to symptoms or to incompleteness of the preceding colonoscopy). In previous applications of IP weighting for strategies with grace periods, the investigators were interested only in strategies that were not sustained beyond the initial decision to treat [14]. Therefore, the contributions to the weights were set to 1 for all time periods after treatment was first received.
Because the IP weights already adjusted for the baseline covariates L _{0}, we did not include them as covariates in the outcome model. That is, we fit the weighted pooled logistic model \( logit\kern0.2em \Pr \left({Y}_{t+1}=0\Big{Y}_t=0,\;x\right)={\alpha}_{0,t}+{\alpha}_1f(x)+{\alpha}_2f(x)\times t \). To check the robustness of our estimates to different choices of functional form for time and x, we explored different parameterizations of the outcome model, including a quadratic functional form for time, cubic terms for x, and additional interaction terms between f(x) and time.
Grace Period
Because our strategies of interest include grace periods, the abovementioned IP weights W _{ t } need to be modified [14]. Specifically, the numerator of the factors corresponding to months included in the grace period need to change to ensure that surveillance colonoscopies will be uniformly distributed during the grace period. For trial #2, the numerator of factors corresponding to month j of the grace period is replaced by \( \frac{1}{m+1j} \) with j = 0, 1, … 5 when A _{ t } = 1, and replaced by \( \frac{mj}{m+1j} \) when A _{ t } = 0. For trial #3, where there can be multiple surveillance colonoscopies, we use the same approach during all grace periods.
Estimates from NORCCAP Data
Estimated risk of colorectal cancer at 5 and 10 years under selected surveillance strategies, intervention arm of the NORCCAP trial
Risk, % (95 % CI) x = 36  Risk, % (95 % CI) x = 84  Risk difference, % (comparing x = 36 with x = 84) (95 % CI)  Risk ratio (comparing x = 36 with x = 84) (95 % CI)  

Static strategies  
At 5 years  0.15 (0.03–0.37)  0.30 (0.08–0.59)  −0.15 (−0.31–0.00)  0.47 (0.06–0.87) 
At 10 years  0.31 (0.05– 0.69)  0.63 (0.27–1.14)  −0.32 (−0.67–0.01)  0.49 (0.10–1.01) 
Dynamic strategies  
At 5 years  0.12 (0.00–0.36)  0.25 (0.01–0.50)  −0.13 (−0.30–0.01)  0.49 (0.03–1.18) 
At 10 years  0.30 (0.05–0.90)  0.44 (0.17–0.76)  −0.14 (−0.46–0.03)  0.67 (0.10–1.76) 
Note that had the dataset included no cancer diagnoses after surveillance colonoscopy, the conclusion that delaying colonoscopy increases risk would be foregone. In our dataset, only one individual who has a surveillance colonoscopy between months 36 and 84 subsequently developed colorectal cancer, and he was censored before getting cancer under most clinically relevant strategies. Any changes to the strategies that led to him not being censored would result in substantial changes to the estimates. Therefore, our analysis needs to be replicated in a larger dataset.
Conclusions
After a medical procedure or medication has been shown to be effective, the next question is usually how often it should be administered. In this paper, we reviewed an approach that, when applied to a sufficiently large and rich dataset, helps decide among various timing strategies. Specifically, we outlined the design and analysis of hypothetical randomized trials to compare different strategies, and provided a methodology for emulating these trials using observational data.
As a motivating example, we compared the effectiveness of different strategies for scheduling surveillance colonoscopies in patients with adenomas, a clinical question for which the available evidence is sparse [9, 15, 16, 17, 18, 19, 20]. Our analysis suggests that more frequent surveillance colonoscopies leads to a greater reduction in colorectal cancer risk; as expected, the analysis also suggests that dynamic strategies are more effective than static strategies. However, our analysis is more an example of implementation than an attempt at providing definite answers to the clinical question because the sample size of our study was small.
The application of the methods outlined in this review allowed us to specify a research question that is directly relevant to decision makers interested in timing questions. Though these methods allow adjustment for both baseline and timevarying covariates, the possibility of unmeasured confounding remains as in any observational study.
Notes
Compliance with Ethics Guidelines
Conflict of Interest
Anders Huitfeldt declares that he has no conflict of interest.
Mette Kalager declares that she has no conflict of interest.
James M. Robins declares that he has no conflict of interest.
Geir Hoff declares that he has no conflict of interest.
Miguel A. Hernán declares that he has no conflict of interest.
Human and Animal Rights and Informed Consent
All studies by A Huitfeldt, M Kalager, JM Robins, G Hoff, and MA Hernàn involving human subjects were performed after approval by the appropriate institutional review boards. When required, written informed consent was obtained from all participants.
Funding
NIH grant P01 CA134294
References
Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance
 1.•Institute of Medicine. Ethical and scientific issues in studying the safety of approved drugs. Washington, DC: The National Academies Press; 2012. This document uses the concept of observational studies as attempts to emulate randomized trials for drug safety.Google Scholar
 2.•Hernán MA. With great data comes great responsibility: publishing comparative effectiveness research in epidemiology. Epidemiology (Cambridge, Mass). 2011;22(3):290–1. This editorial discusses the need to frame research questions as hypothetical randomized trials in order to make them directly relevant for decision making.CrossRefGoogle Scholar
 3.Hernan MA, Alonso A, Logan R, et al. Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease. Epidemiology. 2008;19(6):766–79.PubMedCentralPubMedCrossRefGoogle Scholar
 4.••Danaei G, Rodriguez LA, Cantero OF, Logan R, Hernan MA. Observational data for comparative effectiveness research: an emulation of randomised trials of statins and primary prevention of coronary heart disease. Stat Methods Med Res. 2013;22(1):70–96. An explicit attempt to emulate a trial with static strategies.PubMedCentralPubMedCrossRefGoogle Scholar
 5.••Zhang Y, Thamer M, Kaufman J, Cotter D, Hernán MA. Comparative effectiveness of two anemia management strategies for complex elderly dialysis patients. Med Care. 2014;52 Suppl 3:S132–9. An explicit attempt to emulate a trial with dynamic strategies.PubMedCentralPubMedCrossRefGoogle Scholar
 6.••Cain LE, Logan R, Robins JM, et al. When to initiate combined antiretroviral therapy to reduce mortality and AIDSdefining illness in HIVinfected persons in developed countries: an observational study. Ann Intern Med. 2011;154(8):509–15. This paper develops the theory of inverse probability weighted estimators for dynamic strategies, and also discusses the necessity of grace periods.PubMedCrossRefGoogle Scholar
 7.Vatn MH, Stalsberg H. The prevalence of polyps of the large intestine in Oslo: an autopsy study. Cancer. 1982;49(4):819–25.PubMedCrossRefGoogle Scholar
 8.Winawer SJ, Fletcher RH, Miller L, et al. Colorectal cancer screening: clinical guidelines and rationale. Gastroenterology. 1997;112(2):594–642.PubMedCrossRefGoogle Scholar
 9.Lieberman DA, Rex DK, Winawer SJ, Giardiello FM, Johnson DA, Levin TR. Guidelines for colonoscopy surveillance after screening and polypectomy: a consensus update by the US multisociety task force on colorectal cancer. Gastroenterology. 2012;143(3):844–57.PubMedCrossRefGoogle Scholar
 10.Atkin WS, Valori R, Kuipers EJ, et al. European guidelines for quality assurance in colorectal cancer screening and diagnosis. First edition—colonoscopic surveillance following adenoma removal. Endoscopy. 2012;44 Suppl 3:SE151–63.PubMedGoogle Scholar
 11.Prorok PC, Connor RJ, Baker SG. Statistical considerations in cancer screening programs. Urol Clin North Am. 1990;17(4):699–708.PubMedGoogle Scholar
 12.Winawer SJ, Zauber AG, Ho MN, et al. Prevention of colorectal cancer by colonoscopic polypectomy. The National Polyp Study Workgroup. N Engl J Med. 1993;329(27):1977–81.PubMedCrossRefGoogle Scholar
 13.Sackett DL, Richardson WS, Rosenberg W, Haynes RB. Evidencebased medicine—how to practice and teach EBM. New York: Churchill Livingstone; 1997. Google Scholar
 14.Cain LE, Robins JM, Lanoy E, Logan R, Costagliola D, Hernan MA. When to start treatment? A systematic approach to the comparison of dynamic regimes using observational data. Int J Biostat. 2010;6(2):Article 18.Google Scholar
 15.Jørgensen OD, Kronborg O, Fenger C. A randomized surveillance study of patients with pedunculated and small sessile tubular and tubulovillous adenomas. The Funen Adenoma Followup Study. Scand J Gastroenterol. 1995;30(7):686–92.PubMedCrossRefGoogle Scholar
 16.Kronborg O, Jørgensen OD, Fenger C, Rasmussen M. Three randomized longterm surveillance trials in patients with sporadic colorectal adenomas. Scand J Gastroenterol. 2006;41(6):737–43.PubMedCrossRefGoogle Scholar
 17.Lund JN, Scholefield JH, Grainge MJ, et al. Risks, costs, and compliance limit colorectal adenoma surveillance: lessons from a randomised trial. Gut. 2001;49(1):91–6.PubMedCentralPubMedCrossRefGoogle Scholar
 18.Lieberman DA, Weiss DG, Harford WV, et al. Fiveyear colon surveillance after screening colonoscopy. Gastroenterology. 2007;133(4):1077–85.PubMedCrossRefGoogle Scholar
 19.Winawer SJ, Zauber AG, Gerdes H, et al. Risk of colorectal cancer in the families of patients with adenomatous polyps. National Polyp Study Workgroup. N Engl J Med. 1996;334(2):82–7.PubMedCrossRefGoogle Scholar
 20.Von Karsa L, Segnan N, Patnick J. European guidelines for quality assurance in colorectal cancer screening and diagnosis. 2010.Google Scholar