Introduction

Cost-effectiveness analyses (CEAs) in health care research and public health are considered an important tool to help decision-makers to set funding priorities [1, 2]. CEA can be defined as the comparative analysis of alternative courses of action in terms of both their costs and consequences and is designed to improve health [3]. Exploring the cost-effectiveness of a behavioral health intervention, however, has some methodological implications compared to pharmaceutical interventions. Behavioral interventions encourage individuals to modify their existing behavior and to adopt a healthier behavior. CEAs of behavioral interventions typically use a simple dichotomous (success or failure) outcome criterion [4]. In reality, though, behavioral change is a complex process in which several steps towards success are taken. As most intervention studies have a relatively short follow-up period of 6–12 months, it is likely that effects are achieved after the follow-up period. In fact, any progress in behavioral change without accomplishing full behavioral change may also be considered as a beneficial outcome of the intervention [5]. Not accounting for ‘delayed’ behavioral change may lead to underestimation of effectiveness of behavioral interventions [6–9]. Obviously, extending the follow-up period would be the preferred way to address this issue. However, this is often impeded by practical and financial limitations. An alternative may be to use intermediate outcome measures to model future behavioral change. In their review on this topic, Wagner and Goldstein [4] stated that analysts who conduct a CEA of a behavioral intervention should not focus solely on people who changed their behavior successfully, but they also need to measure partial behavioral change. They conclude that failing to include partial behavioral change in the CEA can bias the results. Studies on interventions that collect stages of change data (e.g., the Transtheoretical model of behavioral change [10]), for example, enable the measurement of partial behavioral change and the subsequent incorporation of these as intermediate outcomes into CEAs. Also, non-stage-based psychological theories can provide measures of partial behavioral change, such as the Theory of Planned Behavior from Ajzen [11] and Bandura’s Social Cognitive Theory [12].

The Transtheoretical model of behavioral change is a stage-oriented model that describes readiness to change [13]. Beginning in 1977, Prochaska and colleagues developed the model, based on an analysis of different theories of psychotherapy. Nowadays, it has been widely adopted for numerous health behaviors [10]. A number of qualitatively different, discrete states, the ‘stages-of-change’, are key constructs of the Transtheoretical model. It provides an algorithm that distinguishes 6 stages; the focus of this study is on the first three ‘pre-action’ stages: (1) precontemplation (e.g., no intention to quit smoking within the next 6 months); (2) contemplation (e.g., intending to quit smoking within the next 6 months); and (3) preparation (e.g., intending to quit smoking within the next 30 days) [10]. The stage algorithm has been developed on the basis of empirical findings [14]. These pre-action stages provide probabilities for the actual transition to the fourth stage, the ‘action stage’, in which full behavioral change is achieved. The other 2 stages are the ‘maintenance stage’ (in which people changed their behavior more than 6 months ago) and the ‘termination stage’ (in which people have achieved maintenance and no longer experience any temptations and have full self-control; people may never enter this stage). Usually, attempts to modify (addictive) behavior are not immediately successful. With smoking, for example, successful quitters make an average of 3 to 4 attempts and go through a spiral pattern of several cycles before they reach long-term abstinence. Relapse and recycling through the stages therefore occur quite frequently as individuals attempt to modify or cease addictive behaviors [10].

Modeling of partial to future behavioral change has been applied previously in the CEA literature. For example, Tengs et al. [15] created the ‘Tobacco Policy Model’ to estimate cost-effectiveness of school-based anti-tobacco education over a person’s life-time [15]. They defined and simulated successfully 3 changes in smoking behavior using a Markovian computer simulation model: The transition from never smoked to being a current smoker (initiation), from current to former smoker (cessation) and from former to current smoker (relapse). Also for public health modeling, Mulder et al. [16] applied changes in smoking status to predict future mortality reduction through smoking cessation. These are behavioral intermediate outcomes. An alternative may be to use cognitive parameters. Cognitive parameters are the antecedents of actual behavioral change, as reported in several behavioral theories in literature (e.g., Transtheoretical model [13], Theory of Planned Behavior [11]).

The aim of this study is to model cognitive parameters into a final cost-effectiveness model of a behavioral intervention to gain more insight into the feasibility and the challenges involved with this method. For this purpose, we used an existing dataset and replicated the CEA with addition of partial behavioral change estimates, based on the stages-of-change algorithm.

Methods

Sample

Data from the SMOKE study [17, 18] were used. The SMOKE study is a randomized controlled multi-centre trial with 1 year follow-up that evaluated the (cost-) effectiveness of the Smoke Stop Therapy (SST) and the Minimal Intervention Strategy for Lung patients (LMIS). A total of 234 COPD patients motivated to quit smoking (checked by their own chest physician) were included in the SMOKE study and randomly assigned: 117 received the LMIS and 117 patients received the SST. Inclusion criteria were clinically diagnosed moderate COPD (% predicted FEV1 = 50–69) or severe COPD (% predicted FEV1 <50) as defined by the American Thoracic Society (ATS) criteria [19], willingness to participate in a smoking cessation program, aged between 40 and 75 years, and adequate knowledge and understanding of the Dutch language. The only exclusion criterion was a counter indication for the use of Bupropion (Zyban®). The chest physician advised each smoking COPD patient to quit smoking and, after providing informed consent, the patient was referred to the SMOKE study. A total of 9 patients dropped out after giving informed consent: 6 from the LMIS and 3 from the SST. At baseline, another 15 patients dropped out: 6 from the LMIS and 9 from the SST. These latter patients were excluded from all analyses. In both conditions, 105 patients remained for analyses. All missing patients at 12 months follow-up were assumed to be smokers. All remaining patients adhered to the counseling sessions.

Baseline characteristics are presented in Table 1. Three baseline differences were found between groups. Patients receiving LMIS were older than those receiving the SST (P < .05). Nicotine dependence, as measured by the Fagerström questionnaire, was significantly stronger in the participants allocated to the SST compared to LMIS (P < 0.05). In relation to this finding, nicotine addiction, as indicated by the categorical outcome of the Fagerström questionnaire, was also stronger in the SST compared to the LMIS (P < 0.01). In a previously published prospective analysis of predictors of quitting in this sample [18], these 3 baseline characteristics appeared not to be predictive of validated abstinence at 12 months follow-up. A bias due to these baseline differences is therefore unlikely.

Table 1 Baseline characteristics of 225 outpatients with COPD, presented as means (SD) or numbers (%)
Full size table

SMOKE study

The SMOKE study compares two smoking cessation interventions in a COPD outpatient setting: the medium-intensity program LMIS and the high-intensity program SST. The SST is a multi-component smoking cessation intervention that consists of group counseling, individual counseling and telephone contacts, supported by the obligatory use of Zyban®, free of charge. The SST provides the possibility to repeat the individual sessions after experiencing a lapse within 3 months. The LMIS is an existing Dutch intervention that is considered as current practice for smoking lung patients in the Netherlands [17]. This intervention consists of individual counseling and telephone contacts. Pharmacological support is recommended during LMIS counseling, but use is voluntary and at the patients’ cost. The SMOKE study [17] showed the SST to be cost-effective compared to the LMIS, expressed as cotinine-validated continuous abstinence rates after 1 year. The number of quitters was 20 in the SST versus 9 in the LMIS, and the associated costs were €3,101 per quitter in the SST and €6,832 per quitter in the LMIS. The SST had dominancy over the LMIS on each outcome parameter in the first 12 months; the SST showed higher effects and lower costs [17].

Economic evaluation

Decision trees were used to outline the cognitive states and the pathways a COPD patient could experience, over the time frame of 12–24 months. They were used to calculate future behavioral change, the associated costs, and subsequently the incremental cost-effectiveness of the SST over the LMIS. Table 2 shows the base case probabilities with the associated 95% confidence intervals (CI). They illustrate the pathways a COPD patient could experience for each arm in the decision tree based on data from the SMOKE study. The primary outcomes are the expected costs of both interventions per quitter.

Table 2 Base case values of the probabilities in the decision tree for the Minimal Intervention Strategy for Lung patients (LMIS) and Smoke Stop Therapy (SST) for the continuous abstinence outcome measure
Full size table

Additionally, probabilities were extracted from the data to determine the distribution in stages-of-change for the smokers at 12 months. Participants who were abstinent at 12 months were all automatically assigned to the ‘action stage’, regardless of the duration of their non-smoking status. For several reasons the ‘maintenance’ and ‘termination’ stages were not distinguished separately. First, the time horizon of the model is limited to 12–24 months. Second, this makes the model more parsimonious and transparent. Third, differentiating the subjects to more than 4 groups would further increase the confidence intervals of the probabilities and this would lower the statistical power with the limited sample size. Participants who reported to be smokers at 12 months filled in a standardized stage-of-change questionnaire [20]. Of the smokers in the LMIS, 30.6% (95% CI: 15.6–45.7) were in the pre contemplation stage, 44.4% (95% CI: 28.2–60.6) were in the contemplation stage and 25% (95% CI: 10.9–39.1) were in the preparation stage of behavioral change. For the SST these probabilities were 27.8% (95% CI: 13.2–42.4), 38.9% (95% CI: 23.0–54.8), and 33.3% (95% CI: 17.9–48.7), respectively.

Probabilities TTM—weighted average

To predict future behavioral change by the stages of change as cognitive parameters, probabilities for the transition from the first 3 ‘pre-action’ stages-of-change to the action stage (in which the actual desired behavior is performed) were collected from literature. The preferred time frame for these probabilities is 12–24 months. A thorough search of the electronic databases indicated that there are no transition probabilities available for smoking COPD patients in this specific time frame. Therefore, a weighted average of multiple transition probabilities reported in literature was used. Included were transition probabilities of smoking cessation interventions, among different populations, interventions and outcome measures. Studies among adolescents were excluded to limit heterogeneity. The formula used for calculating the weighted average with numbers \( x_{1} , \ldots ,x_{n} \) and weights \( g_{1} , \ldots ,g_{n} \) was:

$$ \bar{x} = \frac{{\sum\nolimits_{i = 1}^{n} {g_{i} x_{i} } }}{{\sum\nolimits_{i = 1}^{n} {g_{i} } }} $$

Table 3 shows the characteristics and probabilities of the included studies.

Table 3 Characteristics of included studies for the weighted average of transition probabilities stages of change (Transtheoretical model) for 12–24 months
Full size table

Relapse rate

Delayed negative effects of behavioral interventions should also be taken into account: individuals who relapse into their old (smoking) behavior after they have reached successful behavioral change. An annual relapse rate of 10% (95% CI: 5–17) for the time frame 12–24 months was obtained by Hughes et al. [21]. They conducted a meta-analysis of prospective studies of adult quitters that reported the number of participants abstinent at 1 year follow-up and who remained abstinent at ≥2 years follow-up (prolonged abstinence). In retrospective datasets of non-treatment samples, among those abstinent at 1 year, 2–15% relapsed each year thereafter. The meta-analysis estimated the incidence of relapse to be 10% per year.

Costs

Costs were based on the costs of the SMOKE study for the first 12 months follow-up. They were calculated following a health care perspective, previously reported by Christenhusz et al. [18]. For 12–24 months follow-up, intervention costs were set to 0. Costs regarding exacerbations (€101.25) and hospitalizations (€3,140) were included in the analysis. Because of the different time frames associated with each stage-of-change, we calculated costs per stage-of-change. For example, the Transtheoretical model assumes that a smoker in the ‘preparation’ stage will quit within 1 month. Consequently, this individual will be run through the model as a smoker during 1 month and 11 months as a quitter. Following this procedure, all costs in the cost-effectiveness model were adjusted for the different stages of change the participants were in after 12 months follow-up. Costs and effects were not discounted for time preference.

Figure 1 shows the distribution in smoking status and cognitive states after 12 months of follow-up, the relapse rates for the second year, the weighted averages for prediction of future behavioral change and their associated costs for the Smoke Stop Therapy.

Fig. 1
figure 1

Pathways for the continuous abstinent (CA) and not continuous abstinent arm of the Smoke Stop Therapy (SST) for the time frame 12–24 months, including percentages and costs (€)

Full size image

Sensitivity analyses

All variables were evaluated for uncertainty into the sensitivity analysis. Uncertainty regarding data inputs was quantified by means of a Monte Carlo simulation with 1,000 iterations to explore the variation of the total costs as well as the costs per quitter, and the amount of quitters by varying the cost parameters and probabilities simultaneously over their ranges and the associated 95% confidence intervals. A gamma distribution was assumed for all costs and a logistic normal distribution for all probabilities. Sensitivity analyses were performed using @Risk 5.5 for Excel (Palisade Corporation, 2010; http://www.palisade.com/).

Results

The total costs of an average COPD patient within the SST for the 2nd year (12–24 months follow-up) was €99 compared to €301 for the LMIS. The costs generated by subjects of the SST were considerably lower and the SST had a larger amount of quitters compared to the LMIS. Costs per quitter generated by the subjects for the LMIS were €2,047 and €413 for the SST. The SST had dominancy over the LMIS on each outcome parameter over the first 12 months, and results also show dominancy over 12–24 months.

The weighted averages of the transition probabilities for the three pre-action stages of change to the action stage for 12–24 months were: 0.059 (95% CI: 0.035–0.082) for ‘precontemplation’, 0.085 (95% CI: 0.059–0.111) for ‘contemplation’ and 0.118 (95% CI: 0.087–0.149) for the ‘preparation’ stage. Over the period from baseline to 24 months, 25 patients in the SST quit smoking versus 15 patients in the LMIS, which indicated a slightly lower difference in effect between both interventions compared to the first 12 months. The total costs per quitter, after accounting for a 10% relapse rate, were €3,514 and €8,879, respectively, with a difference of €5,365 in favor of the SST. Analyses for the point prevalence outcome measure showed similar outcomes.

Sensitivity analysis of the decision analytic model

Probabilistic sensitivity analysis was employed to analyse the robustness of the above mentioned findings. The estimates of costs and effects for both the original SMOKE study and this pilot study are represented graphically in Fig. 2.

Fig. 2
figure 2

Monte Carlo simulation results for costs per additional quitter, period 0–12 months (left) and 12–24 months (right). A negative Euro amount and a positive difference in number of quitters favor the SST. Percentages of simulations in each quadrant are given

Full size image

Figure 2 represents the difference in costs associated with the difference in number of quitters. In almost all iterations a higher number of quitters is associated with the SST. In the original SMOKE study [0–12 months; Fig. 2 (left)], the observed costs were, in approximately 58% of the iterations, lower for the SST than for the LMIS [17]. This rate increased to 84.1% of iterations in favor of SST in the data generated for 12–24 months in this pilot study (Fig. 2, right). After simulation, the mean difference in number of quitters at 2 years is 8.95 (95% CI: −0.95–18.84), favoring the SST. The mean difference in total costs between both interventions is €165.21 (95% CI: −450.73–150.15) and the mean difference in costs per quitter is €1,505.57 (95% CI: −3,424.20–74.15), also in favor of the SST. Almost 79% of the iterations are in the south eastern quadrant of the cost-effectiveness plane, which indicates the SST to be dominant over the LMIS for the time frame 12–24 months.

Discussion

Data from the SMOKE study [17] were used to re-analyze a CEA with addition of partial behavioral change estimates based on the stages-of-change algorithm. In the time frame of 12–24 months, the high-intensity smoking cessation intervention for COPD patients (SST) is more effective and less costly in approximately 79% of all simulations compared to 58% of the simulations in the 1st year. Thus, the SST dominates the medium-intensity smoking cessation intervention (LMIS) even more in this further 2nd year of follow-up with inclusion of partial behavioral change.

The present paper illustrates a way to integrate psychological theories into the methodology of health economic evaluations. As the cost of health care rises and consequently CEAs become more important, decision makers have to be optimally informed about the cost-effectiveness of different treatment options [22]. Interventions that aim to accomplish behavioral change can have delayed effects that may influence the cost-effectiveness results [23–25]. This suggests that the commonly applied follow-up period of 12 months may not be sufficient to reflect the true, longer term outcomes. Modeling of partial behavioral change could serve as an alternative way to include future effects in the cost-effectiveness ratio. Smith et al. [26] already reported a way to incorporate future effects by modeling the cognitive ‘pre-action’ stages-of-change. They included partial behavioral change in their CEA of a computer-based smoking cessation intervention in primary care by advancing a smoker’s stage-of-change. However, no transition probabilities or validation of their methods were reported. In the present study, we intended to make the steps that are necessary to model partial behavioral change more transparent. Consequently, this revealed some of the methodological and empirical issues that need to be addressed to further validate this approach.

One of these issues is the predictability of the modeled cognitive parameters. Obviously, one prerequisite a high and empirically supported predictive value of these parameters. Concerning the Transtheoretical model, there is some debate in the literature about the validity of the model. Proponents have argued that application of the model has revolutionized health promotion, but others have suggested that the problems with the model are so serious that it has held back advances in the field of health promotion and, despite its intuitive appeal to many practitioners, it should be discarded [27]. However, critique and debate on the Transtheoretical model is focused mainly on its supposed usefulness for designing stage-based, tailored interventions with superior effectiveness [28–30]. It is the predictive validity of the stages itself that has received strong empirical support; people who are further along the continuum are more likely to change their behavior at a future follow-up point than those who are at an earlier stage [31, 32]. In literature about the model, these stage effects appear to be highly consistent [33]. Nevertheless, some care needs to be taken as our study showed a considerable variability in transition probabilities reported in literature (Table 3).

Considering this, is the cure worse than the disease? Health economic evaluations in general are vulnerable to manipulation due to the use of primary data and the arbitrary definition of outcomes. The definition of meaningful outcome parameters is a precondition for the validity of a study. These endpoints should be clearly relevant in relation to health improvement. Predicting full behavioral change after the intervention period ends, and thus substituting a missing endpoint, may increase uncertainty compared to using an observed outcome parameter like, in this case, smoking cessation. However, uncertainty is pervasive in CEAs [34] and this is generally accepted. Also, developments in health behavior research are promising. More and more evidence is becoming available from theory-based psychological research to determine the uncertainty that comes with predicting full behavioral change using cognitive parameters. This applies to both smoking cessation and other health behaviors. Additionally, the aim was to show the feasibility and challenges of incorporating cognitive intermediate outcomes into CEAs of behavioral interventions. Therefore, no issues regarding discount rates, time dependency or Markov modeling were taken into account, which would probably result in more exact estimates of outcomes and reduce uncertainty.

As partial behavioral changes based on the stage effects of the Transtheoretical model can be incorporated in economic evaluations, this may also be valid for other models of behavioral change [4], such as the Ajzen’s Theory of Planned Behavior [35, 36] for which ample empirical support is available. However, this may require other modeling techniques, like discrete event simulation, as this theory provides a multidimensional continuum, and no discrete Markov states.

In this study, the focus was not on the health effects in the long term, but rather on reducing the risk factor that exacerbates the disease. For decision makers, however, future health benefits and costs are more informative than the costs per quitter following the intervention. The method presented in this article could therefore serve as an extension or antecedent of several predictive models for COPD reported in the literature [37–39], in which disease progression and death are predicted based on, among other factors, smoking status.

Quit rates following smoking cessation interventions have shown to be rather disappointing for the COPD population. These patients tend to have a long smoking history, a long history of failed quitting attempts, and a very strong nicotine addiction [40, 41]. However, transition probabilities for the pre-action to the action stage-of-change (TTM) seem not be very different between populations. Table 3 shows similar probabilities for transitions for COPD patients and the general population. Therefore, applying the method presented in this paper to a CEA of an intervention among the general population will likely show similar effects.

In conclusion, the results indicate that modeling of future behavioral change in a CEA of a behavioral intervention in general may lead to a change in results. As the intervention in the present study was already dominant over the 1st year and merely became more dominant over the 2nd year, the observed change in results would not have led to another decision. In this case, the standard CEA would have been sufficient for decision makers. However, in many cases an ICER may turn out to be less favorable or may approach or even exceed the threshold of willingness-to-pay. Under such conditions, including partial behavioral change in the CEA could have a decisive impact. Furthermore, effectiveness data from existing behavioral interventions that were not assessed with the purpose of conducting a CEA, are often unsuitable for CEAs due to variation in the length of follow-up or due to a lack of adequate behavioral endpoints. Modeling of cognitive parameters of behavioral change may provide a way to overcome such variation between studies, by estimating the required behavioral endpoints for use in CEAs. Ultimately, modeling future behavioral change can have important consequences for public health policy development in general and the adoption of behavioral interventions in particular.