Economic assessment of new healthcare interventions is a necessary requirement for many decision makers today. The use of decision analytic models is now well established and widespread.[1–3] However, the choice of the best modelling practice is of particular concern among modellers. Model structure should be as simple as possible given the decision problem, the course of the disease and the possible clinical pathways.[4,5] Particular attention must be paid to computation transparency and consistency. Models should also serve to synthesize evidence and assumptions in a way that provides decision makers with insight into the implications of the model results.[6]

For instance, Kim and Goldie[7] highlighted that the taxonomy used in the literature to describe model types for the cost effectiveness of vaccination programmes is very inconsistent. The authors suggested classifying models according to three principal distinctions: (i) static versus dynamic; (ii) stochastic versus deterministic; and (iii) aggregate versus individual-based. Further subdivision is possible for more detailed categorization such as discrete versus continuous, linear versus non-linear and open versus closed.[7,8]

In cost-effectiveness evaluations of treatments, it is mainly disease and intervention considerations that have been focused on in the selection and discussion of modelling approaches. Previous reports have principally focused on how to structure the clinical pathways and the outcomes most appropriately in the form of a decision tree, a Markov chain, a discrete-event simulation or a disease transmission dynamic model.[3,8–13] Similarly, handling parameter uncertainty has also received much attention.[14,15]

In contrast, considerations of the target population included in cost-effectiveness models have generally received much less attention from modellers. However, characteristics of the modelled population are of particular importance as they relate directly to the generalizability of the results and the usefulness of those results for decision makers. A target population may range from the general public (e.g. pandemic influenza vaccination) to a highly specific disease group (e.g. patients with late-stage metastatic breast cancer), with or without the presence of externalities. This must influence the way a model addresses a specific decision problem.

The scope of the decision problem and the requirements of the decision maker are important considerations for economic evaluation and should therefore guide model development. Generally speaking, we affirm that any modelling endeavour should incorporate three fundamental considerations in any decision-making question: the target population, the disease and the intervention characteristics (figure 1). Ideally, how to model the target population in order to properly address the decision problem is a concern that should precede the development of a model and is just as important as the issues most often focused on (i.e. the conceptualization of the disease clinical pathway and intervention outcomes).

Fig. 1
figure 1

The three fundamental considerations in developing a model. Discussions principally focus on considerations of disease and intervention characteristics. Modelling the target population should not be omitted and is a concern that precedes other issues in the development of a model intended to aid in the decision-making process.

Full size image

There are essentially two ways of modelling a population over time: a cohort-based approach or a population-based approach. Cohort-based models correspond to closed-type models, in contrast to population-based models that correspond to open-type models, as the taxonomy of Kim and Goldie[7] suggests. A single-cohort approach is generally used in most published healthcare decision models.[16,17] This choice is rarely discussed. Kuntz et al.[17] even suggested that current health economic modelling guidelines have not satisfactorily addressed the definition of cohort models. However, giving a rationale for the selection of the model type is arguably a minimum requirement for good quality modelling.[7,18] In this article, we challenge the assumption that a single-cohort model is generally appropriate for most healthcare decision problems and we propose a set of conceptual definitions.

1. Target Population

A target population is a collection of individuals who are exposed or not exposed to a particular disease and who are targeted by a specific new healthcare intervention for disease treatment or prevention purposes. These individuals may be different in many respects but the one thing they have in common is the fact that they may benefit directly or indirectly from the intervention studied. The intervention can be a drug, a vaccine, a device or any procedure or policy aimed at improving or sustaining the health of the target population in the short or long term. As for any population in general, a target population is likely to be characterized by various types of heterogeneity and a dynamic evolution over time.

1.1 Heterogeneity

The individuals constituting the target population may be heterogeneous in many aspects. Sculpher[19] suggested a list of six possible causes of heterogeneity that may affect the outcomes of an economic assessment. They are summarized in table I. The first four of these may potentially be known at the time of treatment selection as they concern patient baseline, disease or clinical characteristics. This essential feature implies that individuals may respond differently to the intervention or that the costs associated with providing them the intervention may differ — or both.

Table I
figure Tab1

Six potential sources of heterogeneity that may affect the outcomes of an economic assessment (from Sculpher[19])

Full size image

An important difference needs to be made between variability and heterogeneity. Variability relates to differences that occur due to chance (e.g. the degree of treatment response may differ for two very similar patients). Heterogeneity pertains to differences that can be observed and explained, at least in part (e.g. the degree of treatment response may differ according to patient baseline, disease severity or clinical characteristics). As a result of heterogeneity, some stratification might then be needed to better capture the outcomes achieved by a target population. When establishing whether a particular source of heterogeneity should be reflected in a model intended to inform decision making, a key criterion noted by Sculpher[19] is whether the identification of relevant subgroups can be effectively operationalized in clinical practice.

Kuntz and Goldie[20] were two of the first authors to present a systematic examination of the consequences of not taking heterogeneity into account. In their model, patients either had or did not have an unknown risk factor for a lethal disease (i.e. the heterogeneity factor). Individuals with this factor were at a greater risk of disease than those without it. The authors defined a heterogeneity bias as the estimated survival gain, not adjusting for the unknown risk factor, compared with a model that precisely adjusts for this risk factor. They reported that the heterogeneity bias in some scenarios may be greater than 100%.

Building on this model, Zaric[21] went further and took QALYs and costs into account. The author compared two ways of modelling: pooled (combining the risk into a single value equal to the weighted average of high and low risk) versus separate (calculating the results separately for the two groups and finding the overall outcome as the weighted average of the outcomes per group). Four different preventive interventions (diminishing the risk for both or either of the groups, reducing the prevalence of the high risk) and one treatment (reducing the probability of death for those falling sick) were examined. The general conclusion of the analysis was that the two methods only very rarely and under highly particular parameter constellations gave the same results.

1.2 Dynamic Feature

A population has a dynamic feature by nature (i.e. it is formed by the inflow and outflow of individuals as time progresses). At any specific timepoint, the prevalent target population consists of the individuals known to be affected by the disease (in the case of a drug) or targeted by the intervention (in the case of a vaccine or other preventive measures such as screening) and amenable to benefit from the new intervention. At any future point in time, the prevalent patient population will have changed because of new incident patients entering and previously prevalent patients leaving the population (generally because of recovery or death).

This dynamic feature reflects the epidemiology, the natural history and the management of the disease under consideration. For example, occurrence of new cases is determined by the incidence of the disease or by improvements in diagnostic accuracy. Departures depend on the natural history of the disease and on its ongoing clinical management. The patients’ distribution over states of disease severity may also change over time as a result of disease progression or treatment effect. In addition, demographic dynamics such as increasing longevity or changes in the birth rate may affect the evolution of a target population over time.

Another dynamic feature is the diffusion of the intervention that may actually enlarge the target and eligible population. In effect, the launch of a new technology may result in individuals with a particular medical condition who were not previously treated (for various reasons such as lack of effect, too many side effects or complete absence of treatment) receiving treatment. Recent innovative interventions such as atypical anti-psychotics for schizophrenia or new drugs for erectile dysfunction provide precise examples of this phenomenon.[22]

2. Modelling the Target Population

2.1 Cohort-Based Approach

A cohort is a closed group of individuals who have at least one specific characteristic or experience in common over a defined period of time (e.g. the same disease, disease severity or risk exposure). Cohort-based models follow a fixed group of individuals over time, with no further entry as time progresses. Exit from the cohort is conditional on reaching the absorbing state of the model, usually death. In general, the cohort is composed of a hypothetical number of representative or ‘average’ individuals. The cohort is then run through a state transition model and followed over a predefined period of time (i.e. number of cycles) or until the entire cohort has reached the absorbing state.

Based on the fundamental distinction between prevalence and incidence in epidemiology, Hoyle and Anderson[16] argued that economic models should account for the costs and benefits of all individuals in the prevalent cohort and in all future incident cohorts over the life of the new technology. The authors then explored the mathematics of doing so in the calculation of an incremental cost-effectiveness ratio (ICER). They concluded that in many circumstances, both the prevalent cohort and all the future incident cohorts should be modelled, notably for long-term therapies (as in the case of chronic conditions) and when the discount rates for costs and benefits differ.

2.2 Population-Based Approach

We define a population-based approach as a model that intends to evaluate the effect of an intervention over its entire target population. This approach aims at projecting the evolution of the estimated prevalent target population and to account for its distribution over relevant health states and attributes (e.g. age, disease severity, risk exposure). Ideally, a population-based model should reflect all the demographic, epidemiological and clinical characteristics of the prevalent target population relevant for the decision problem.

A population-based model differs principally from a cohort-based model in three ways. First, a population model starts with the estimated size of the prevalent target population and does not stipulate a hypothetical cohort size. Second, a population model accounts for the heterogeneous composition of the target population and does not focus on a representative or average individual. This implies that the population is stratified across health states and attributes (e.g. age, disease severity, risk exposure) identified as being relevant for the decision problem. Third, a population model factors in the dynamic nature of the target population by allowing future incident patients to enter into the model in conjunction with those leaving the model as they reach the absorbing state.[7,23]

An example of a population-based model is the one developed by Mauskopf.[24] The author built what she refers to as a prevalence-based model by expanding a conventional cohort Markov model to allow for new incident cohorts to join the modelled population. The model defined five health states determined by CD4 T-cell count for the analysis of a new treatment for persons living with HIV and AIDS (PLWHA). Effective treatment moved the PLWHA into a state with a higher CD4 count. The cohort model started with 1000 PLWHA that began with the observed distribution of CD4 count health states. The PLWHA were followed over their remaining lifetime. The prevalence-based model started with the estimated prevalent number of PLWHA in a given year (10 680 patients in the UK in 1994), distributed across the CD4 count states in the proportions expected from disease natural history data. The PLWHA only left the diagnosed HIV population when they died. Each year a cohort of newly diagnosed PLWHA (n = 1258) was added with a CD4 count distribution typical for newly diagnosed PLWHA. This prevalence-based Markov model was used to track the population each year over 20 years with and without the new drug. In this model, the annual impact on healthcare costs, opportunistic infections and hospital days of the new drug in the treated population of PLWHA was determined for each year as well as the cumulative impact over the entire period.

2.3 Multi-Cohort Approach

A hybrid approach, combining features from both cohort and population modelling, is possible. Careful examination of the target population distribution over different attributes may actually reveal clear heterogeneity. Modelling the target population as a homogeneous cohort may then become noticeably inappropriate for the decision problem. One option is to subdivide the cohort into two or more sub-cohorts, according to different baseline health states, age groups, disease severity, risk factors for progression or any other characteristics relevant for the decision problem. Alternatively, it might not be necessary to account for the dynamic inflow of the population in order to address the decision problem. In such an approach, each of the sub-cohorts is fixed at the outset of the simulation and no new cohorts of patients are entered as time progresses. Ideally, in a multi-cohort design, each sub-cohort size should be as representative as possible of the target population dispersion across the relevant attributes for the decision problem.

This multi-cohort approach is surely best exemplified by a multi-birth cohort model. In this type of model, age-dependent distribution and heterogeneity of a population can be readily taken into consideration but not the future births that will fuel the target population as time progresses. In this case, a population model can be viewed as an extension of a multi-birth cohort model, in which all future incident birth-cohorts enter the model for a given number of years as time progresses in addition to the prevalent multi-birth cohorts.[17] Figure 2 summarizes the salient features of the two modelling approaches and the resulting three types of models: cohort, population and multi-cohort.

Fig. 2
figure 2

Stylized difference between a cohort model, a multi-cohort model and a population model. (a) A single-cohort model is necessarily a closed group of patients. One prevalent or incident cohort enters at t0 and at a given starting age. Modelling a target population as a single cohort implies the population is considered to be homogeneous. (b) A multi-cohort model can account for potential heterogeneity. Different prevalent or incident cohorts enter at t0 but at different starting ages (e.g. a multi-birth cohort model to capture the effects of a screening programme, which applies to more than one age group). (c) A multi-cohort model can also consider the target population as a homogeneous population but be opened to future incident patients. A multi-cohort model can be formed by different cohorts of individuals entering at different times, but with the same starting age (e.g. a multi-cohort model to capture the effects of an intervention that becomes less costly, more effective and more widely used as time goes by). (d) A population-based model considers the entire prevalent target population, accounts for future incident individuals and takes potential heterogeneity into account by distributing the target population according to particular characteristics considered most relevant for the decision problem. In a way, a population-based model combines the features of the two multi-cohort approaches (b) and (c). New individuals can be newborn or new incident patients. The number of incident individuals can be fixed or variable every year (e.g. steady, increasing or decreasing trends depending on demography, epidemiology, disease management and market data). Single-cohort models () and multi-cohort models (b) are evaluated accumulatively over cohort(s) lifetime(s) or any definite time horizon. Multi-cohort models (c) and population models (d) can be evaluated cross-sectionally or accumulatively. Cross-sectional evaluation can be at population steady state or at any definite timepoint (e.g. 5 years after the new technology launch). Accumulated evaluation is necessarily over a definite time horizon.

Full size image

A good example of handling known heterogeneity with a multi-cohort approach was provided by Dewilde and Anderson.[25] The authors compared a single versus a multi-birth cohort model to examine a screening intervention. The outcomes of screening programmes have typically been simulated using single-cohort models. However, such models only consider the impact on individuals in the target population who are about to enter the screening age range, ignoring the impact on the rest of the eligible population. First, a single-cohort simulation was run for the youngest age group of interest to produce a Markov trace of all the possible states. Five-year intervals were selected for the multi-birth cohort model, using 11 cohorts between 15 and 65 years of age. All cohorts ended at age 85 years. Age-specific prevalence rates and transition probabilities were produced for the various states.

The final result of this multi-birth cohort model was determined as the weighted average of the outcome for each birth cohort, the weights being established as the proportion of each age cohort within the target population. Compared with the single-cohort model, the multi-cohort simulation gave different (lower) estimates of both costs and health benefits and a less favourable ICER. The authors concluded that to achieve more precise estimates of costs and outcomes, it is not sufficient to have a valid model with structures, transition probabilities and Markov states accurately reflecting broadly homogeneous prognostic and risk categories. It is also necessary to ensure that the age distribution of the simulated population is similar to the target population of the intervention.

2.4 Model Evaluation

In a single-cohort-based model, the evaluation is done by accumulating the outcomes of interest over a finite time horizon or over the cohort lifetime. The same applies to a multi-cohort model. An overall assessment is carried out by running the model for each sub-cohort separately, with divergent characteristics and state transition probabilities. The overall result is then obtained by the weighted average of the results for each sub-cohort. Weights are determined by the proportion of the target population belonging to each sub-cohort.

A population-based model can be evaluated cross-sectionally or accumulatively. A cross-sectional evaluation can be performed when it is reasonable to assume that the target population has reached a steady state for the decision problem (i.e. if the target population and its distribution across the relevant attributes are not expected to change in the future). A cross-sectional evaluation can also be performed at any definite timepoint (e.g. 5 years after the new technology launch). An accumulative evaluation is necessarily performed over a definite time horizon. The evaluation period can be determined following various criteria such as the expected dynamics of the target population or the expected diffusion pace of the new intervention within the target population.

In both multi-cohort and population models, the difficulty lies in knowing which factors to use to stratify the population. For instance, should the model be stratified by age, risk group, disease severity, gender, socioeconomic status, previous exposure to a disease or to a treatment, etc.? Admittedly, this stratification is to be guided by the scope of the decision problem and the decision-maker requirements to address it.

Standaert et al.[26] recently compared a cohort- and a population-based approach to examine the effect of a vaccine against pneumococcal diseases. Both models were applied to two target populations with widely divergent age structures in two countries (the UK and Mexico). The cohort model followed a single birth cohort over a lifetime of 100 years. The population model subdivided the population into 123 age groups (monthly from birth to 23 months, yearly thereafter), with the number of individuals in each group determined by the current age structure of the corresponding country.

This difference in age structure manifested itself clearly when comparing the cumulated number of person-years at risk between the two models. For instance, the difference between the two models in terms of person-years at risk was marginal (2%) for the UK but significant (42%) for Mexico. The difference between the two models is indeed relatively small for the UK, which may give a false sense of equivalence between the models. However, the divergence found for Mexico plainly exemplifies the fact that cohort and population models address the potential impact of an intervention differently. A cohort model focuses on the accumulated outcomes over time among individuals of the cohort. Conversely, a population model focuses on the short-term outcomes achieved across the different sub-cohorts making up the population at the time of the cross-sectional assessment. In this particular example, the results would only be equivalent if the age structure and the disease transmission dynamics were the same in both models and remained stable over the long term.

3. Discussion

Guidance drawn from a decision analytic model can only be as good as the model’s quality. Good quality modelling goes beyond model structure, data sources or the handling of uncertainty. Quality modelling should also embrace the way the target population is modelled. Suitable and reliable estimates of benefits and costs of a new intervention can only be obtained if the target population characteristics are adequately reflected at the model outset. Health economists constantly claim that modelling is a necessary step to palliate the gap of information between clinical trials and decision-maker inquiries. Omission of important target population characteristics might well jeopardize the intended modelling object (i.e. helping the decision-making process).

Many models depart from single cohorts, which are generally fixed, hypothetical and composed of representative or ‘average’ individuals. Considering a target population as a homogeneous group is a strong assumption. It is rarely discussed by modellers, while it may have worrisome effects on the implications of the results. For example, a state transition model implies that the same state transition probabilities apply to all individuals sojourning in a state. Therefore, the individuals are assumed to be identical in all those respects thought to influence the transition probabilities. Sparse but convincing examples have been published, and all have shown non-negligible effects on intervention cost effectiveness.

Individual-level simulation is increasingly used in cohort-based modelling to factor in individuals’ variability and to reflect the potential dependence of transition probabilities on individuals’ attributes and histories.[27–29] This approach results in a distribution of outcomes rather than in a single average, so it provides an indication of the likelihood of specific or extreme outcomes, depending on individual patient characteristics. Yet, this approach might not produce target population-level estimates of outcomes the same as multi-cohort or population-based models. Individual variability (e.g. different response between two female patients) and target population heterogeneity (e.g. 75% female vs 25% male) are different matters. The former relates to the individual-level outcomes, while the latter relates to the population-level outcomes yielded by the intervention.

A single-cohort approach may suffice as long as relative homogeneity of the target population is guaranteed. As soon as heterogeneity is recognized, a multi-cohort approach should also be explored to account for the target population dispersion over the heterogeneous characteristics.[30] Depending on the decision problem, one may perform a cost-effectiveness appraisal for each sub-cohort separately and make decisions per subgroup. Alternatively, one may choose to provide an overall appraisal as a weighted average of subgroup results.

Eligibility criteria for a new intervention and potential sources of heterogeneity for its evaluation are mainly health policy concerns. They principally occur when decision makers intend to circumscribe the target population and to define the reimbursement condition of an intervention. Modellers should thus make every endeavour to embrace health policy objectives. Decision makers are not necessarily interested in overall or average estimates. In contrast, they increasingly value targeted allocation of treatment and prevention strategies, depending on the likelihood of response, capacity to benefit or risk exposure. For instance, a multi-cohort approach is probably more akin to the emerging concept of personalized or stratified medicine.[31]

Cost-effectiveness appraisal necessitates the accrual of both costs and benefits over time for each alternative to inform decision making. However, cost-effectiveness models do not generally account for possible future changes in technology, prices and information, which are by essence uncertain.[32] A decision maker has generally some discretion on the timing of the decision (i.e. the decision maker can wait until further information is available). In addition, a decision entails a certain degree of irreversibility (i.e. a decision can irreversibly exhaust an amount of resources that will not be further available for other purposes).

Several approaches have thus been proposed to address these issues using cohort models. For instance, Palmer and Smith[33] used an option-pricing technique to value the flexibility in the timing of an investment decision. Philips et al.[34] drew on a value-of-information analysis to assess the value of acquiring additional information to inform a decision problem.

As recently demonstrated by Hoyle and Anderson,[16] it is also possible to incorporate into the ICER computation the costs and benefits for both prevalent and all future incident patients over a new technology life-cycle, thereby using an open multi-cohort approach. Uncertainty about future costs or the number of eligible patients can then be incorporated into sensitivity analysis.[35]

A particular situation is the presence of interactions between individuals, such as in the case of vaccination against infectious diseases. In this case, cohort-based models are potentially inappropriate. Population-based models that reflect population dynamics and that can fully encompass the disease transmission dynamic across multiple population segments are better suited.[36,37]

To complement cost-effectiveness appraisals, budget impact analyses (BIA) are often performed to provide the decision makers with the expected impact of introducing the new intervention on their budget.[22] A decisive factor for a BIA is the anticipated diffusion of the new intervention within the target population. In this circumstance, the target population dynamic must be coupled with the market dynamic (change in the mix of interventions). It is then advisable to use a population-based model.

More generally, cohort models are to be used in cost-effectiveness analyses in which both costs and effects must be accrued over time. Nevertheless, an open population approach is also possible over a new technology life-cycle to account for all future incident cohorts of patients who will receive the technology.[16]

Population-based models are better suited in the presence of interactions between individuals and for any type of impact analyses. This could include, for example, the estimated impact on the health of the target population and on the target disease burden.[38,39] For instance, changes in public health indicators such as the disease prevalence and incidence or the number of bed-days will be more appropriately simulated through a population-based model rather than a cohort-based model.

An interesting option may be to combine the various modelling approaches in the spirit of Mauskopf,[24,39] who combined the incidence- and prevalence-based modelling for the economic assessment of new HIV treatments. Cohort or multi-cohort models are to be used to assess intervention efficiency. Population-based models are to be used to assess affordability and population-level outcomes. Of interest, such an approach has also been suggested by some epidemiologists[40] and operational researchers.[38,41]

4. Conclusion

No single type of model can provide adequate answers under all circumstances. However, a basic requirement of any modelling project must be the consideration of the target population dispersion across the set of characteristics potentially pertinent for the decision problem. Modellers and decision makers should be aware of both cohort- and population-based modelling approaches and accordingly be able to assess the implications of a model’s results.