FormalPara Key Points for Decision Makers

Agent-based modelling (ABM) is more flexible than other modelling methods commonly used for economic evaluations, and can provide insights into dynamic interactions among real-world phenomena.

ABMs can be more challenging to construct than other modelling methods.

The need for ABMs is expected to increase as decision-making becomes more complex and as data availability increases.

1 Introduction

Computational or mathematical models in healthcare can evaluate interventions that cannot be tested in the real world, either because of ethical reasons or because of the need for long follow-up time or substantial monetary investment. Such models can be utilized to inform a funding decision or the reimbursement rate of a new technology, to optimize the use of scarce resources, or to guide public health practice [1].

Commonly used modelling approaches include state-transition models (STMs), discrete-event simulation (DES) and differential equation-based compartmental models [2]. These methodologies have made several contributions to the field of medical decision-making and cost-effectiveness analysis; however, these approaches may impose restrictive assumptions that limit their use in many healthcare problems. These assumptions include linearity, homogeneity and stationarity [3].

For example, STMs and DES typically do not capture interaction among individuals, such as transmission of an infectious disease. As a result, these models can sometimes underestimate the benefits of interventions by ignoring the indirect benefits [4]. For example, vaccination against human papillomavirus can reduce transmission of infections—a benefit that needs to be captured when estimating the value of a vaccination program. Similarly, human immunodeficiency virus (HIV) treatment can also reduce transmission of the virus to uninfected people (i.e. treatment as prevention). STMs and DES, by design, cannot capture transmission of disease (or interactions among individuals), but these effects can be simulated indirectly. Differential equation models, on the other hand, can capture interactions among individuals but have limited scope in terms of modelling heterogeneity among individuals [5]. Therefore, traditional modelling techniques may not be applicable in many complex situations or may provide a limited scope for studying real-world problems.

Agent-based modelling (ABM) can overcome some of these limitations. ABMs consist of agents that interact with each other and their environment. In disease modelling, agents can be people who have certain heterogeneous features, such as age, sex, risk behaviour or disease status, which can change with time. Examples of environments can be specific geographic locations, such as hospitals, schools, or neighbourhoods. Figure 1 shows a schematic representation of agents transmitting disease to other agents through interactions in their network (shown by connected lines). Infected agents (shown in red) can progress to more advanced stages of disease without interventions.

Fig. 1
figure 1

A network of people (agents) with specific characteristics. Infected agents are shown in red and uninfected agents are shown in black. The lines show connections among agents with possible transmission of disease. Each infected agent can progress to advanced stages of disease, shown by a Markov chain in the top right corner

Full size image

ABM is a relatively new modelling technique, both in the modelling field and in health technology assessment. Though the practical ideas of ABM originated in the 1970s [6, 7], its applications started appearing in the 1990s, with the availability of simulation packages and software [8]. ABMs have been applied successfully to a wide range of topics, such as traffic management, customer flow management, prediction of stock markets, analysis of operational risk, the military, biological sciences and infectious disease control [8, 9]. The need for ABM is expected to increase as decision-making becomes more complex and as data availability increases.

In this tutorial, we discuss the basic concepts and important features of ABM. We present a case study of an application of a simple ABM to evaluate the cost effectiveness of a screening program. We also discuss software, resources, challenges and future research opportunities for ABM.

2 Properties of Agent-Based Models

2.1 Agents

Agents are autonomous entities, i.e. their behaviour is not determined by a global controller; instead, they do what they are programmed to do in their current situation. All agents have certain predefined characteristics or attributes. The agents interact with each other and with the environment. For example, patients with specific disease characteristics can infect other agents in their contact. Agents can die or give birth to new agents. Agents also possess memory and have the ability to learn new behaviours from other agents or from the environment. Agents follow predefined rules and heuristics that determine their behaviour in a given situation on the basis of their characteristics and history. Interactions between agents can produce network effects. Therefore, agent-based methodology can explicitly capture social networks, which sometimes are critical for simulating the spread of an infectious disease.

2.2 Environment

An environment provides a channel for interaction between agents. All agents act within their environment and can also interact with the environment. Examples of environments include hospitals, schools and cities. The environment may also include passive objects, such as roads or air travel for agents to travel within their environment or to provide agents with resources, such as access to healthcare.

2.3 Emergence

One of the most important and unique characteristics of ABM is that it captures emergent phenomena that result from the interactions of individual entities. By definition, an emergent phenomenon evolves in an unpredictable way as the process evolves. The key question about emergence is what dynamics of the system or what model outcomes emerge from what behaviours of the agents and what characteristics of their environment [10]—for example, how an individual-level decision to vaccinate or to stay at home when sick during the influenza season affects the peak of an influenza epidemic [11]. Simulating individual-level behaviours and their interactions and observing the systems-level outcomes can capture such ‘bottom-up’ phenomena. Other examples of emergent effects are how sexual behaviours, such as tendency to use condoms or norms about indulgence in polygamous behaviours, impact the spread of an HIV epidemic in a particular society; and how an individual’s adherence to volunteer quarantine could affect an epidemic of Ebola virus disease in a community [12]. An emergence phenomenon therefore can be defined using the following characteristics: it is not simply the sum of the properties of the model’s individuals; it is a different type of result from individual properties, and it cannot be predicted from the properties of individuals [10].

2.4 Network Structure

One of the main advantages of ABMs is their ability to capture relationship networks among individuals [13]. The network structure determines which individuals can possibly interact with which other individuals and potentially transmit disease [14]. The topology of the network of relationships defines such interactions. Several standard network structures can be used to define interactions among agents, such as fully connected networks, random networks, small-world networks and scale-free networks [14]. These network structures have different properties, such as the speed of the spread of disease and connectivity between agents. For example, an individual with a bigger network of relationships plays a crucial role in the spread of disease, or an infectious person in a highly clustered network is highly likely to infect others in the network. Figure 2 provides two examples of infectious disease networks. Figure 2a demonstrates the spread of a virus through a network where each node may be in one of three states: susceptible, infected or resistant [15, 16]. Figure 2b presents a network with the small-world phenomenon, i.e. a person is only a few connections away from any other person in the world [16, 17].

Fig. 2
figure 2

a Infectious disease network showing the spread of a virus through nodes, which represent susceptible people (in green) and infected people (in white). The arcs represent a network through which one agent could probabilistically transmit a virus (or disease) to another agent. b Small-world network showing that a person is only a few connections away from any other person in the world. Each node represents an agent and each arc represent a social network (or contact) between two agents. An agent can transmit information to another agent only within its network

Full size image

2.5 Stochasticity

ABMs typically are stochastic in nature. Stochasticity refers to the fact that individuals facing the same probabilities and outcomes will experience the effects of a disease or intervention differently [18, 19]. For example, consider that 10 % of patients in a clinical trial die within 1 year. If this event is modelled in a group of 100 cohorts, approximately ten people will die and the others will survive. The stochastic nature of ABMs provides the advantage of capturing randomness in real-life scenarios. This randomness is also known as first-order uncertainty and is captured by the use of pseudo-random numbers. In contrast, deterministic models always produce the same results each time they are run. While first-order uncertainty relates to randomness in outcomes, second-order uncertainty relates to parameter uncertainty, which arises from the fact that the true value of a parameter cannot be known from a clinical study and only an estimation can be made [18, 19]. Second-order uncertainty is typically evaluated under sensitivity analysis.

The level of first-order uncertainty can be higher in ABMs than in STMs or DES. This is because ABMs capture interactions among agents, as well as progression of disease (or other events) within each agent. First-order uncertainty can be reduced by running the model several times, using different random numbers [2]. This replication of runs is also commonly known as first-order Monte Carlo simulations or number of trials. The sufficient number of the required Monte Carlo runs depends on the structure, the number of parameters and the time horizon of the model, and it can be assessed either qualitatively or by evaluation of the variance in model outcomes [20, 21].

2.6 Time Increments

Time in ABMs can advance either in discrete time steps, knows as cycles (as in STMs), or in a continuous manner in which a unique event determines the next increment (as in DES). Within each discrete time step, specific events can occur (e.g. transmission of disease, progression of disease or movement of agents); therefore, the sequence of events within each cycle is important. Within each cycle, all agents are asked to update their characteristics in a certain order. As in STMs, discretization of time can introduce error into model results. For example, counting membership at the beginning of the cycle in STM results in overestimation of the life expectancy, and counting membership at the end of the cycle in STMs results in underestimation of the life expectancy. Therefore, half-cycle correction may also be needed in ABMs.

The second option for time increments in ABMs is to use scheduled events instead of fixed intervals. In this approach, only agents who need to take action are invoked and time advances to the next event. This approach is similar to time increments in DES. Instead of advancing the time by fixed cycles, the model advances to the next event occurring at anytime in the future (i.e. continuous time). This requires scheduling of all future events. However, scheduling of events generally becomes more complicated with the discrete-event approach in comparison with the fixed-cycle approach in ABM. In addition, use of events to advance time could make implementation of interactions more complicated than in the fixed-cycle approach. Therefore, fixed discrete time steps could be a more convenient approach than discrete-event time increments in many cases [10].

2.7 Open Versus Closed Populations

Open population refers to the fact that new people are added into the model as births, immigrations, incidence, etc. Like DES and compartmental models, ABMs allow the flexibility of modelling both open and closed populations. In contrast, Markov models generally start with a closed cohort and do not allow births in the model. An advantage of open populations is that they can effectively capture real-world phenomena, such as vertical transmission of disease (or other characteristics) from mother to child, or immigration to a country. In addition, open-cohort models more closely represent the real world than closed-cohort models by capturing population dynamics such as increasing longevity over time. Open models can simulate multiple-birth cohorts over time [22, 23]. In contrast to closed-cohort models, open-cohort models can provide benefits dependent on the timing (e.g. month, year) of implementation of the intervention [23].

3 Modelling Features

In this section, we describe several features of ABMs that may be relevant to other individual-level simulation models as well. However, these features may require special attention in the context of ABMs.

3.1 Variance Reduction Techniques

ABMs, like other individual-level models, require multiple runs (Monte Carlo runs) to reduce first-order variability (i.e. simulation noise) in outcomes. This variability can be reduced by variance reduction techniques [24–26]. Several variance reduction techniques have been used in the field of operations research [27]. These include the use of common random numbers, antithetic variates, control variates, importance sampling and stratified sampling. Because most decision-analytic models for economic evaluations are designed to compare two or more strategies, use of common random numbers between the strategies has been widely used in health economic modelling [24]. However, implementation of common random numbers in ABM can be challenging. In STM, use of common random numbers forces an individual in the model to follow an identical simulation path in multiple interventions. However, in ABM, the interaction among agents can alter the simulated path and thus change the sequence of events across interventions. One approach to implementation of common random numbers in ABMs is to assign a specific sequence of random numbers to each agent, i.e. to assign a unique random seed to each agent. This will ensure that the natural history of a disease in each agent or their behaviour does not vary across intervention strategies unless the specific agent accepts/rejects the intervention of interest. Alternatively, the use of antithetic variates or control variates could help reduce the simulation noise in ABMs.

3.2 Probabilistic Sensitivity Analysis

Performing probabilistic sensitivity analysis (PSA) in ABMs is computationally challenging and can sometimes be impractical [28, 29] because of two levels of uncertainty: uncertainty due to the stochastic nature of such models (first-order uncertainty); and uncertainty due to lack of full knowledge of the model inputs (second-order uncertainty). The use of stratified sampling methods, such as Latin hypercube sampling, can reduce the computational burden of PSA [30, 31]. These stratified samples can be generated either by use of a statistical package or with off-the-shelf tools such as SaSAT [32]. In addition, statistical methods such as Gaussian process emulation can be used to conduct PSA efficiently by formulating a statistical approximation function between the inputs and outcomes of interest [33, 34]. In ABMs, many transmission- or network-related parameters are correlated and such correlations need to be preserved in performing PSA. However, such correlations may not be known. Therefore, performing PSA in complex ABMs may not be always feasible.

3.3 Validation

Trust and confidence in a model’s results is critical for decision makers and can be built by validation of the model [35]. As in any simulation model, validation is an important step in ABM. The International Society For Pharmacoeconomics and Outcomes Research–Society for Medical Decision Making (ISPOR-SMDM) Modeling Good Research Practices Task Force recommends evaluation of models with five types of validations: face validation, verification (or internal validation), cross validation, external validation and predictive validation [35]. Validation implies determination of whether the model is consistent with a real-world system. For example, external validation may investigate whether the predicted 5-year survival from the model matches the available clinical evidence.

Validation plays an especially important role in ABMs because of the complex non-linear interactions within ABMs. Sometimes the only way to gain trust in a complex model is to compare the outcomes with known data from other sources. For example, in infectious disease modelling, transmission of disease parameters may have high variability, whereas the incidence of disease may be well known. Therefore, a comparison of model’s projected incidence with the known incidence could be an important validation step.

3.4 Calibration

Calibration is defined as an iterative process of adjusting the model inputs so that the outcomes match the observed or real system. Calibration of parameters may be needed to improve the external validity of the model, to include unobservable parameters in the model or to adapt the model to a different setting (e.g. another country). The calibration process generally includes definition of a target, a goodness-of-fit metric, a search algorithm, an acceptance criterion and a stopping rule [36, 37].

Because ABMs are more complex and can include several heterogeneous parameters, calibration is needed more frequently than with other modelling approaches. For example, the transmission rate among a subgroup of a population may not be known, but the incidence of disease in that subgroup is available. In this case, one would need to calibrate the transmission-related parameters of that subgroup such that the predicted incidence of disease matches the available data. Calibration can sometimes be challenging in complex ABMs because of the huge computational burden. Therefore, the choice of calibration method can play an important role in improving the efficiency of the calibration step [37].

4 Comparisons with Other Modelling Methods

We compare ABM with STMs, compartmental models and DES models. ABMs share some common features with these methods and also possess some unique features not found in them.

4.1 Agent-Based Models Versus State-Transition Models

STMs can be cohort- or individual-level models, whereas ABMs are, by design, individual-level only. Both STMs and ABMs can model disease progression as Markov chains. As individual-level STMs, ABMs are also stochastic in nature. The primary difference between ABMs and STMs is that ABMs are dynamic, whereas STMs are static, which implies that STMs do not capture indirect effects as commonly observed in infectious disease modelling. Therefore, STMs can underestimate the benefits (e.g. disease reduction and cost effectiveness) of interventions by ignoring these indirect benefits [4]. ABMs can model interactions of individuals using different network structures, thus they can more effectively capture real-world disease transmission effects than STMs. ABMs generally have greater first-order variance than their corresponding STMs, therefore they generally require more Monte Carlo runs.

4.2 Agent-Based Models Versus Compartmental Models

Compartmental models are another type of dynamic transmission models, which are frequently used in infectious disease modelling [38]. The compartments represent population pools, and transition between compartments is represented by differential equations [39]. Compartmental models generally require simpler assumptions and typically assume homogeneity of individuals within each compartment (e.g. the SIR model). In contrast, agents in ABMs are heterogeneous by definition. Although some compartmental models can capture heterogeneity, it comes at the expense of more compartments, which can substantially increase the size of the model. While ABMs can capture different network structures, compartmental models assume homogeneous mixing of individuals. Compartmental models typically do not take into account spatial components, whereas agents in ABMs can have a well-defined geographic location, and interactions among agents can have specific movement-related properties. Time is represented in a continuous manner in compartmental models. The main advantage of compartmental models is that several real-world insights can be obtained from analytical (i.e. mathematical) analysis of these models, which is not possible with ABMs. Compartmental models provide a deductive or ‘top-down’ approach, i.e. system-level properties define the average behaviour of individuals, whereas ABMs provide an inductive or ‘bottom-up’ approach, i.e. individual-level behaviours define system-level components.

4.3 Agent-Based Models Versus Discrete-Event Simulation Models

DES was developed in the 1960s, but its application to economic evaluations started to emerge only in the last decade [40–42]. DES is based on the concepts of entities (analogous to agents in ABMs), attributes, queues and resources. In DES, entities are generally passive and do not adapt their behaviour to, or learn from, the environment. DES provides a centralized approach, whereas ABM describes the process from an entity’s viewpoint. Therefore, DES, like compartmental models, provides a ‘top-down’ modelling approach. Entities in DES typically compete for limited resources (for example, patients waiting in an emergency department). Entities follow some logic of entering or exiting queues (e.g. first-in, first-out or triaging on the basis of entities’ attributes). This logic of queues can also be implemented in ABMs whenever there are limited resources. Considering these factors, ABM can be considered as an extension of DES. More details about comparison of DES and ABMs can be found elsewhere [43, 44].

5 Case Study: Cost Effectiveness of a Screening Program

This case study evaluates the impact of screening for a hypothetical infectious disease, using ABM. We modelled the transmission of an infectious chronic disease (e.g. hepatitis or HIV) [Fig. 3], which progressed among an isolated population. We assumed that untreated infection leads to a progressive disease that follows a Markov chain. Screening followed by treatment could cure patients. Our objective was to evaluate the cost effectiveness of a screening program in a closed population of infected and uninfected people (who are susceptible to the infection).

Fig. 3
figure 3

Transmission and progression of disease in the agent-based model. Each infected agent can transmit the infection to other susceptible people, with some probability per contact. The infected agent, if screened and treated, can be cured and move to a susceptible state with a possibility of reinfection. Untreated infected agents can progress to advanced stages of disease, shown by a Markov chain represented by the following states: state A, state B, state C, and disease-related death (DRD). From each state, the person can die from age- and sex-specific background mortality (not shown, for clarity)

Full size image

5.1 Population Description

Our baseline population consisted of 5,000 people, who were described using the following characteristics: age, sex, infection status and health stage. The initial prevalence of the disease was assumed to be 1 % (i.e. 50 people had the infection at the beginning of the model). We assumed that no person had been screened at the beginning of the model. The mean age of the population was assumed to be 45 years, and the male-to-female ratio was 50:50. Time advanced in the model on a yearly basis (i.e. we used an annual cycle length). People aged with the advancement of time. A person could die from background mortality, which depended on age and sex, as determined by US life tables [45]. An infected person entered state A and could advance to more severe stages of the disease, represented by state B and state C. People in state B or state C had higher mortality rates than other people, which were shown by transitions to disease-related death from state B and state C. The model parameters are provided in Table 1.

Table 1 Parameters of the case study comparing the cost effectiveness of the screening versus no-screening scenarios, using state-transition and agent-based models
Full size table

5.2 Interventions and Outcomes

We compared two scenarios: screening and no screening. The screening scenario assumed that 1 % of the total population was screened in every cycle. The infected people who were screened were eligible for a treatment that could cure the infection with 60 % effectiveness. The no-screening scenario assumed that infected people continued to progress to advanced stages of the disease. For each scenario, the model estimated the total cost and quality-adjusted life-years (QALYs). The total cost was a sum of the health maintenance costs of advanced stages, treatment costs and screening costs. We estimated the incremental cost-effectiveness ratio (ICER) of screening in comparison with no screening. We also estimated the reductions in the number of deaths and new infections due to screening.

5.3 Implementation

We implemented ABM in an open-source software program, NetLogo version 5.0.5 [46]. The model can be downloaded from the Supplementary Material of this article (available at doi:10.1007/s40273-015-0254-2).

5.4 Agent-Based Approach

The agents in our model were defined as people who are infected, susceptible or at the chronic stage. We assumed that disease could be transmitted from infected to susceptible people during a contact. Examples of contact include sexual contact, contact with contaminated needles and physical proximity. In each cycle, an infected person could come into contact with several susceptible persons in his/her contact radius and could probabilistically transmit disease to a them (Table 1). The contact radius in our problem’s context was defined as being in one of three categories: small, medium and large. Infected individuals, if not screened or if unsuccessfully treated, progressed to chronic stages, where they accrued annual costs. These individuals also had lower health-state utilities than individuals in infected states or cured states (Table 1).

In each cycle, the following events could happen: transmission of disease, progression of disease as defined by a Markov chain, screening of agents and treatment of infected agents (Fig. 3). In contrast to an STM, agents in our ABM interacted with other agents and changed their attributes on the basis of their interactions. The ABM explicitly captured new infections due to interactions among agents. By projecting the number of new infections under the screening and no-screening scenarios, the model predicted the number of infections that could be avoided by implementation of a screening program.

We ran the model with the no-screening and screening scenarios 5,000 times to capture first-order uncertainty. Table 2 shows the outcomes of the ABM. The ICERs of screening were between $23,091 and $60,352 per additional QALY, depending on the infection radius. Using a willingness-to-pay threshold of $50,000 per additional QALY, the model predicted that screening is cost effective for medium and large infection radii. This can be interpreted in two ways: (1) screening is cost effective in a highly connected population; or (2) screening is cost effective when use of quarantine measures is very low among infected individuals. The model also predicted that 1–73 new infections could be avoided by screening. These insights could not be predicted from an STM. In addition, screening could avoid 4–47 disease-related deaths, in comparison with no screening. We also observed that the number of averted infections and deaths increased nonlinearly with the increase in the contact radius. These results that emerged from the simulation could not be predicted either in advance or using an STM.

Table 2 Results comparing the cost effectiveness of the screening versus no-screening scenarios, using an agent-based model
Full size table

We have presented a simple, hypothetical example to highlight some of the advantages of ABM over other modelling techniques. In addition, there are several more advantages of ABM, including analysis of outcomes by patient subgroups and analyses of the spread of disease in different networks.

6 Resources

We provide resources for the implementation of ABM and educational material that could help interested modellers. Because the field is advancing rapidly, new resources are expected to become available in the near future.

6.1 Software and Tools

Several off-the-shelf software packages are available for ABM. Commonly used packages include Repast, NetLogo, Swarm, MASON and Any Logic. In addition, ABMs can be built using general programming languages, such as C++, Python or Java. Off-the-shelf software programs provide an ease of building models and can be efficient for beginners. Another advantage of such software packages is their capability to display simulations and graphs dynamically, which can be very helpful to end users such as policymakers. However, these software packages may pose challenges for scaling up the number of agents or may limit the flexibility of the analysis. On the other hand, general programming languages require a substantial upfront effort in building the models but can provide much more flexibility. Modelling tool selection should depend on the complexity of the problem, the quantitative training of the modeller, and the time available to complete the project. More details on ABM software and toolkits can be found elsewhere [9, 47].

Others have used a hybrid approach by developing a generic ABM framework, using general programming language that can be easily adapted to different infectious diseases and countries by end users [11]. One such example is the Framework for Reconstructing Epidemic Dynamics (FRED), which is a freely available open-source ABM system [11]. FRED uses open-access census-based synthetic populations, which capture the demographic and geographic heterogeneities of the population, including realistic household, school and workplace social networks. FRED epidemic models are currently available for every state and county in the USA and for selected international locations. Simulations of an influenza spread, such as the H1N1 pandemic, in a population of 1 million people takes less than 2 min on a typical dual-core laptop computer.

6.2 Books and Tutorials

In 2001, the National Academies of Science and Engineering organized a colloquium on social science modelling and simulation, which served as a platform for presentation of numerous papers across a variety of ABM applications and methods [48]. The proceedings of the colloquium have been organized into five distinct papers. Several books on ABM have become available in the last 5 years. These include introductory-level books [49], software-specific books for NetLogo [10] and Swarm [50], and books in the fields of social science [51, 52] and business management [53]. In addition, several tutorials have been published in different fields that can be useful in learning the scope of ABM [9, 54–56].

7 Discussion and Conclusions

ABM can provide insights into dynamic interactions among real-world phenomena by capturing non-linear interactions and feedback loops, thereby predicting outcomes that emerge out of complex dynamics in the real world. In communicable disease modelling, ABMs can capture indirect effects, such as ‘herd immunity’. ABMs are more generic than other modelling methods commonly used for economic evaluations. This does not imply that ABM is a replacement for these methods. Several applications can be solved efficiently using STMs, DES or compartmental models. Therefore, the modelling choice depends on several factors, including the purpose, scope and complexity of the problem, the modeller’s expertise and the available time.

ABM is needed for problems in which emergent phenomena cannot be predicted by the combination of micro-level interactions. Such phenomena can occur when interactions between the agents are important and complex, when the behaviour of agents can be changed on the basis of external factors and when agents learn from their interactions. As with other individual-level models, ABMs are needed when the averages outcomes are not sufficient and there is a need to model individual-level randomness (or stochasticity). In addition, one may use ABM when the population is heterogeneous (each agent can be different from others) and/or when agents have a spatial component to their interactions and when their position is not fixed. Finally, ABMs can be used when the study population is an open cohort.

This tutorial provides an introduction to the field of ABM and its application to economic evaluations. ABM is growing rapidly, and there are several advanced features that provide unique advantages for this modelling approach over others. Some of these features include learning of new behaviours as agents interact with other agents, use of evolutionary computation such as genetic algorithms, use of social network theory to define interactions among agents, and linking ABMs with a geographical information system so that agents traverse a realistic landscape. These advanced topics are beyond the scope of this introductory tutorial.

ABMs have several challenges and limitations. ABMs require enormous computational power to perform the individual-level simulations and several interactions among agents at each cycle. With increasing computer power, this may not remain an issue for simple to moderate-level models. However, advances in technology will provide opportunities to model increasingly complex real-world scenarios; thus, the computational challenge may not be resolved in the near future. Future research is needed to define methods for PSA, validation and calibration using ABMs. Finally, ABMs also face the same statistical issues associated with parameter estimation as do other simulation modelling frameworks such as DES.

As the applications of ABM in the healthcare field increase, more educational resources will be needed. In addition, there is a need to compare the pros and cons of ABM with commonly used modelling approaches in the context of economic evaluations. Future research is also needed in developing hybrid models that can embed features of other modelling approaches in ABM. Finally, there is a need for definition of best modelling practices, similar to the recently published series of papers by the ISPOR-SMDM Modeling Good Research Practices Task Force on the use of STM, DES and dynamic transmission modelling [57]. Though some of these recommendations apply to ABMs, a focused set of best practices needs to be defined for a complex approach such as ABM. In fact, developing standard practices sooner rather than later would help the community avoid confusing terminology and encourage development of new applications.