Biological Invasions

, Volume 16, Issue 6, pp 1273–1288

Meeting the challenge of quantitative risk assessment for genetic control techniques: a framework and some methods applied to the common Carp (Cyprinus carpio) in Australia


    • CSIRO Mathematics, Informatics and Statistics
  • Brian Leung
    • Department of BiologyMcGill University
  • Ronald Thresher
    • CSIRO Marine and Atmospheric Research
  • Jeffrey M. Dambacher
    • CSIRO Mathematics, Informatics and Statistics
  • Geoffrey R. Hosack
    • CSIRO Mathematics, Informatics and Statistics
Original Paper

DOI: 10.1007/s10530-012-0392-9

Cite this article as:
Hayes, K.R., Leung, B., Thresher, R. et al. Biol Invasions (2014) 16: 1273. doi:10.1007/s10530-012-0392-9


In Australia the European carp is widespread, environmentally damaging and difficult to control. Genetic control options are being developed for this species but risk-assessment studies to support these options have been limited. The key science challenge in this context is our limited understanding of complex and highly variable ecosystems. Hierarchical models are one way to approach this complexity and heterogeneity. These models treat the factors that determine risk as a joint probability distribution that can be factored into a series of simpler conditional distributions to allow Bayesian inference following observed outcomes. Designing a risk assessment around this approach, however, requires that the assessment endpoints (such as impacts on native species) are measurable, and that monitoring strategies are carefully designed and implemented in order that risk predictions are compared to outcomes. We therefore suggest that an evidence-based framework, supported by careful hazard analysis and quantitative risk assessment, and implemented within a stage-released protocol, is the safest way to move beyond the current emphasis on contained laboratory studies and qualitative risk assessments. We highlight impediments to this approach, and use the non-target impacts of daughterless carp in Australian billabongs as a case study to illustrate three methodological tools that not only provide solutions to some of these impediments but also encourage stakeholder participation in the risk assessment process.


Genetic controlInvasive fishRisk assessmentFault tree analysisLoop analysisBayesian networks


The common or European carp (Cyprinus carpio) was deliberately introduced into Australia in 1964 for aquaculture. It escaped and is now widely distributed throughout Victoria and New South Wales, with smaller populations in Queensland, Western Australia and Tasmania (Koehn et al. 2000; Koehn 2004). In many waterways, particularly in the Murray-Darling Basin, they are the most abundant species (Gehrke et al. 1995; Gilligan and Rayner 2007), and a growing body of evidence indicates that they have a significant impact on Australian freshwater ecosystems. For example, mesocosm experiments and experimental manipulations of carp in billabongs suggest that they can lower the biomass of aquatic plants and native fish, and increase turbidity and the intensity of algal blooms (King et al. 1997; Robertson et al. 1997; Swirepik 1999; Gehrke et al. 2011), although the impacts within any given catchment can be complex and mediated by factors such as sediment type, water temperature and depth, and wind velocity (Fletcher et al. 1985). Similar impacts of feral carp populations have been reported elsewhere in the world (Lever 2002).

The evidence of significant impacts, together with their large biomass in many waterways, have made carp a target for control and eradication programmes in Australia and elsewhere. Carp have been declared as a noxious species in most Australian states and territories and have been subject to various control programmes. Local eradication using toxicants or harvesting has had limited impact, in part because of the resilience and fecundity of the species, and in part because of the difficulty of maintaining a sustained effort. Tasmania has undertaken a multi-million dollar, 15-year campaign of netting, radio-tracking and reproductive sabotage in an attempt to eradicate carp from two lakes with mixed success (Inland Fisheries Service 2009). The wide distribution of feral populations and the limited options for widespread control have prompted efforts to examine alternative control options, including genetic options (Lapidge 2003).

In 1999 a CSIRO working group proposed genetic control, specifically via daughterless technology, as a means to eradicate carp from Australian waterways. CSIRO provided seed funding to investigate the option further and inter alia undertake an in-house risk assessment to inform future investment strategies. A subsequent risk assessment review discussed two assessment endpoints: technological failure and non-target environmental impacts (Lonsdale et al. 2002). The review highlighted the significant benefits of genetic control technologies, but also warned that the risk of technological failure was high due to the genetic challenges involved. The review team was unable to comprehensively address non-target impacts in the time available to them, but recommended that these be addressed using formal hazard identification techniques and quantitative population models.

Genetic control techniques for invasive fish have advanced considerably in the twelve years since the working group’s study. CSIRO, for example, have developed a functional female lethal construct (a gene sequence that is lethal to females but not males) for Zebrafish that is at least 80 % effective but not inherently species-specific. It is therefore timely to re-examine the challenges faced by regulators and analysts who are interested in the environmental risk of genetic control technologies for invasive fish. In this paper we define a framework to assess the environmental risks associated with gene-based control of invasive fish, and highlight tools to assist managers and biologists overcome some of the practical impediments to this framework. We use the non-target impacts of daughterless carp in Australian billabongs as a case study to illustrate some of the methodological solutions at the analyst’s disposal.

This paper is aimed at biologists and policy makers who are currently contemplating risk assessment for genetic control technologies of invasive species. The paper recommends a staged, scientific and participatory risk assessment framework for genetic bio-control technologies, and highlights hurdles to scientifically-defensible risk assessments that we believe are often overlooked in current risk assessments for this technology. The paper also describes three specific methodologies that are well suited to the recommended framework, but we emphasise that the examples presented here are primarily illustrative. Whilst they include realistic elements of the relevant technology and carp biology, they are neither comprehensive nor definitive, and are not meant to represent the outputs of a complete risk assessment.

Formulating a scientific risk-assessment framework

Genetic bio-control programs for invasive fish face three, inter-dependent, hurdles: (a) developing a reliable, cost-effective technology; b) public acceptability; and, (c) the potential for unintended consequences when releasing Genetically Modified Organisms (GMO’s) into the environment. Risk assessment methods can be designed to address all of these issues, but here we restrict attention to the latter. Risk assessments do not aim to guarantee safety, but attempt to predict the consequences of managerial actions (including no action), appropriately identify all sources of uncertainty, and characterise the effect of that uncertainty on predictions about impacts on endpoints of interest. These predictions can then be used by policy makers and managers to decide what avenues are worth pursuing, and what risks they are willing to accept, given competing demands for time and resources.

We envisage an ideal, scientific, risk-assessment framework to be evidence-based, wherein research under contained laboratory conditions informs the experimental design and risk assessment for release in contained field trials. These trials then inform monitoring strategies and risk assessment for confined, but real-world, environments (e.g. lakes, billabongs), before unconfined release in rivers or estuaries. At each stage the risk assessment should be transparent and repeatable, incorporate the beliefs and values of relevant stakeholders, and communicate its methods and results to them in an understandable manner (Hayes et al. 2007). Crucially all risk assessment predictions should be amenable to falsification, and all assumptions should be open to critical evaluation.

The key science challenge inherent in delivering this goal is how to deal with two important sources of uncertainty in complex, heterogeneous systems: (a) variability in key ecological processes; and (b) our limited knowledge of how these systems operate (epistemic uncertainty), and hence how to represent and predict their behaviour. Both issues occur at all scales of ecological organization. Heterogeneous processes at the individual level, such as phenotypic responses to genetic manipulation, and at the population level, such as the variability in growth rates and carrying capacity, create variance in fitness traits and population dynamics (Bjornstad and Grenfell 2001; Morris and Doak 2002). Community composition and dynamics can differ between locations, and community interactions generate feedback that may result in unanticipated responses to perturbations such as the stocking of a novel organism (Andow et al. 2006; Kurle et al. 2008). Additional sources of uncertainty include gene-by-environment interactions (Devlin et al. 2004), polygenic and pleiotropic effects (Gong et al. 2007), uncertainty about how to model ecological processes (Wood and Thomas 1999), observation error, evolutionary changes and environmental trends. The combination of these factors limits our ability to extrapolate results at all stages of the framework described above—i.e. from the laboratory to confined field trials, from confined field trials to confined environments and from confined environments to unconfined environments, particularly over the time scales necessary for some genetic control methods.

The limitations that uncertainty imposes on an empirical, staged-release, risk assessment framework can be countered, at least to some extent, by applying systematic hazard-analysis tools such as fault tree analysis, event tree analysis and Hazard and Operability Studies (Hayes 2002a, b; Hayes et al. 2004). These types of hazard analysis tools are typically advocated as the first step in a risk assessment (NRC 2002). Here we advocate that these tools are used repeatedly throughout the assessment process, before moving from one release stage to another. Their main purpose is to ask “what if” type questions in order to identify what and how things may go wrong (the hazards) if GM fish are progressed to the next stage of the framework. These methods cannot provide complete protection against the uncertainty-induced limitations of a staged-release framework, but they can help minimise the potential for unpleasant surprises but forcing assessors to think beyond their current operational experience in a structured and facilitated fashion.

Careful hazard identification is also an essential component of experimental design programs. A meta-analysis of experimental studies and field trials with Bacillus thuringiensis (Bt) crops, for example, suggests that laboratory studies are able to identify effects that are consistent with, or more conservative than, those found in field trials, but only when these studies are carefully designed to consider all relevant hazards (Duan et al. 2009). Laboratory studies can be controlled to represent a wide variety of conditions, including extremes that are not often seen in the real world. They cannot, however, replicate the full diversity of natural systems into which GM fish might be released and will inevitably under-represent the total variation that GM fish will experience in the field. Progressing beyond the laboratory therefore requires a protocol for the staged release of genetic bio-control agents, that is informed by, and subsequently informs, risk assessment.

Impediments to an ideal framework

There are a number of practical impediments to the framework outlined above. The first is that many existing regulatory frameworks and their associated guidance documents emphasize qualitative risk assessment (Table 1). In the context of a science-based risk assessment, qualitative risk estimates have two important drawbacks. Firstly, qualitative estimates - such as “high”, “medium” or “low” risk—cannot be empirically (in)validated unless these terms are quantitatively defined. It is not possible to take observations of an assessment or measurement endpoint and confirm whether or not an outcome or its frequency is “low” unless this term is given a quantitative definition. In practise this becomes problematic because risk assessments that are compliant with these frameworks and guidelines cannot be empirically evaluated. For example, AquaBounty Technologies Incorporated (2010) state that the likelihood of genetically engineered Salmon escaping from their facilities is “extremely small”. This term however is not defined so it is impossible to design a monitoring strategy to test compliance with this prediction with any degree of confidence. Qualitative estimates such as these may be very useful for the purposes of communicating risk, but they are not well suited to a staged-release protocol that relies on monitoring, and transparently comparing risk predictions with outcomes, before moving to the next stage.
Table 1

Summary of existing regulatory frameworks and risk assessment approaches for genetically modified organisms



RA approach

Key legislation and technical guidance

United States

Environmental Protection Agency, Food and Drug Adminstration, United States Department of Agriculture

Qualitative risk assessment

Federal Food, Drug, and Cosmetic Act; Coordinated Framework for the Regulation of Biotechnology 1986


Office of the Gene Technology Regulator (OGTR)

Qualitative risk assessment

Gene Technology Act 2000; Risk assessment framework for license applications to the Office of the Gene Technology Regulator, November 2001

New Zealand

Environmental Risk Management Authority (ERMA)

Risk Assessment (qualitative or quantitative)

Hazardous Substances and New Organisms Act 1996; Biosecurity Act 1993; ERMA Technical Guides ER-TG-01-1 9/99 and ER-TG-03-1 7/00

United Kingdom

Department for Environment Food and Rural Affairs

Risk assessment (unspecified “state of the art” techniques)

EU Directive 2001/18; The UK Department of Environment Transport and Regions (DETR) guidance on the principles of risk assessment and monitoring for the release of GMOs


Fisheries and Oceans Canada (DFO)

Qualitative risk assessment

Canadian National Code on Introductions and Transfers of Aquatic Organisms (

Secondly, qualitative risk estimates mix together three sources of uncertainty in risk assessment: epistemic uncertainty, variability and uncertainty associated with language (linguistic uncertainty). Two important types of linguistic uncertainty are vagueness where the boundaries of the thing being described are not clearly delineated, and context-dependence, where the meaning of a word varies depending on the context, and this is not adequately described (Regan et al. 2002; Burgman 2005). The terms used in qualitative risk estimates are often vague and context-dependent, hence the effects of variability and epistemic uncertainty on the assessment’s predictions cannot be separated from the meaning (or different interpretations) of terms such as “low risk”. Uncertainty cannot be measured effectively on an ordinal scale for the same reason. Statements such as “high confidence” inevitably beg questions such as: what does “high” mean, and is “high” high enough for the purposes of a stage-released framework?

These limitations can be overcome by providing numerical definitions for terms such as high, medium or low, tailored to the measurement or assessment endpoints of the risk assessment. Numerical definitions can be presented as precise scalars (e.g. low means 1 occurrence per annum), precise intervals (e.g. low means between 0 and 5 occurrences per annum), or as fuzzy measures that use membership functions, on the range [0,1], to describe the extent to which vague and ambiguous terms belong to precise sets or intervals (Klir and Folger 1988). Numerical definitions can eliminate the most important sources of linguistic uncertainty in a risk assessment, and can also be used to test for, and correct, a variety of undesirable features of the risk matrix that forms the centrepiece of many qualitative risk assessments (Cox et al. 2005; Cox 2008; Hayes 2011).

The second impediment to the risk assessment framework outlined in the previous section is that quantitative methods require a higher degree of technical expertise, and this can marginalise stakeholders who are not familiar with them. For example, if quantitative models are not constructed in a participatory fashion they may exclude the opinions and beliefs of stakeholders, and this could cause distrust in the assessment process and its conclusions. Presenting stakeholders with a set of differential equations, however, is not a good way to include them in a model-building exercise. Stakeholder trust in a risk assessment can be engendered, at least partially, by placing an emphasis on their participation throughout the risk assessment process, but in a quantitative assessment this needs to be facilitated by methods that quantify stakeholders beliefs about uncertain variables (Kuhnert et al. 2010) and pictorial representations of process models, via (for example) influence diagrams, fault trees or signed directed graphs, and conditional probabilistic relationships via Directed Acyclic Graphs (“Case study: Daughterless carp in Australia” section).

The third important impediment to a quantitative staged-approach to bio-control risk assessment is uncertainty and sensitivity to assumptions. Important assumptions surround the structure of the quantitative risk assessment model(s), the probability distributions used to represent variable parameters in the model, and how dependency between uncertain parameters is handled and propagated through the risk assessment. In this context, however, it is important to recognise that qualitative assessments are not free of these issues, they are simply more transparent in quantitative assessments.

Strategies to help meet these challenges include developing and testing the effects of alternative model structures for the problem in hand (Pascual et al. 1997; Ellison 2004), employing imprecise probability methods to propagate the effects of epistemic uncertainty and variability separately through the risk assessment (Ferson and Hajagos 2004; Ferson et al. 2004), using the hazard analysis techniques highlighted above to identify factors that are important determinants of the overall risk, and then build multi-level statistical models that reflect, and account for, the site- or species-specific variability associated with these factors (Clark 2007; Arhonditsis et al. 2008).

Case study: Daughterless carp in Australia

The risk assessment activities that have accompanied the development of the daughterless carp project highlight some of the issues identified in “Impediments to an ideal framework” section. Since the initial CSIRO risk review, the project has been subject to five additional formal processes that wholly or in part address the hazards and risks associated with the genetic control technology, including an explicit public awareness and consultation campaign (Table 2).
Table 2

History of hazard analysis and risk assessment for the genetic control of common carp in Australia




Recommendations and outcomes

February 2002

Internal CSIRO Review of Risks of the Daughterless Technology for the Control of carp in Australia (Lonsdale et al. 2002)

Expert panel

Listed and evaluated risks associated with technical, regulatory, management and non-target ecological impacts. Recommended project proceed in laboratory setting, continued engagement with regulators and public, assessment of species-specificity of technology, and internal CSIRO Hazard Analysis to scope out potential field program and regulatory environment

April 2002

Daughterless carp Technology Business Plan (Fung and Yau 2002)

Facilitated stakeholder workshop

Recommendation for project development and support. Specification of staged technology testing (laboratory, controlled field trials and full release), and need for parallel development of communication and population dynamic strategies

March 2003

National carp Control Workshop (Lapidge 2003)

Facilitated public symposium

Review of technology development and regulatory environment, leading to recommendations for formal development of Communications, Integrated Pest management and Risk Management Strategies to complement technical program


Analysis of legal, technical and other risks (Dall and Neumann 2004)

Qualitative risk assessment in accordance with AS/NZ 4360:1999

List of possible hazards, risks and impediments to project technological development and implementation, with partial lists of possible remediations/actions

January 2005

Survey of community attitudes to using gene technology methods for managing common carp (Fisher and Cribb 2005)

Professionally organised focus-group and telephone surveys

Strong support for continued laboratory development of technology, but concerns about safety (species-specificity) and need for full and transparent public consultation and approval before field trials

April 2008

An Independent Review of the Freshwater Products and Strategies Program (Hoey et al. 2008)

Expert panel

Continue support for laboratory studies, including alternative genetic options (e.g., inducible fatality), accelerate current project reviewing policy issues on use of the technology and enhance dialogue with national regulatory body (Office of Gene Technology Regulator) about possible impediments to implementation

The result of these processes is that three risk areas have been canvassed: (1) the risk of the project failing to deliver the technology (high), for reasons such as inadequate stocking effort or unquantified, but “known” biological confounding variables, such as gene silencing and density-dependent population dynamics in the target population (Bax and Thresher 2009); (2) the risk that it will conflict with policy and legislative mandates regarding GM technology (low to moderate); and, (3) the risk the public will not accept it (low to moderate). With regard to the last, a formal public consultative process (Fisher and Cribb 2005) found strong public support for continued development of the science, but also highlighted two key public concerns: that the technology be species-specific (and hence minimizes the risk of direct collateral damage to native fish), and that a full and fully transparent public approval process be undertaken before fish are released into the wild.

With the exception of the population modelling conducted by Bax and Thresher (2009), all of the risk assessment activity conducted for the daughterless carp project has been qualitative, and generally resulted in conservative outcomes, namely continued support but only for contained laboratory development. These assessments have also identified key drivers of public concern and safety, namely species-specificity and the potential ecological impacts of genetic control programmes. Progress in this respect has been partly hampered by the lack of a clear understanding of the genetic options being considered, but also hindered by the technical and methodological challenges of quantitative ecological risk assessment discussed in “Impediments to an ideal framework” section.

In the following sections we highlight three methods to help overcome some of these challenges and facilitate progress towards the scientific framework outlined in “Formulating a scientific risk-assessment framework” section. We focus on an endpoint that reflects one of the public’s main concerns about genetic control technology: its effects on native species. In this context, Dall and Neumann (2004) identified horizontal gene flow to species other than carp, and disruption of ecosystems due to stocking of genetically modified carp, as two of the highest priority hazards associated with the daughterless project. We use fault tree analysis to investigate the former and loop analysis to investigate the latter. We then use Bayesian networks to gauge the evidence for different ecosystem models. For the purposes of this demonstration, we will assume that genetic control is achieved via a female-lethal construct, consisting of carp-native genes, that will kill female carp before they can reproduce, but is not inherently species-specific—i.e. if incorporated into the genome of a native fish, it will kill female offspring.

Fault tree for horizontal gene flow

The qualitative risk assessment conducted by Dall and Neumann (2004) identified two processes for the spread of the female-lethal construct: vertical transfer via normal breeding between a carrier and a non-carrier and subsequent Mendelian inheritance, and horizontal transfer by means of a virus. The group considered that, in an Australian context, the risk of vertical transfer of the constructs due to hybridisation with native species is deemed extremely low because there are no native cyprinids in Australia. They therefore concluded that the only plausible hazard in this context is horizontal transfer.

Figures 1 and 2 show the results of a fault tree analysis for viral-mediated horizontal gene flow from carp carrying the female-lethal (FL) gene to native species. Note this analysis focusses on the conditions necessary for gene flow to individual native fish. Introgression of the FL gene to a native population would require further steps that are discussed in detail by Kapuscinksi et al. (2007).
Fig. 1

Fault tree depicting the events leading up to viral mediated horizontal gene flow from daughterless carp carrying the female lethal (FL) gene to native fish
Fig. 2

Continuation of the fault tree depicting the events leading up to viral mediated horizontal gene flow from daughterless carp carrying the female lethal (FL) gene to native fish

Fault trees were originally developed to identify failure scenarios in large engineering systems, such as nuclear power stations and petro-chemical plants (Pate-Cornell 1984; Kletz 1999), but they can also be used as a graphical hazard and risk assessment tool in ecological contexts (Hayes 2002a). Fault trees identify the causative chain of events that lead to the occurrence of a hazard (the “top event”) using two logic functions. One function requires that all preceding conditions must be met (an “AND” gate), whilst the other requires only one preceding condition to be met (an “OR” gate), in order to progress through the gate. The causative events are laid out in a tree with the branches connected by one of these two gates. In this manner they break the sequences of events that lead to the top event down into its contributing parts.

The fault trees developed here show how viral mediated gene flow requires the genetically modified carp to be infected with a viable, replicating virus, together with RNA-RNA, or DNA-DNA, recombination of the entire female-lethal gene into the virus genome, and then subsequent integration into the genome of a native fish, again either through RNA-RNA recombination or DNA-DNA recombination, directly into a native fish, or via circuitous pathways involving secondary fish hosts or secondary viral hosts. The key limiting steps in the process are considered to be: (a) the integration of the entire female-lethal gene into the viral genome (although the probability of this event might be increased if Cauliflower Mosaic Virus (CMV) sequences are present in the construct); and, (b) the native fish is a suitable host for the virus (personal communication Stan Roberts, CSIRO Marine and Atmospheric Research).

The utility of fault trees as a hazard analysis tool lie in their transparent graphical depiction of potentially complex events, and in the fact that they force the assessor(s) to think very carefully about all the possible ways an undesired event can occur. This process can also identify ways to minimise risk, for example by avoiding CMV-based constructs, and identify potentially important pieces of information, such as the prevalence of viruses that can infect carp and native fish. Furthermore if data is available, or can be elicited, regarding the probability of the steps depicted in the tree, they can also be used to quantify the overall probability of the top event (Bedford and Cooke 2001). They are therefore useful as a qualitative hazard identification tool and as a quantitative risk assessment tool.

Loop analysis for ecological impacts

The other hazard high on the list of public concerns surrounding genetic control techniques is the ecosystem impacts associated with stocking genetically modified fish into aquatic ecosystems. In this section we use loop analysis, also known as qualitative mathematics or qualitative modelling, to investigate the direct and indirect effects that stocking, followed by the eventual removal, of carp may have on an Australian billabong.

Loop analysis is a quantitative technique that treats the growth rates of n interacting populations of species as a system of deterministic, Lotka-Volterra equations
$$ \frac{dN_i}{N_idt}=\sum_{j=1}^n\alpha_{ij}N_j+\beta_i-\delta_i+\iota_i-\varepsilon_i \quad (i=1,\cdots,n) $$
where the growth of each population is determined by its abundance Ni, a vector of density-independent growth rate parameters - birth (βi), death (δi), immigration \((\iota_i)\), and emigration \((\varepsilon_i)\) - and interactions between species captured by the “community matrix” \(\mathbf{A}=\alpha_{ij}\) (Levins 1974).

Loop analysis is an intermediate step between completely qualitative representations of a system, such as cartoons or influence diagrams, and a fully quantitative ecosystem model. It achieves this by restricting the elements of \(\mathbf{A}=\alpha_{ij}\) to only the sign (+, −, 0) of the interaction terms. In this way it concentrates on how a system’s structure determines the way in which variables within this system will respond to a “press perturbation”, defined as a sustained change to a rate of birth, death or migration of a species, or the equivalent increase or decrease in mass, value or flow of other non-biological variables in the system (Levins 1974; Dambacher et al. 2002).

In loop analysis the system structure is defined by the variables of the system and the sign of the relationships by which they are linked. These variables and relationships are portrayed by sign directed graphs (SDGs), where a link from one variable to another ending in an arrow \((\rightarrow)\) represents a positive direct effect, such as births produced by consumption of prey, and a link ending in a filled circle ( represents a negative direct effect, such as death from predation. All possible ecological relationships can be described in this manner: predator-prey or parasitism (, mutualism \((\leftrightarrow)\), commensalism \((\rightarrow)\), interference competition (, and amensalism ( Self-effects are shown by links originating and ending in the same variable, and are typically negative, as in self-regulated variables, but can also be positive where variables are self-enhancing. While loop analysis ignores the strength of the pairwise relationships in the SDG, it can provide the basis for models that explore the effects of interaction strength (Dambacher et al. 2003; Metcalf et al. 2008; Hosack et al. 2009; Novak et al. 2011). Furthermore, the interactions in the SDG are typically considered to be fixed and independent of population size. However, there can be interactions that are modified by the abundance of a third variable, which creates additional direct effects in the system that can be considered in a qualitative analysis (Dambacher and Ramos-Jiliberto 2007).

The models portrayed in Figs. 3 and 4 are based on the trophic relationships depicted in Gehrke et al. (2010) and Colvin (2012). Gehrke et al. (2010) presents a food-web for Australian rivers that include carp, and uses this to hypothesize the effect of removing carp populations from these ecosystems. Similarly, Colvin (2012) presents a conceptual model of positive and negative interactions between the physical and biological components of Clear Water Lake in Iowa, which includes the impact of Carp on other components of the Lake ecosystem. Here we present four models showing the effects of stocking and genetic control under flood and drought conditions. The models’ subsequent predictions are summarised in Table 3. The reliability of qualitative model predictions is based on an analysis of the sign determinacy of the weighted predictions matrix (Dambacher et al. 2003; Hosack et al. 2008) and here we only report predictions whose sign has at least an 80 % chance of being correct, under the assumptions of the model.
Fig. 3

Signed directed graphs depicting an ecosystem model of an Australian billabong with stocking and genetic-control of non-native carp under drought conditions. These models allow us to postulate the ecosystem impacts of stocking genetically modified carp, and their eventual genetic control. Variables are 1: Carp, 2: Small native fish (SNatFi), 3: Benthic invertebrates (BenInv), 4: Large native fish (LNatFi), 5: Large zooplankton (LZooPl), 6: Small zooplankton (SZooPl), 7: Phytoplankton (Phytop), 8: Macrophytes (Macrop), 9: Benthic algae (BenAlg), 10: Suspended sediment (SusSed), 11: Drought (Drough) and Flood, 12: Stocking (Stock) and Genetic control (GenCon). Models are based on the trophic interactions depicted by (Gehrke et al. 2010) and the unpublished work of Colvin et al. (
Fig. 4

Signed directed graphs depicting an ecosystem model of an Australian billabong with stocking and genetic control of non-native carp under flood conditions. Model amends the drought model by removing the negative influence of carp on suspended sediment, which under flood conditions is dominated by inputs from flooding rather than carp. Variables are 1: Carp, 2: Small native fish (SNatFi), 3: Benthic invertebrates (BenInv), 4: Large native fish (LNatFi), 5: Large zooplankton (LZooPl), 6: Small zooplankton (SZooPl), 7: Phytoplankton (Phytop), 8: Macrophytes (Macrop), 9: Benthic algae (BenAlg), 10: Suspended sediment (SusSed), 11: Drought (Drough) and Flood, 12: Stocking (Stock) and Genetic control (GenCon)

Table 3

Qualitative model predictions for four models shown in Figs. 3 and 4






1. Carp


(+ )


(+ )

2. Small native fish

(+ )


(+ )


3. Benthic invertebrates

(+ )




4. Large native fish

(+ )




5. Large zooplankton





6. Small zooplankton


(+ )


(+ )

7. Phytoplankton

(+ )


(+ )


8. Macrophytes

(+ )


(+ )


9. Benthic algae

(+ )




10. Suspended sediment


(+ )



Direction of change for predictions with a high (≥0.8) degree of sign determinacy are shown as either positive (+) or negative (−). Ambiguous predictions with a low (<0.8) degree of sign determinacy are shown as “?”. Variables that are predicted to remain unchanged with a high degree of sign determinacy are shown as “0”

The results of this analysis suggest that populations of large and small native fish could decline as carp numbers are increased during stocking. This situation is reversed, however, as the female-lethal gene spreads through the carp population and their numbers decline, although the prediction for large native fish is less certain under flood conditions. The model also predicts that benthic invertebrates, phytoplankton, macrophytes and benthic algae will also increase as carp numbers decline, but small phytoplankton biomass will decrease, where again the predictions for benthic invertebrates and benthic algae are less certain under flood conditions (Table 3). The model predictions for native fish and large zooplankton (biomass increase but with low sign determinacy) agree with observed response of these groups to experimental removal of carp from Australian billabongs (Gehrke et al. 2011). These experiments also reported no change in the abundance of small phytoplankton following carp removal which is at odds with the model predictions.

One of the key advantages of loop analysis is that the effects of model structure uncertainty are easily propagated through the assessment. The signed directed graphs can be quickly constructed allowing a range of different conceptions, for example, from different stakeholders, to be analysed and the potential effects of model structure uncertainty investigated. The system’s stability properties and predicted response to press perturbations can also be assessed using existing methods and software (Dambacher et al. 2003). Note that the models portrayed here were developed following a sensitivity analysis of an initial structure based on Colvin’s et al. model that included strong positive feedback between suspended nutrients and phytoplankton. A sensitivity analysis using the techniques described in Hosack et al. (2009) indicated that the system’s overall stability was highly sensitive to this positive feedback, and plausible unambiguous predictions (such as carp increase with stocking) could only be achieved without this feedback cycle. This indicates that in a fully quantitative model the parameterisation of these sub-systems will be especially important to overall model predictions.

Bayes network analysis of model structure

The term “Bayesian network” was coined by Pearl (1986) to describe the “dependency-graph” representation of any joint probability density \(p(x_1, \cdots, x_n)\). The graphical representation of the joint density is achieved via a directed acyclic graph (DAG), that consists of a set of nodes linked by directed (one-way) arrows that indicate the conditional relationship between nodes. Nodes are comprised of states that are independent, mutually exclusive and exhaustive propositions about the values that the variable represented by the node can take. The arrows between nodes describe the particular product-rule decomposition of the joint density that in turn reflects the presumed or inferred cause and effect relationship in the system being studied. For example, this factorisation of a three-variable joint density p(x1x2x3) = p(x3|x1x2)p(x2|x1)p(x1) represents a unique DAG with two arrows linking the node x3 to its “parents” (x1,x2) to represent the factor p(x3|x1, x2), and one arrow linking the node x2 to its parent x1 to represent the factor p(x2|x1) (Bishop 2006).

In this section we use Bayesian networks to gauge the extent to which alternative loop analysis models are consistent with observations. Bayesian networks are one of the few uncertainty analysis methods that can perform forward uncertainty propagation based entirely on expert opinion, but also statistical inference when data becomes available. In this instance we use Gehrke et al.’s observations of the community response following removal of carp from two Australian billabongs and test the fidelity of the loop analysis predictions to these observations.

The DAG at the top of Fig. 5 shows the predicted response of the billabong community with the prior probability that the Carp4A drought control model is an adequate representation of that system set to 1. Notice that these predictions are consistent with those of the model shown in Table 3. The probabilities of the direction of change implied by this model were estimated using quantitative simulation studies designed to test the sign determinacy of SDG’s, together with a series of assumptions regarding inter alia ecosystem transfer efficiencies and the linear dynamics implied by Eq. 1. These studies were designed to test how often the direction of change predicted by loop analysis was consistent with the direction of change predicted by a fully quantitative model, where the latter was developed by randomly parameterising the elements of the community matrix A subject to trophic transfer efficiency constraints (Dambacher et al. 2003; Hosack et al. 2008).
Fig. 5

Two Directed Acyclic Graphs (DAGs) showing (top) the predicted probability of increase, decrease or no change in the functional groups of an Australian billabong following the removal of carp, assuming that the Carp4A drought control model is an adequate representation of the billabong ecosystem. (Bottom) DAG showing the posterior probability of the Carp4A and Carp4B model conditioning on the observed response of the billabong to the removal of carp

In the DAG at the bottom of Fig. 5 we have imposed the observations of Gehrke et al. (2011). The Bayes Net uses this information to update the prior probability that the loop analysis model is an adequate representation of the system. Here we have allowed for one alternative representation of the system, the Carp4B drought control model, that does not include a strong positive effect of carp on suspended sediment. The analysis also includes a “null model” that allocates equal probabilities of observing an increase, decrease, or no response across every possible prediction, given a press perturbation. The null model acts as a benchmark to judge the performance of the other models.

The results of this analysis indicate that Gehrke’s observations are more consistent with model 4B than model 4A. It is important to recognise, however, that this is a relatively simple inferential approach that serves best as a heuristic procedure to guide model building. It is not meant to replace the hierarchical analysis of the joint distribution of the risk factors discussed above, and it does not represent a formal model selection procedure within this context. The analysis shows that model 4A is unlikely to be an adequate representation of the system, and the 20 % posterior probability allocated to the null model warns us that other representations of the billabong ecosystem are plausible and if they were included in the analysis may be more probable than any of those that are represented here.

Summary and conclusions

Risk assessment plays a critical role in the regulatory frameworks for genetic pest control of many nations. As with all biological stressors, however, risk assessment for genetic control methods face substantial challenges in dealing with selection and development of appropriate risk models, environmental variability and epistemic uncertainty. The use of genetic technology also creates high levels of public interest that often lead to demands for a much greater level of analysis and transparency than that typically associated with conventional methods of pest control.

Qualitative risk assessments are relatively simple to implement, can be more understandable to the lay person, and are often less constrained than quantitative assessments (Woodbridge 2008), although national and international standards do prescribe approaches in some application domains. Qualitative methods play an important role in all risk assessments, and here we have demonstrated how the construction of a fault tree, for example, can assist in the hazard analysis stage of a risk assessment. Qualitative risk estimates, however, cannot be compared with actual outcomes, and they cannot coherently express uncertainty. We therefore assert that the challenges associated with risk assessment for genetic control methods can be best overcome by making quantitative risk estimates, within a participatory, stage-released protocol that moves from laboratory studies through to contained, small-scale field tests and, where appropriate, larger scale open release. The two quantitative methods highlighted here are well suited to participatory approaches to risk assessment because they use graphical representations of complex systems. Each of these methods can serve as pre-cursors to, or components of, a complete evaluation of the potential ecosystem impacts of genetically modified carp.

It is important to emphasise that the results portrayed here are realistic but hypothetical, and if the methods described above were to be applied to a real problem they should involve a much wider group of stakeholders and experts than we have canvassed. It is also important to emphasise that these are only a few techniques from among a much bigger tool-box available to risk analyst, and there are range of other relevant issues associated with the risk acceptance criteria and the power of monitoring strategies, that we have not discussed in this article. Kapuscinski et al. (2007) provide comprehensive guidance on these issues, within the context of genetically modified fish.

Results such as Duan et al. (2009) lend some credence to the stage-release model advocated here although we recognise that laboratory studies and small scale field trials are inherently more difficult for genetically modified vertebrates and invertebrates than they are for plants, and that the effects of gene by environment interactions may be more significant in these groups than they are in plants. This places additional logistic and methodological hurdles on genetic control technologies for fish.

In terms of risk assessment methodology, we believe that there are a number of strategies to help overcome these hurdles:
  • choosing endpoints carefully and specifically: before the risk assessment, dialogue between stakeholders should identify the simplest possible endpoints that are sufficient for management purposes. Endpoints that occur at the end of complex casual chains are harder to predict reliably;

  • undertaking systematic hazard analysis and scenario analysis using methods that postulate ’what if’ scenarios and ask the question ’what can go wrong’. These help guide the risk assessment and monitoring strategies by identifying as complete a set of risk factors as is possible;

  • using formal elicitation methods to identify and make specific the conceptual models and subjective beliefs that underlie all risk assessments, qualitative or quantitative. Formal elicitation methods help avoid cognitive bias and psychological frailties during the process of expressing models and beliefs (Kynn 2008);

  • using both statistical (empirical) and process-based modelling techniques, as these can be mutually informative and ultimately merged in hierarchical, data-assimilating methods. The significant computational hurdles associated with these methods are being met, but often in non-ecological disciplines. Migrating these methods into ecological practise will help address the total uncertainty in the risk assessment problem;

  • monitoring and testing predictions against observations. Although sounding trite, accurate and precise monitoring and testing can be empirically challenging, and these challenges should be considered early in the analysis rather than later.

Although there are a large number of interacting factors in the natural world, the number of risk assessment endpoints can usually be limited through a careful dialogue between managers, risk analysts and stakeholders. This allows analysts to manage the complexity of the real world by treating the risk assessment endpoints as conditional probability distributions, conditional on the factors that are known to influence them, but with additional variability attributable to factors that we are either not aware of, or that we consciously decide to group together to keep the study manageable. This approach to risk assessment embeds understanding of the processes that influence risk into a statistical model, specifically developing a statistical model that reflects the variability and influence of the conditioning factors that we know influence risk outcomes, but which includes a process error term that captures the effect of factors that are not explicitly recognised in the model. This approach organises the joint distribution of factors known to influence risk into a hierarchical model that can be split into a series of simpler conditional distributions to allow Bayesian inference following observed outcomes (Clark and Gelfand 2006; Cressie et al. 2009).

Implementing such an approach requires inter alia that the risk assessment endpoints are measurable, and that monitoring strategies are carefully designed and implemented in order that risk predictions are compared to outcomes. To date, however, all research involving genetic control options for invasive fish has been conducted in contained laboratory settings and there are no observed outcomes in the field that we can use to parameterise the statistical models discussed above.

Monitoring forms an important component of virtually all modern environmental management paradigms (risk assessment, management strategy evaluation, integrated pest management, ecosystem-based management) and it is the center piece of the staged-released strategy envisioned here. Monitoring and (in)validation of risk predictions are also essential to any science-based approach to risk assessment. Importantly, monitoring strategies should be aimed at the entire risk-generating process and not just focussed on the endpoints. The formal hazard tools and model building strategies identified here will help to identify interim steps that contribute to the overall risk assessment. Some of these steps may be easier or more cost-effective to monitor than the endpoint itself, particularly if this endpoint has a low likelihood.

Currently investment decisions regarding GM control technologies for invasive fish reflect (perhaps unconscious) risk assessments by funding agencies about the risks of GM technologies failing to meet their objectives either in the laboratory or the field. The risk of ecological damage is consciously managed through physical containment. If GM control technologies are to become a reality, however, these risks will have to be managed in some other way. We believe that a stage-released protocol where containment controls are lifted gradually based on the evidence they provide and carefully designed risk assessments studies is the safest alternative.


We sincerely thank Anne Kapuscinski and Leah Sharpe for organising, and the Minnesota Sea Grant Program for supporting the international symposium on which this paper is based, and the diverse participants in the symposium, whose insightful comments on the emerging technology greatly facilitated development of the ideas expressed in this paper. We are indebted to Stan Roberts (CSIRO Marine and Atmospheric Research) for useful comments on the fault tree analysis for viral mediated gene flow. We also thank Peter Caley (CSIRO) and two anonymous reviewers for comments that helped improve the manuscript.

Copyright information

© Springer Science+Business Media Dordrecht 2013