Survey results are organised by credibility, legitimacy and saliency attributes of effective evidence-based decision-making (Cash et al. 2003) with attention to both the physical ‘hardware’ (WRM infrastructure and resource commitments) and the institutional ‘software’ (governmental roles and shared knowledge capacities within each basin) that acted to enable or prevent the effective use of WRMs to guide water resource management decisions.
Credibility
The CRSS was widely considered scientifically credible by all respondents across the CRB, but not without critiques. Survey respondents noted the modelling software platform, hydrologic data and methods used to evaluate uncertainty are well supported by peer-reviewed literature, and equations simulating water management are structured as hierarchical rules that can be readily mapped onto the Law of the River. Throughout the basin, water demands were provided by the individual states based on population projections. However, assumptions of exaggerated future demands from the upper basin states to safeguard future allocations challenged the credibility of WRM projections. To manage this critique, the USBR documented model assumptions thoroughly during the production of each EIS so that decisions could be accountable to the collaborative decision-making process. Maintaining a cooperative working environment with the upper basin states required accepting these aspirational demands.
In the MDB, the tools used to prepare the Basin Plan 2012 can be distinguished from the subsequent platform and model developments. The historical sub-basin models—IQQM, REALM and MSM-BIGMOD—were generally perceived to be scientifically credible due to the regard held for the model developers and the established institutional usage of these models. The scientific credibility of the new Source software and the models actively being developed in this framework was supported by detailed documentation describing guidelines for model development (Black et al. 2014), quality control measures and procedures (MDBA 2012) and the scrutiny during the development of its internal algorithms (Welsh and Black 2010). The development of new models using this platform has however attracted some criticism among various non-federal stakeholders. One respondent stated they ‘have often been frustrated by the lack of evidence to support the modelling of the Murray-Darling Basin Authority’. Perceptions of credibility were influenced by limited access to the models, which are generally owned by individual states, but the credibility of the scientific methods themselves were not directly questioned by respondents.
Legitimacy
According to respondents, the legitimacy of the CRSS was largely attributed to the direct access to the model given to stakeholders and the transparency that the platform provides. The USBR-developed model is freely available to participants through the Stakeholder Modelling Workgroup; however, users must purchase a RiverWare software licence to be able to modify the model or hire external consultants to do so. Survey respondents commented that the ability for stakeholders to analyse, operate, challenge and modify the CRSS helped them gain a sense of ownership during the interstate negotiations that resulted in the EISs and allowed them to formulate new alternatives to be considered. Perceptions of model legitimacy among various non-modelling stakeholders such as some Native American tribes were not included in this assessment. These groups often rely on the USBR to interpret the information that the CRSS produces.
Within the MDB, the legitimacy of historical WRMs was initially established through their relative simplicity at the time of initial development, followed by the continuous use and enhancement across state offices. As new models were developed using the Source platform, expert input and algorithms from the legacy models have been incorporated; thus, the developers are hopeful their legitimacy would transfer as well. While this is logical from an engineering perspective, cautions to conform to a basin-wide norm could be seen. State governments are generally responsible for developing their own local Source models while managing and stakeholder engagement with respect to WRMs and their outputs. Although the move to the new platform was generally seen as positive, some state’s constituents expressed hesitations. One respondent noted that their opportunity to provide feedback was limited to the results of the models and not the models themselves. Another respondent questioned the accessibility of the model when claiming the ‘custom approach, along with limited documentation, makes use of the models other than (by) the MDBA very difficult’. The prolonged adoption of the platform and implementation of new models reflects a degree of caution among the states or the constituents they represent. Problems of institutional fragmentation and historical model developments have challenged coordination and boundary work efforts between government and civil society, among state governments and between states and the MDBA.
The perception of legitimacy of the WRMs in both basins is presumably affected by the degree of inclusion in the modelling process. Sufficient knowledge of WRMs, the removal of barriers which inhibit their usage, and a willingness to share and learn from them are prerequisites to fully realize and leverage their potential value.
Saliency
In order to fulfil their respective mandates to manage water resources sustainably into the future, the USBR and MDBA are the primary drivers of new model development. One challenge facing both basins was the evolution of water management concerns that required the function and scope of their WRMs to expand. In the case of the CRB, the saliency of the CRSS model remained strong among federal and state governments; however, NGOs found the narrowness of the design to be a significant limitation when modelling environmental objectives. As one respondent noted, ‘The fact that environmental flows are not assessed likely makes it easier for water managers to make decisions, as you are less likely to be concerned if you aren’t getting information that says you should be concerned’. This can raise significant information equity issues. As one respondent observed, ‘Stakeholders could be disadvantaged because the model is not geared/suited towards their needs’. This suggests the need for active management to avoid the inadequacy of models inadvertently or intentionally foreclosing affected interests.
While inclusion or exclusion of particular aspects in a WRM is subject to debate, respondents agreed that complex management decisions are difficult if not impossible to make without a basin-wide model. Concurrently, many respondents recognised that models are generally not able to provide single best solutions to resolve water management conflicts. This ‘necessary but insufficient’ role of WRMs was exemplified during the negotiations to develop the EISs when solutions proposed by one state or coalition would undergo a technical review by other parties using the CRSS. These reviews would occur alongside evaluations from legal and strategic policy-oriented perspectives before acceptance or counterproposals could be offered. This process would iterate until mutually agreeable outcomes could be reached. Throughout the development of Minute 319, similar iterations of proposed solutions followed by legal, political and technical analyses occurred between the two nations.
As the experience of WRM development and application in the MDB highlights, issues such as severe drought can put enormous pressure on models to deliver immediate results to support the development of new water management guidelines. WRMs developed using historical conditions must be adapted to reflect unforeseen extremes resulting from non-stationarity (Milly et al. 2008). The need for integrated models to provide highly relevant decision support for basin-wide policies became increasingly clear. While many respondents highlighted that model saliency became a focus in terms of their capabilities, strengths and limitations, another respondent astutely noted that ‘A lot of energy can be expended in arguing about the model rather than what its saying’.
The responses also provided evidence that the WRMs were performing the functions of boundary management: convening, translation, collaboration and mediation (Cash et al. 2006) and therefore we organise other key insights by these functions.
Convening
An important initial function of boundary work is convening affected parties, but this task can face numerous logistical challenges such as the availability of sufficient funding to enable stakeholder input, or context-specific challenges such as the selection of participants (Glicken 2000). This latter is delicate as excluding certain participants can delegitimize any boundary work before it begins, while including too many voices can render a discussion ineffective. Convening stakeholders in a federal context poses unique challenges of representation. The degree to which interests are adequately represented by subnational governments depends on the democratic nature and political focus of the government. As an example, non-economic interests may not be on the agenda of governments in certain regions, and the inclusion of NGOs may be necessary to voice these concerns. Establishing clear objectives at the outset can assist with deciding the composition of a group (Liu et al. 2008), but can also prematurely influence the outcome through inclusion or exclusion of interrelated topics. This comparative analysis highlights that the capacity of water management institutions to work with WRMs and for stakeholders to engage in WRM information is key to the function of convening.
Development of the CRSS for managing the CRB began in 1970s by the USBR and from 1973 to 1978, it was ‘given serious scrutiny and many changes were made to solve some of the problems and strengthen some of areas of weakness that had been detected’ (Cowan et al. 1981: p. 11). Participation from the states likely began then, but one respondent to our questionnaire surmised that the ‘entities least satisfied [today] with Colorado River Management were not at the table: tribes, conservation organizations, Mexico’. While the lack of inclusion of these stakeholders certainly simplified its initial development, agreements over future shared management would eventually require integration of these affected parties into the modelling process.
During the interstate negotiations leading to the development of recent guidelines and agreements, the states increased their in-house modelling capacity and began to operate the CRSS independently of the USBR. Shortly thereafter, NGOs also invested in their own modelling expertise, which allowed greater access to, and understanding of, the alternatives under consideration. Within the USA, the USBR facilitates a Stakeholder Modelling Workgroup that includes any stakeholder that ‘actively runs the model or uses its results’, which continues to be a key forum to share model assumptions, structure and outputs. Through this technical working group, relationships that were developed throughout the negotiations are maintained and continue to be a cornerstone of the acceptance of the CRSS. Upon commencement of the binational negotiations that resulted in Minute 319, the USBR also allowed the Mexican government to access the CRSS. This allowed Mexico to ask informed questions regarding how the USA administered their treaty allocation and better understand management process of the Colorado River. Throughout the binational negotiations, interstate and international committees of technical stakeholder representatives convened regularly to explore mechanisms of cooperation.
In the MDB, the need to meet the obligations under the Water Act 2007 and to determine the SDLs became a major impetus to develop a nationally consistent approach to modelling for water management and planning. Significant efforts were made to engage and inform stakeholders throughout the development of the Basin Plan (MDBA 2009). State water agencies provided the models used to develop the integrated modelling framework and offered comments throughout the process, but convening of technical individuals across the states during the integrated WRM development and application was less apparent than in the CRB (MDBA 2010a; MDBA 2010b; MDBA 2012). Since the development of the new Source modelling platform began, significant efforts have been made to convene stakeholders. Early workshops were held across the basin to elicit user requirements from eWater CRC partners. A team of senior project staff travelled across Australia to hold meetings with key management organisations in all states and territories with the aim of gaining their engagement (Welsh and Black 2010). In addition, a technical user group (TUG) was convened that included individuals from the eWater CRC partner organisations that were actively involved in software development.
Translation
The translation of information and knowledge across language or cultural differences is a primary function of boundary organisations and can manifest with effective boundary objects developed or applied by those institutions. This can be either through translators, a common spoken language, or in the case of certain endeavours, the nature of the boundary work itself (Robinson et al. 2014). Models can be the medium for effective communication, even when language or other cultural differences exist, but can also present a barrier to some stakeholder’s understanding given uncertainty inherent in such decision support tools (Weichselgartner and Kasperson 2010).
The benefits of developing strategies to facilitate structured knowledge translation between multiple actors and water governance organisations were highlighted as key ingredients to the effective use of WRMs. In the case of the CRB, one respondent noted the importance of WRMs to not be a ‘black box’ and many emphasised the advantages of its transparent structure. The USBR believed that ‘transparency facilitated stakeholders being on relatively equal ground, rather than [certain parties] having an advantage’ and one non-governmental stakeholder recognised that a model could create ‘a common language’ to enable participants from different levels of decision-making to engage in the complex task of policy development. The IBWC emphasised the translational function of the model by stating, ‘with the aid of the modelling information, the stakeholders were able to visualize the effects of drought and expected water allocations to users in both countries’.
Translation across disciplines is an inherent challenge when using expert systems like WRMs. Experience from both basins demonstrate that participants must have sufficient technical expertise to engage in a model-based dialogue. This often requires substantial investments in time and resources for capacity building, resulting in a ‘limited community of skilled modellers’ according to one respondent. Stakeholders unable to build or hire needed capacity can be quickly disadvantaged due to a lack of understanding of the logic and limitations of models, the rationale behind the assumptions imbedded within them and the value they provide to the decision-making process. The inability to communicate these aspects can cause a general scepticism of models, exacerbated in the midst of a critical situation such as a severe drought. Those with the required capacity perceived limitations when ‘Resources were required to explain modelling results to decision makers (and also explain what models could and couldn’t do)’, particularly when complex results are presented with jargon and statistics that users of the information cannot relate to. Communicating hydrologic uncertainty, particularly in face of unprecedented conditions, was a critical factor in the MDB.
Collaboration
The process of building consensus towards a particular objective inherently requires collaboration (Margerum and Robinson 2015).The development of a shared fact basis and the co-production of knowledge to underpin water models (Jasanoff 2004) or the process of overcoming adversity (Susskind and Cruikshank 1987) are examples of powerful forces that can bond parties together as boundary-spanning exercise. Such actions require time and effort to achieve success and can be difficult in water management contexts where there are multiple decision-makers, users and values to consider (Islam and Susskind 2012; Linnerooth-Bayer et al. 2001; Robinson et al. 2014). The benefits of collaborative model development for facilitating a broader understanding of trade-offs among stakeholders has been emphasised (van de Belt 2004); however, such collaboration can be hindered by time pressure for an outcome, the specialization of knowledge required, or fear of political debates subverting the process or results (Gilfedder et al. 2016).
Collaborative decision-making within a federal river context is the main justification for the formation of a river management organisation or interstate compacts. A common WRM platform for analysis provides a medium for this collaboration. Agreements reached among individual states are expected be accepted by the federal government, thus minimizing federal interference or regulations being imposed. The ability to reach such a consensus is facilitated by parties having equal access to the analytical tools and decision-making process and presumably some capacity to influence them. An iterative exchange of possible solutions during negotiations is accelerated considerably if each party has trust in a common tool to develop and analyse new ideas, thus avoiding the risk of multiple models providing conflicting results.
In the case of the CRB, ‘official’ modelling is conducted by the USBR, with regular input and review from the states and other stakeholders in the Stakeholder Modelling Workgroup. When the CRSS was transferred to the RiverWare software platform, the goal was largely to encourage more collaborative participation; however, most respondents indicated that it still requires a high level of expertise to understand, operate and modify. Many emphasised the need for a significant amount of time to understand and become comfortable with the model. One respondent stated, ‘Building trust in the model, model framework and assumptions, takes much longer than the actual time to run the model and produce the results’. Given Mexico’s initial inexperience with the CRSS during Minute 319 negotiations, one Mexican modeller identified a challenge as the ‘Lack of available resources for problem solving [and] model building and operation apart from model developer [sic]’. This was reinforced by responses from the USA acknowledging, ‘The U.S. agencies had greater knowledge than their Mexican counterparts in terms of experience’, yet ‘The U.S. worked with Mexico to insure adequate training and knowledge transfer’. While respondents from both sides of border expressed they ‘worked as a team’ with ‘subgroups of modellers and decision makers [that] we always worked together’, others believed that Mexico was reluctant to ‘buy into CRSS analysis’.
The success of collaboration is not necessarily measured by the outcome, but by the process itself. The development of the surplus guidelines helped build interstate cooperation and model acceptance that was leveraged for the subsequent and more contentions shortage EIS. Similarly, the modelling work during Minute 319 was perceived to form a basis for future collaboration between the USA and Mexico. Process benefits were also realized by the NGO’s ability demonstrate a relatively minor impact on other basin users of water dedicated to environmental objectives (Wheeler et al. 2007).
In the MDB, states contributed their individual sub-basin models to form the integrated river system modelling framework used to develop the Basin Plan 2012 (MDBA 2012). The modelling itself was conducted by the MDBA and CSIRO, and comments were provided by the states. Our survey replies and media reports (Kotsios 2017) indicated that various stakeholders continue to hold a deep frustration over a lack of access to the models. Whether development and application of the MDB WRMs for the Basin Plan could have been more transparent or inclusive is still a subject of debate and speculation.
The technical challenges faced in assembling the disparate models for development the Basin Plan have both demonstrated the need for a unified platform and encouraged members of the eWater CRC to work cooperatively to produce the new analytical tools. The new development of the Source modelling platform benefits from modern approaches to facilitate understanding and cooperation such as object-oriented programming, graphical user interfaces, databases and internet communication. Welsh and Black (2010) describe extensive efforts to involve stakeholders during development including solicitation of user requirements, incorporating feedback, holding monthly project update meetings and conducting regular planning meetings. To satisfy the needs of all eWater CRC partners, frequent debates occurred on the appropriate modelling approaches to incorporate. Managing equity in stakeholder influence was often necessary, and when conflicts could not be resolved, multiple methods were incorporated into the software, often prolonging its development.
Mediation
In the original framing of the concept of boundary work, the function of mediation describes the process of reaching consensus across multiple and often competing interests. In the context of a transboundary river, we consider mediation between upstream and downstream jurisdictions, between national and subnational governments, and between governments and a potential plethora of water users. With a sound design to incorporate and adapt to different types of knowledge, interests and geographical domains, WRMs can facilitate this effort as long as parties establish what knowledge the tool can and cannot support or provide.
With the broad understanding that conflicts over water resources are likely to intensify in both basins, increasing stakeholder engagement in WRM design and application will be useful for mediating those conflicts. One respondent in the CRB noted WRMs provide value to negotiations by stating ‘Without models, there would be more speculation about future conditions, and likely more conflict as modelling has helped form the foundations of many important water management decisions’. An NGO stakeholder believed models add value because ‘decisions are complex and intertwined even at small scales and are difficult to evaluate in any other way at large scales’. This statement supports a USBR belief that, by including various stakeholders in the modelling process, it would ‘improve capacity so as to improve understanding regarding negotiations’. The process by which a model becomes a trusted mediation tool relies on its acceptance by conflicting parties. When describing lessons learned in the negotiations, one respondent in the CRB stated “Agreeing to use a particular model in a negotiation setting requires parties’ ‘buy-in’ of the model”. and ‘Model competition and too many models impedes investment in any one’. Even more fundamental was the belief that ‘sound technical data is the foundation of effective/informed negotiation and decision making’ and a key component is to ‘remove … obstacles by sharing data and working off the same dataset’. This aligns with the notion that acceptance of a models requires the ability to explore and challenge them (Olsson and Andersson 2007).
In the MDB, this process continues, but even so, most respondents recognised that model outputs are very influential and heavily relied on to reach agreements. With respect to their role as a mediation tool between states during the development of the Basin Plan, one respondent stated that advantages were held by ‘Upstream states … because they own and develop models of the tributaries’ which they only share with the MDBA not the other states. Issues regarding model ownership were also problematic between the states and the MDBA since the MBDA was only allowed to use the state models under restrictive licences. The assemblage of models were linked together to form an integrated assessment tool that was sufficient to develop the Basin Plan in response to the immediate need; however, it was not distributed among stakeholders and thus limited its ability to serve the function of a common mediation tool. However, after initial proposals, states provided comments and a number of model developments occurred to help refine the SDLs (MDBA 2012).