Skip to main content

A framework for testing dynamic classification of vulnerable scenarios in ensemble water supply projections


Recent water resources planning studies have proposed climate adaptation strategies in which infrastructure and policy actions are triggered by observed thresholds or “signposts.” However, the success of such strategies depends on whether thresholds can be accurately linked to future vulnerabilities. This study presents a framework for testing the ability of adaptation thresholds to dynamically identify vulnerable scenarios within ensemble projections. Streamflow projections for 91 river sites predominantly in the western USA are used as a case study in which vulnerability is determined by the ensemble members with the lowest 10% of end-of-century mean annual flow. Illustrative planning thresholds are defined through time for each site based on the mean streamflow below which a specified fraction of scenarios is vulnerable. We perform a leave-one-out cross-validation to compute the frequency of incorrectly identifying or failing to identify a vulnerable scenario (false positives and false negatives, respectively). Results show that in general, this method of defining thresholds can identify vulnerable scenarios with low false positive rates (< 10%), but with false negative rates for many rivers remaining higher than random chance until roughly 2060. This finding highlights the tradeoff between frequently triggering unnecessary action and failing to identify potential vulnerabilities until later in the century, and suggests room for improvement in the threshold-setting technique that could be benchmarked with this approach. This testing framework could extend to thresholds defined with multivariate statistics, or to any application using thresholds and ensemble projections, such as long-term flood and drought risk, or sea level rise.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8


Download references


Any opinions, findings, and conclusions are those of the authors and do not necessarily reflect the views or policies of the NSF. We further acknowledge the World Climate Research Program’s Working Group on Coupled Modeling and the climate modeling groups listed in the Supplement of this paper for producing and making available their model output.


This work was partially supported by the U.S. National Science Foundation grants CNS-1639268 and CNH-1716130.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Bethany Robinson.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material


(PDF 821 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Robinson, B., Herman, J.D. A framework for testing dynamic classification of vulnerable scenarios in ensemble water supply projections. Climatic Change 152, 431–448 (2019).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: