Harmonisation of Reliability Performance Indices for Planning and Operational Evaluation of Water Supply Reservoirs

The planning and operational performance evaluation of water supply reservoirs routinely use the volume-based (Rv) and time-based (Rt) reliability indices but decision making is often complicated by trade-off necessitated by the fact that the two are never the same, with Rv ≥ Rt. This study has resolved the problem by harmonising the two indices. Using data from ten global rivers, simulations of hypothetical reservoirs were carried out to determine capacity for specified demands and Rt values. The corresponding Rv values were then evaluated and the resulting reliability biases (i.e. Rv – Rt) were found. To harmonise the two indices, i.e. to nullify the biases, the concept of water shortage threshold was introduced, which is the minimum quantity of water shortage that can be taken as constituting real failure for the purpose of Rt evaluation; shortage quantities below this will be disregarded. The results showed that the water shortage threshold that nullifies the reliability bias can be as high as 60% of the demand, depending on the runoff variability, the demand and the specified Rt. When averaged over all the situations analysed, the water shortage threshold was found to be 51% of the demand. Although this might appear high, it is argued that it is plausible both within the context of developed economies, where unaccounted-for-water can be much higher than 51%, and of underdeveloped economies where large sections of the population have no access to adequate water supply. In the latter case, a reduction of 50% in water supplied that guarantees uninterrupted supply of the other 50% will be deemed satisfactory and reliable, while for the former, a shortage of 50% that forces a change in behaviour to waste less water will also be deemed satisfactory. The significance and novelty of this study stem from the fact that it has removed the need for the trade-off between the two reliability indices, thus enabling unequivocal characterisation of water supply reservoir performance for effective decision making.


Introduction
Reservoirs are a major component of water supply systems in most countries, helping to regulate river flow by storing excess water during high flows for later release toward meeting demand during low flows. Unless the overall demand is very low, typically less than 5% of the average flow (Twort et al. 1974), any water supply system relying exclusively on surface water resources needs a reservoir. In order to ensure that the reservoir meets the demand placed on it, the reservoir must be accurately planned. Such planning requires analysing the available runoff data at the reservoir site using a variety of techniques as discussed by McMahon and Adeloye (2005). The end result is an estimate of the reservoir capacity for meeting the demand.
Most reservoir planning analysis techniques rely on the runoff during the Bcritical periodî n the historic record to determine the reservoir capacity. The critical period can be defined as the period of persistent low flows in the record such that, if the reservoir was full at the beginning of this period, it will become empty at the end of the period after releasing its stored water towards meeting the demand.
The notion of the critical period and the implied failure-free reservoir capacity estimate that it produces would be valid, if the future runoff when the reservoir is operated is no worse in terms of the Bdryness^of its critical period as the historic runoff record used for planning the reservoir. Where this is not the case, the reservoir will fail on occasions to meet the water demand obligation placed on it.
As a way of avoiding situations where the smooth operation of reservoirs is hampered by lack of prior information on the likelihood of the reservoir failing to meet the demand, new reservoir planning analysis techniques that attempt to design for specific failure probability (or reliability) have emerged. Behaviour simulation (McMahon and Adeloye 2005;Giakoumakis 2013), the modified sequent peak algorithm (Lele 1987;Adeloye et al. 2001) and Gould's Transition Probability approach (see McMahon and Adeloye 2005) are examples of planning techniques that allow considerations of reliability.
Of these, behaviour simulation (BS) is the most versatile, principally because of its amenability to different reservoir configurations and its ability to accommodate different operating policies as well as secondary surface processes, e.g. net evaporation losses, relatively easily. When applying the BS, reliability can be expressed in terms of time (time-reliability) or volume (volume-reliability). However, in any given analysis, the two are never the same. Indeed, in general the outcome is that (Adeloye 2012): where R v and R t are the volume-and time-based reliability respectively. If as is usual both indices are derived by simulating the behaviour of the reservoir over an inflow period, the R v is the ratio of the total amount of water actually supplied to the total amount demanded; R t is the ratio between the total number of occasions in which the full demand was met to the number of time periods in the simulation period. The inequality in Eq. 1 arises because whereas every failure (in which the full demand was unmet) is weighted equally in R t , the actual magnitude of the water shortage in every failure period is taken into account in R v . Thus, in R t , a shortage of 0.1% of the demand is weighted equally as a shortage of 99%, when in actual fact for users of water, the former is far more tolerable than the latter. The fact that the R t and R v are unlikely to be equal in a given planning analysis makes their use for decision making during reservoir planning extremely difficult. The pertinent questions are these: & Which of the two (R v or R t ) should be adopted for reservoir planning? & Does a low R t really mean that a reservoir performance is woeful, implying that the associated reservoir planning characteristics e.g. reservoir capacity, are unacceptable or should consideration be also given to the estimated R v before making a decision? & More importantly, is it possible to develop a planning analysis rule that ensures that both the R t and R v are the same thus, removing the current difficulty in using these traditional metrics for reservoir capacity planning?
From the point of view of water users, the R v is a more useful measure of water availability. Although there are other indices of performance for reservoir evaluation (see Adeloye 2012;McMahon et al. 2006), Soundharajan et al. (2016) have recently shown that the two reliability measures R v and R t are the least variable to justify their popularity in water resources studies. Of the two indices, however, the R t is much easier to derive; indeed, while the R v requires the time-consuming behaviour simulation to evaluate, R t has been accommodated in simple, empirical planning measures such as the Gould-Dincer approach (see McMahon et al. 2006). Thus, the R t would appear to be a more convenient and attractive index to use.
The issue with the R t is that some occasions counted as failures should not be counted as such because the amount of water shortage involved is too small. Thus, a way to harmonise the R t and R v is to identify a threshold water shortage below which a reservoir release is not assumed to constitute a failure for the purpose of calculating the R t . As the water shortage threshold becomes larger, the number of successes of reservoir operation will increase making the R t to approach the R v . However, there is very little guidance in the literature on what the threshold water shortage level should be. Fiering (1982) once observed that water shortages below 25% of full demand are acceptable in that most consumers are able to adapt. Relatively more recently, Raje and Mujumdar (2010) seemed to adopt this 25% threshold (see also Mujumdar 2000 ) when evaluating the reliability of the hydropower potential of the Hirakud reservoir in India, by assuming that water shortages less than 25% of the demand do not constitute failures.
While the water shortage threshold of 25% of the demand has been used in the literature, its basis is not made clear. Indeed, according to Fiering (1982) the selection of 25% is arbitrary and subject to change. Additionally, given the effect which runoff characteristics, especially the coefficient of variation, have on the capacity-demand function of reservoirs, it will not be out of place to expect the water shortage threshold to be also affected by similar runoff characteristics. Finally, how the level of demand, or indeed the prevailing volume-based reliability (R v ), affects this water shortage threshold is also unknown. As far as the authors are aware, this is a novel development as, to date, no study has systematically addressed the above problems.
Thus, the aim of this study is to systematically determine the water shortage threshold at which R t = R v during reservoir system planning, and the runoff and reservoir systems characteristics, if any, that influence this threshold. The objectives are to: & Assemble river runoff data that cover the broad range of variabilities of global rivers; & Use the behaviour simulation approach, derive the full reservoir capacity-demand-reliability functions for the rivers and assess the divergence (or bias) between the R t and R v ; & Manipulate the reservoir simulation results to determine the water shortage threshold that equalises R t and R v ; & Explore any relationship between the water shortage threshold and reservoir system (e.g. the demand, R v ) and runoff (e.g. coefficient of variation, CV) characteristics; & Make recommendations on a harmonisation of R t and R v for reservoir planning and operational performance evaluation.
In the following Section, further details about the adopted methodology are given. This will be followed by consideration that went into the selection of the 10 global runoff records used in the study. The results and discussions will then be presented, followed by the main conclusions of the study. The behaviour simulation uses the mass balance approach as shown in Eq. (2) and can be used to calculate the required capacity of the reservoir to meet the demand for any failure criterion (McMahon and Adeloye 2005): where S t and S t+1 are, respectively, reservoir storage at the beginning and end of time period t; Q t is the inflow to the reservoir during t; D 0 t is the actual water release during t; E t is the net evaporation (i.e. evaporation-rainfall) during t; L t are other losses (e.g. seepage) from the reservoir storage during t; and K a is the capacity of the reservoir. The relationship between the release D 0 t and the real demand D t in period t depends on the amount of water available during the period (W t ) and the operating policy for allocating this water. Assuming the inflow into the reservoir is known at the start of t, the available water for allocation during t becomes: In the absence of a be-spoke operating policy for the reservoir, which is usually the case during the planning of new reservoirs, the standard operating policy, SOP (Hashimoto et al. 1982) is normally used. SOP stipulates fully supplying the demand if there is sufficient water; otherwise, all the available water should be supplied to leave the reservoir empty. The three supply possibilities for the SOP are therefore as follows: Case A (insufficient water to meet demand, i.e. W t < D t ) Case B (sufficient water to meet demand but reservoir is not full/spilling, i.e. D t < W t < D t + K a ) Case C (more than sufficient water, i.e. reservoir is full and spilling W t ≥D t + K a ) Case A above is failure in that the reservoir is unable to meet the demand. At the end of the simulation, such incidences of failures are identified and used to evaluate the reliability (time-and volume-based) of the reservoir of size K a to meet the demand D t : where N is the total number of time periods and all other symbols are as defined previously. As seen above in reservoir planning using BS, the unknown reservoir capacity K a features prominently in decisions on the water allocation. Thus, determining reservoir capacity for meeting specific demand at a given reliability level using BS involves a trial-and-error process in which different capacities are assumed to start the simulation until the capacity resulting in the desired reliability is obtained. This iterative process, illustrated in Fig. 1, has been suggested by Adeloye et al. (2001) as a possible cause of the misbehaviour of BS as a planning tool first identified by Pretto et al. (1997).
Because BS is a trial-and-error process and can thus misbehave, it will be necessary to ensure that its outcome is correct before adopting the technique for the entire study. This will be done in this study by comparing the BS solution with that obtained using an exact approach, the sequent peak algorithm, SPA described in the next Section.

Sequent Peak Algorithm (SPA)
The SPA (Loucks et al. 1981) gives exact solution to the problem of determining the failurefree (i.e. 100% reliability) reservoir capacity using: where K t and K t+1 are respectively the sequential deficits at the start and end of time period t. As a critical period technique (McMahon and Mein 1986), the SPA assumes that the reservoir is full at start and end of the cycle, i.e. K o = K N = 0. If K N ≠ 0, the SPA cycle is repeated by setting K 0 = K N . Adeloye et al. (2001) have extended the SPA to accommodate failures; however, its use in this study will be limited to the failure-free situation. Thus, the SPA will be implemented here to test the accuracy of failure-free capacity estimates obtained with the BS. If the BS is working correctly, its capacity-demand function must be the same as a similar function obtained with the SPA.

Given:
• Inflow record Q t , t=1,….N • Demand • Shortage criterion, e.g. R t =80% (N= length of simulation period) Assume initial reservoir capacity, K a Simulation for the considered reservoir capacity K a , resulting in a number of shortage time periods, N s (and associated volume of water shortage) Shortage criteria met? (e.g. R t = 80%) Assumed K a is correct Repeat with a new K a assumption: • If R t < required shortage criterion, then increase the trial K a . • if R t > required shortage criterion, then decrease the trial K a .

Yes
Obtain corresponding volume reliability, R v No Fig. 1 Flow chart of reservoir behaviour simulation A.J. Adeloye et al.

Harmonisation of R v and R t : Identifying the Water Shortage Threshold
Identification of the water shortage threshold for the purpose of harmonising R v and R t is also an iterative process. First, the BS is carried out to obtain the K a for the desired R t . The various water shortages in the simulation are then expressed as ratio of the demand. The corresponding R v (see Eq. 6) is then obtained and checked to see if it differs from the R t . If a difference exists, a water shortage ratio threshold (T s ) is assumed and any shortage ratio in the simulation result that is at least equal to T s is considered as a true failure; otherwise it is not a failure. With the new failure criterion implemented to identify the adjusted failures in the simulation, an adjusted time-based reliability, R adj t , is evaluated using Eqs. (9-11).
where f t adj is adjusted failure measure during time period t based on the shortage ratio threshold (T s ) and Δ t is the water shortage ratio during period t.
The resulting R adj t is then compared with the R v and if both are equal, the assumed shortage ratio threshold T s is the required one. If, however, there is still a discrepancy between the R adj t and R v , a new water shortage threshold will be assumed and the process will be repeated until R adj t and R v are equal. As a guide, reducing R adj t will require reducing the threshold T s and increasing R adj t will require the threshold to also be increased.

Data
Monthly and annual time series of flow data for 10 global rivers were used in the study. Table 1 lists the rivers and their summary statistics. From Table 1, it can be noted that the chosen rivers cover the observed variability in global river systems as analysed by McMahon et al. (1992), with the CVof their annual runoff varying from 0.20 to 1.07. In terms of size, the rivers vary in catchment areas between 101 and 12,561 km 2 , with the mean annual runoff (MAR) varying between 2.28 and 8485 Mm 3 . Record lengths vary between 15 and 69 years. Although the variability of reservoir capacity-demand estimates will decrease with increasing record length (Adeloye 1990(Adeloye , 1996Kuria and Vogel 2015), consideration of such an issue is beyond the purview of this study and has therefore been ignored. Nonetheless, the fact that some of the records were relatively short meant that the planning analyses had to be based on the monthly time scale so as to avoid sudden jumps in the estimated R adj t as the water shortage threshold is changed.
Initial exploratory analyses carried out showed that for the short-record length rivers, the number of time steps when using annual time scale was far too few to obtain a smooth trajectory of the R adj t as the water shortage threshold was changed. Using the monthly time scale eliminated this problem, in addition to ensuring that both the within-year and over-year storage capacities are catered for in the planning analyses (Adeloye and Montaseri 1999). For the low variability rivers, within-year storage requirements will be expected to dominate especially at low-medium levels of development. Consequently, using annual time scale to plan such rivers will result in significant under design.

Results and Discussion
3.1 Testing the Performance of Behaviour Simulation for Failure-Free Capacity Planning Figure 2(a-j) charts the BS and corresponding SPA derived failure-free (i.e. 100% reliability) reservoir capacity-demand functions for all the ten rivers studied. As noted earlier, this information was to serve as a confirmation of the accuracy of the trial-and-error BS approach for reservoir capacity planning and as seen in the Figures, the BS storage-demand function (continuous line) is indistinguishable from its SPA-derived counterpart (dotted line) for all the rivers. Although this has only been implemented for the failure-free situation, it is a sufficient proof that the implementation of the BS was correct and can hence be used for subsequent aspects of the study.

Reservoir Capacity-Demand-Reliability Functions
The BS-derived capacity-demand-reliability curves are shown in Fig. 3(a-j) for all the 10 rivers studied. The reliability in Fig. 3 is the R t and cover the range of 50-100% typical of most water supply reservoirs.
The impact of runoff variability (i.e. CV) on the capacity estimate is quite evident in the derived functions, with the capacity ratio for a given demand ratio increasing significantly as the runoff becomes more variable. The Dincer-normal model for reservoir capacity (see McMahon and Adeloye 2005) showed that for normally distributed annual runoff, the capacity ratio is directly proportional to the square of the annual runoff CV. A similar behaviour was also demonstrated by the Gould-gamma model (McMahon and Adeloye 2005) for annual flows that exhibit the gamma distribution. Although establishing the probability distribution hypothesis of the runoff records used in the current study is outside the scope of the study, a cursory examination of the capacity-demand functions in Fig. 3 will confirm this approximate quadratic relationship between reservoir capacity ratio and the CV.

Fig. 2 Comparison of behaviour simulation (for 100% reliability) and SPA results
A further feature of the capacity-demand function and the possible influence of the runoff variability is the fact for low CV (typically CV ≤ 0.4) Rivers, the storage requirement is zero at low demand ratios whereas when the CV is high, there is always the need for storage irrespective of the demand to be met. The implication of this is that for low-variability Rivers, low to moderate demands can usually be met by directly abstracting from the river without the need for any impoundment whereas the same is not true for high variability streams. Where a river is highly variable, the total storage will be dominated by over-year   Fig. 3 Reservoir capacity-demand-reliability functions storage requirements at all levels of demand and the influence of within-year requirements will be small. Consequently, the storage requirements for high variability streams will be non-zero irrespective of the demand ratio. On the contrary for low variability streams in which over-year storage requirements are almost non-existent, low demand ratios do not require storage for meeting them. A final feature that can be observed in Fig. 3 is the decreasing reservoir capacity for a given demand ratio as the reliability decreases.

Reliability Bias, R v -R t
Although the reliability in Fig. 3 was based on the time (i.e. R t ), the corresponding volumebased reliability (R v ) for each of the situations was also evaluated. As expected, the R t and R v were different, with R v ≥ R t . Figure 4 shows the absolute difference (R v -R t ) or bias in the reliabilities and reveals some interesting features. First is that none of the differences is zero, further confirming the R v ≥ R t norm. A second feature is that the bias while high at R t = 50% decreases in an exponential-like manner as the R t increases. Indeed, for the highest R t (= 98%) considered in the study, the bias although still non-zero is very small for all the rivers.
The influence of the CV on the reliability bias was opposite to that observed for the estimated reservoir capacity. As seen in Fig. 4, the reliability bias was highest for the lowto-medium variability rivers where for example in the case of River Dee, the bias was as high as 35. As the annual CV increases, the reliability bias decreased and was generally less than 10 for all the six rivers exhibiting annual CV ≥ 0.6. This behaviour can be explained by further examination of the capacity-demand-reliability functions of Fig. 3. For example as noted earlier, the low variability streams are mostly dominated by within-year capacity requirements whereas the high variability streams are dominated by over-year capacity requirements. For the low variability streams at low levels of demand and R t , there was no need for storage, which would have led to regular failures and the low R t . However, since the demand is monthly without the need to meet carryover (or over-year) demands, the quantity of shortage during those failure periods will be low, translating into a high R v which when combined with the low R t has produced the high reliability bias recorded for the low variability rivers. Of course as the R t increases, the tendency is to require storage in order to meet the demand as can be seen in Fig. 3. While this storage is expected to further boost the R v , such enhancement in the already high R v will be very small, given the small size of the required storage capacity, which when combined with the high R t will result in the observed lowering of the reliability bias at high R t for the low variability rivers.
For the high variability rivers, on the other hand, the need to also meet the carryover demands means that water shortages will be much higher than for the low variability rivers, implying lower R v . The net effect of this is the much depressed reliability bias for the high variability streams, when compared with the low variability streams.
All this will mean that the need for the harmonisation of the two reliability measures R v and R t is more pertinent for reservoirs on low variability streams than those on high variability streams. Figure 5 has been produced which shows the reliability bias averaged over the different demands and further reinforces the earlier observation that low variability rivers produce the most bias especially at low R t . A comforting feature in Fig. 5 is that for the commonly adopted R t range of 95-98% in reservoir planning, the reliability bias is very low whatever the variability regime of the river runoff.

Harmonisation and Water Shortage Ratio Threshold
The water shortage thresholds have been plotted in Fig. 6, as a way of illustrating how these thresholds vary with both the R t and demand ratio. Superimposed on the raw data are also the average thresholds for each of the investigated R t values.
As Fig. 6 clearly demonstrates and contrary to the limited guide in the literature on the level of water shortage threshold, the threshold that harmonises the two reliability indices is not  Fig. 4 Reliability bias, R v -R t constant but varies depending on the R t , the demand ratio and the variability of the river runoff. The threshold water shortage ratio is in general much higher than the 25% suggested by Fiering (1982) especially for low R t although as the R t becomes higher and approaches the 95-98% range for which most reservoirs are often planned, the threshold becomes closer to 25% (or even lower especially for the rivers exhibiting CV > 1.0). Figure 7 shows the average threshold curves for the different rivers as extracted from Fig. 6, with the 10-river average curve superimposed, which further confirms that the water shortage threshold is in general much higher than 25%. The 10-river average threshold starts at 55% (for R t = 50%) and progressively declines to 40.5% (at R t = 98%), which when averaged over the seven R t values tested gives a global average water shortage threshold of 51%.
Although the above water shortage thresholds harmonising R v and R t may appear high, they are still within the level of unaccounted water in most places. An unstated assumption in the Fiering (1982) guide is that all water released from the reservoir reaches the consumers, in which case a reduction of 25% in this amount will not be expected to hence does not constitute a failure. However, when it is realised that up to 55% of the released water can be lost and hence unaccounted for in developed economies (Twort et al. 1974), much of this loss occurring at the point of delivery (e.g. the consumer tap, irrigation fields, etc.), the thresholds arrived at in this study are still plausible to constitute failure-free situation, especially if they force behavioural changes that result in reductions of water wastage and losses. The situation in less developed economies where over 32% of the population lack an adequate (in terms of quantity and quality) water supply (Cairncross and Feachem 1993) is even more pertinent. In such situations, a reduction in supply that guarantees uninterrupted supply of 50% of the demand should be deemed satisfactory or reliable and hence constitute failure-free operation.

Using the Harmonised System for Water Supply Reservoir Planning
The significance of the harmonisation is that situations where the R v and R t are different, thereby complicating the decision making process will no longer happen because both reliability indices will now be equal. To illustrate this, a simple example involving reservoir BS with annual runoff will be presented as follows.
Scenario Consider a hypothetical reservoir that receives the following runoff inflow (× 10 6 m 3 ) over a 3-year period: Year 1: 6; Year 2: 3; and Year 3: 3. The reservoir capacity was 0.05MAR and the annual demand was 0.9MAR, where MAR is the mean annual runoff. It is required to evaluate the performance of the reservoir in terms of both the R v and R t and make recommendations as to the performance of this reservoir. The SOP is to be assumed for reservoir operation and secondary processes such as net evaporation will be ignored. The reservoir is assumed to be full at the start of the simulation period.
Solution From the historical runoff data, MAR = 4; hence K a = 0.05MAR = 0.2 and D = 0.9 MAR = 3.6. Using Excel spreadsheet (details of the trial-and-error BS implementation can be obtained from the 1st author on request), the BS outcome is summarised in Table 2. Using the information in Table 2:  Consequently, in this simple example, the reliability bias is 58%. Indeed while the R t (= 33%) would imply that this hypothetical reservoir is of no value, the R v is indicating a system with a respectable 91% pass mark. Without evaluating the R v , which is usually the case for most analysts fixated with the R t , the system will be condemned completely. The harmonised system developed in this study will remove this complexity. For example also shown in brackets in the water shortage column of Table 2 are the water shortages expressed as ratio of the demand. Since none of these is above the 51% water shortage threshold obtained in this study, none of them should qualify as failure for the purpose of evaluating the R t . This will give R adj t = 100, which is much closer to the R v . It should be noted that the outcome of this hypothetical case has been influenced by the length of data record used for the BS-it will be foolhardy for any analyst to evaluate a reservoir using a 3-year data record. With a longer data record, one would expect that this should converge on the R v using the harmonised system of reliability developed in this study.

Conclusions
This work has studied the difference between the time-based (R t ) and volume-based (R v ) reliabilities in assessing the reservoir performance, and introduced a new concept of the water shortage threshold to nullify this bias. The reliability bias (R v -R t ) is very high (up to 35%) at low reliability levels but low at high R t . Similarly, the reliability bias is strongly influenced by the runoff variability, with high variability rivers exhibiting low bias and vice versa.
The water shortage threshold at which the two reliabilities are equal (or very close) ranges between 14.4 to 60.2% of the demand. When averaged over all the scenarios -demand, runoff variability and R t values-investigated, the water shortage threshold was 51%. A simple numerical example presented to illustrate the new methodology did confirm that a threshold water shortage of 51% resulted in the R t converging on the R v