Water Resources

, Volume 45, Supplement 1, pp 90–101 | Cite as

Validation of a Hydrological Model Intended for Impact Study: Problem Statement and Solution Example for Selenga River Basin

  • A. N. Gelfan
  • T. D. MillionshchikovaEmail author


The study is aimed to evaluate a hydrological simulation model intended for assessing climate change impact. A new test was suggested and applied to evaluate the performance of a physically based model of Selenga River runoff generation. In this test, to calibrate the model, an enhanced Nash–and-Sutcliffe efficiency (NSE) criterion was used, including trend-oriented reference (benchmark) models instead of the simple reference model used in the original NSE criterion. Next, modifications were made in the Differential Split Sample test (DSS-test) of V. Klemeš (1986), focused on differences in the model performance criteria for climatically contrasting periods, and a new statistical measure was proposed to estimate the significance of these differences. After that, model performance was evaluated for four sites within the catchment, three indicators of interest (daily, monthly, and annual discharge series), and the model ability to reproduce the observed trends in annual and seasonal discharge values was assessed. The model proved robust enough to be applied to assessing climate change impact on the annual and monthly runoff in different parts of the Selenga River basin.


hydrological modeling Selenga River Basin validation model robustness impact assessment crash-test 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andréassian, V., Le Moine N., Perrin C., Ramos M.-H., Oudin L., Mathevet T., Lerat J., and Berthet L., All that glitters is not gold: The case of calibrating hydrological models, Hydrol. Processes, 2012, vol. 26, no. 14, pp. 2206–2210. doi 10.1002/hyp.9264CrossRefGoogle Scholar
  2. 2.
    Beven, K., Towards a coherent philosophy for modelling the environment, Proc. R. Soc. London, 2002, Ser. A, vol. 458, pp. 2465–2484.CrossRefGoogle Scholar
  3. 3.
    Chalov, S.R., Jarsjö, J., Kasimov, N.S., Romanchenko, A., Pietron, J., Thorslund, J., and Belazerova, E., Spatiotemporal variation of suspended transport in the Selenga Basin (Mongolia and Russia), Environ. Earth Sci., 2014, vol. 73, no. 2, pp. 663–680. doi 10.1007/s12665-014-3106-zCrossRefGoogle Scholar
  4. 4.
    Chalov, S.R., Thorslund, J., Kasimov, N., Nittrouer, J., Iliyecheva, E., Pietron, J., Shinkareva, G., Lychagin, M., Aybullatov, D., Kositky, A., Tarasov, M., Akhtman, Y., Garmaev, E., Karthe, D., and Jarsjo, J., The Selenga River Delta—Geochemical barrier for protecting Lake Baikal’s waters, Regional Environ. Change, 2016, vol. 17, no. 7, pp. 2039–2053. doi 10.1007/s10113-016-0996-1CrossRefGoogle Scholar
  5. 5.
    Coron, L., Andréassian, V., Bourqui, M., Perrin, C., and Hendrickx, F., Pathologies of hydrological models used in changing climatic conditions: a review, IAHS Publ., 2011, vol., 344, pp. 39–44.Google Scholar
  6. 6.
    Ecological atlas of the Lake Baikal basin: Irkutsk: Institute of Geography, Sib. Branch, Russ. Acad. Sci., 2014.Google Scholar
  7. 7.
    Euser, T, Winsemius, H.C., Hrachowitz, M., Fenicia, F., Uhlenbrook, S., and Savenije, H.G., A framework to assess the realism of model structures using hydrological signatures, Hydrol. Earth Syst. Sci., 2013, vol. 17, pp. 1893–1912. doi 10.5194/hess-17-1893-2013CrossRefGoogle Scholar
  8. 8.
    Ewen, J. and Parkin, G., Validation of catchment models for predicting land-use and climate change impacts. 1, Method. J. Hydrol., 1996, vol. 175, pp. 583–594.CrossRefGoogle Scholar
  9. 9.
    FAO/IIASA/ISRIC/ISS-CAS/JRC, Harmonized World Soil Database (version 1.2), Rome-Laxenburg: FAO, 2012.Google Scholar
  10. 10.
    Frieler, K., Lange, S., Piontek, F., Reyer, C.P.O., Schewe, J., Warszawski, L., Zhao, F., Chini, L., Denvil, S., Emanuel, K., Geiger, T., Halladay, K., Hurtt, G., Mengel, M., Murakami, D., Ostberg, S., Popp, A., Riva, R., Stevanovic, M., Suzuki, T., Volkholz, J., Burke, E., Ciais, P., Ebi, K., Eddy, T.D., Elliott, J., Galbraith, E., Gosling, S.N, Hattermann, F., Hickler, T., Hinkel, J., Hof, C., Huber, V., Jägermeyr, J., Krysanova, V., Marcé, R., Müller Schmied, M., Mouratiadou, I., Pierson, D., Tittensor, D.P., Vautard, R., van Vliet, M., Biber, M.F., Betts, R.A., Bodirsky, B.L., Deryng, D., Frolking, S., D. Jones, C.D., Lotze, H.K., Lotze-Campen, H., Sahajpal, R., Thonicke, K., Tian, H., and Yamagata, Y., Assessing the impacts of 1.5°C global warming—simulation protocol of the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP2b), Geosci. Model Dev., 2017, vol. 10, no. 12, pp. 4321–4345. doi 10.5194/gmd-10-4321-2017CrossRefGoogle Scholar
  11. 11.
    Frolova, N.L., Belyakova, P.A., Grigor’ev, V.Yu., Sazonov, A.A., and Zotov, L.V., Many-Year Variations of River Runoff in the Selenga Basin, Water Resour., 2017, vol. 44, no. 3, pp. 243–255.CrossRefGoogle Scholar
  12. 12.
    Garrick M., Cunnane C., and Nash J.E., A criterion of efficiency for rainfall-runoff models, J. Hydrol., 1978, vol. 36, no. 3–4, pp. 375–381.CrossRefGoogle Scholar
  13. 13.
    Gelfan, A., Gustafsson, D., Motovilov, Y., Arheimer, B., Kalugin, A., Krylenko, I., and Lavrenov, A., Climate change impact on water regime of two great arctic rivers: modeling and uncertainty issues, Clim. Change, 2017, vol. 141, no. 3, pp. 499–515. doi 10.1007/s10584-016-1710-5CrossRefGoogle Scholar
  14. 14.
    Gelfan, A., Motovilov, Yu., Krylenko, I., Moreido, V., and Zakharova, E., Testing the robustness of the physically- based ECOMAG model with respect to changing conditions, Hydrol. Sci. J., 2015, vol. 60, pp. 1266–1285. doi 10.1080/02626667.2014.935780CrossRefGoogle Scholar
  15. 15.
    Gottschalk, L., Beldring, S., Engeland, K., Tallaksen, L., Sælthun, N.R., Kolberg, S., and Motovilov, Yu., Regional/macroscale hydrological modelling: A Scandinavian experience, Hydrol. Sci. J., 2001, vol. 46, pp. 963–982.CrossRefGoogle Scholar
  16. 16.
    Gupta H.V., Kling H., Yilmaz K.K., and Martinez G.F., Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling, J. Hydrol., 2009, vol. 377, pp. 80–91. doi 10.1016/j.jhydrol.2009.08.003CrossRefGoogle Scholar
  17. 17.
    Huang, S., Kumar, R., Flörke, M., Yang, T., Hundecha, Y., Kraft, P., Gao, C., Gelfan, A., Liersch, S., Lobanova, A., Strauch, M., van Ogtrop, F., Reinhardt, J., Haberlandt, U., and Krysanova, V., Evaluation of an ensemble of regional hydrological models in 12 large-scale river basins worldwide, Clim. Change, 2017, vol. 141, pp. 381–397. doi 10.1007/s10584-016-1841-8CrossRefGoogle Scholar
  18. 18.
    Kalugin, A.S. and Motovilov, Yu.G., Runoff formation model for the Amur river basin, Water Resour., 2018, vol. 45, no. 2, pp. 149–159.CrossRefGoogle Scholar
  19. 19.
    Kirchner, J.W., Getting the right answers for the right reasons: Linking measurements, analyses, and models to advance the science of hydrology, Water Resour. Res., 2006, vol. 42, W03S04. doi 10.1029/2005WR004362CrossRefGoogle Scholar
  20. 20.
    Klemeš, V. Operational testing of hydrological simulation models, Hydrol. Sci. J., 1986, vol. 31, pp. 13–24.CrossRefGoogle Scholar
  21. 21.
    Konikow, L.F. and Bredehoeft, J.D., Groundwater models cannot be validated, Adv. Water Resour., 1992, vol, 15, pp. 47–62.CrossRefGoogle Scholar
  22. 22.
    Krysanova, V., Donnelly, C., Gelfan, A., Gerten, D., Arheimer, B., Hattermann, F., and Kundzewicz, Z.W., How the performance of hydrological models relates to credibility of projections under climate change, Hydrol. Sci. J., 2018, vol. 63, pp. 696–720. doi 10.1080/02626667.2018.1446214CrossRefGoogle Scholar
  23. 23.
    Lehner, B., Verdin, K., and Jarvis, A., New global hydrography derived from spaceborne elevation data, Eos Trans., 2008, vol. 89, no. 10, pp. 93–94.CrossRefGoogle Scholar
  24. 24.
    Logachev, N.A., History and geodynamics of the Baikal rift, Russ. Geol. Geophys., 2003, vol. 44, no. 5, pp. 391–406.Google Scholar
  25. 25.
    Loveland, T.R., Reed, B.C., Brown, J.F., Ohlen, D.O., Zhu, Z., Yang, L., and Merchant, J.W., Development of a global landcover characteristics database and IGBP DISCover from 1 km AVHRR data, Int. J. Remote Sens., 2000, vol. 21, pp. 1303–1330.CrossRefGoogle Scholar
  26. 26.
    Moreido, V.M. and Kalugin, A.S., Assessing possible changes in Selenga R. water regime in the XXI century based on a runoff formation model, Water Resour., 2017, vol. 44, no. 3, pp. 390–398.CrossRefGoogle Scholar
  27. 27.
    Motovilov, Yu.G., Hydrological simulation of river basins at different spatial scales: 1. generalization and averaging algorithms, Water Resour., 2016, vol. 43, no. 3, pp. 429–437.CrossRefGoogle Scholar
  28. 28.
    Motovilov, Yu., Gottschalk, L., Engeland, L., and Rodhe A., Validation of a distributed hydrological model against spatial observation, Agric. Forest Meteor., 1999, vols. 98–99, pp. 257–277.CrossRefGoogle Scholar
  29. 29.
    Murphy A.H., Skill scores based on the mean square error and their relationships to the correlation coefficient, Mon. Weather Rev., 1988, vol. 116, pp. 2417–2424.CrossRefGoogle Scholar
  30. 30.
    Nash, J.E. and Sutcliffe, J.V., River flow forecasting through conceptual models, Part I—A discussion of principles, J. Hydrol., 1970, vol. 10, pp. 282–290.CrossRefGoogle Scholar
  31. 31.
    The National Atlas of Mongolia, The Institute of Geography of Mongolian Academy of Science, 2009.Google Scholar
  32. 32.
    Oreskes, N., The role of quantitative models in science, in Models in ecosystem science, Princeton University Press, 2003, pp. 13–31.Google Scholar
  33. 33.
    Refsgaard, J.C., Madsen, H., Andréassian, V., Arnbjerg-Nielsen, K., Davidson, T.A., Drews, M., Hamilton, D.P., Jeppesen, E., Kjellström, E., Olesen, J.E., Sonnenborg, T.O., Trolle, D., Willems, P., and Christensen, J.H., A framework for testing the ability of models to project climate change and its impacts, Clim. Change, 2013, vol. 122, pp. 271–282.CrossRefGoogle Scholar
  34. 34.
    Schaefli, B, Hingray, B, Niggli, M, and Musy, A., A conceptual glaciohydrological model for high mountainous catchments, Hydrol. Earth Syst. Sci., 2005, vol. 9, pp. 95–109.CrossRefGoogle Scholar
  35. 35.
    Seibert, J., On the need for benchmarks in hydrological modelling, Hydrol. Processes, 2001, vol. 15, pp. 1063–1064. doi 10.1002/hyp.446CrossRefGoogle Scholar
  36. 36.
    Seibert, J., Reliability of model predictions outside calibration conditions, Nord. Hydrol., 2003, vol. 34, pp. 477–492.CrossRefGoogle Scholar
  37. 37.
    Thirel, G., Andréassian V., and Perrin C., On the need to test hydrological models under changing conditions, Hydrol. Sci. J., 2015, vol. 60, nos. 7–8, pp. 1165–1173. doi 10.1080/02626667.2015.1050027CrossRefGoogle Scholar
  38. 38.
    Thirel, G., Andréassian, V., Perrin, C., Audouy, J.-N., Berthet, L., Folton, N., Furusho, C., Kuentz, A., Lerat, J., Lindström, G., Martin, E., Mathevet, T., Merz, R., Parajka, J., Ruelland, D., and Vaze, J., Hydrology under change: an evaluation protocol to investigate how hydrological models deal with changing catchments, Hydrol. Sci. J., 2015, vol. 60, nos. 7–8, pp. 1184–1199. doi 10.1080/02626667.2014.967248CrossRefGoogle Scholar
  39. 39.
    Tornqvist, R., Jarsjo, J., Pietron, J., Bring, A., Rogberg, P., Asokan, S.M., and Destouni, G., Evolution of the hydro-climate system in the Lake Baikal basin, J. Hydrol., 2015, vol. 519, pp. 1953–1962.CrossRefGoogle Scholar
  40. 40.
    World Meteorological Organisation. Intercomparison of Models of Snowmelt Runoff, Operational Hydrology Report No. 23. Secretariat of the World Meteorological Organization, Geneva, Switzerland, 1986.Google Scholar
  41. 41.
    Wöhling, T., Samaniego, L., and Kumar, R., Evaluating multiple performance criteria to calibrate the distributed hydrological model of the upper Neckar catchment, Environ. Earth Sci., 2013, vol. 69, pp. 453–468. doi 10.1007/s12665-013-2306-2CrossRefGoogle Scholar
  42. 42.
    Yilmaz, K.K., Gupta, H.V., and Wagener, T., A process-based diagnostic approach to model evaluation: Application to the NWS distributed hydrologic model, Water Resour. Res., 2008, vol. 44, W09417. doi 10.1029/2007WR006716CrossRefGoogle Scholar

Copyright information

© Pleiades Publishing, Ltd. 2018

Authors and Affiliations

  1. 1.Water Problems InstituteRussian Academy of SciencesMoscowRussia
  2. 2.Department of Land Hydrology, Faculty of GeographyMoscow State UniversityMoscowRussia

Personalised recommendations