Theoretical Ecology

, Volume 10, Issue 1, pp 1–20 | Cite as

Optimization methods to solve adaptive management problems

  • Iadine Chadès
  • Sam Nicol
  • Tracy M. Rout
  • Martin Péron
  • Yann Dujardin
  • Jean-Baptiste Pichancourt
  • Alan Hastings
  • Cindy E. Hauser
REVIEW PAPER

Abstract

Determining the best management actions is challenging when critical information is missing. However, urgency and limited resources require that decisions must be made despite this uncertainty. The best practice method for managing uncertain systems is adaptive management, or learning by doing. Adaptive management problems can be solved optimally using decision-theoretic methods; the challenge for these methods is to represent current and future knowledge using easy-to-optimize representations. Significant methodological advances have been made since the seminal adaptive management work was published in the 1980s, but despite recent advances, guidance for implementing these approaches has been piecemeal and study-specific. There is a need to collate and summarize new work. Here, we classify methods and update the literature with the latest optimal or near-optimal approaches for solving adaptive management problems. We review three mathematical concepts required to solve adaptive management problems: Markov decision processes, sufficient statistics, and Bayes’ theorem. We provide a decision tree to determine whether adaptive management is appropriate and then group adaptive management approaches based on whether they learn only from the past (passive) or anticipate future learning (active). We discuss the assumptions made when using existing models and provide solution algorithms for each approach. Finally, we propose new areas of development that could inspire future research. For a long time, limited by the efficiency of the solution methods, recent techniques to efficiently solve partially observable decision problems now allow us to solve more realistic adaptive management problems such as imperfect detection and non-stationarity in systems.

Keywords

Adaptive management Markov decision process MDP Partially observable Markov decision process POMDP Stochastic dynamic programming Value of information Hidden Markov models Natural resource management Conservation 

Supplementary material

12080_2016_313_MOESM1_ESM.docx (27 kb)
Fig. S1(DOCX 27 kb)
12080_2016_313_MOESM2_ESM.docx (24 kb)
Table S1(DOCX 23 kb)
12080_2016_313_MOESM3_ESM.docx (35 kb)
Table S2(DOCX 34 kb)

References

  1. Amato C, Oliehoek FA Scalable Planning and Learning for Multiagent POMDPs. In: Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015Google Scholar
  2. Åström KJ (1965) Optimal control of Markov decision processes with incomplete state estimation. J Math Anal Appl 10:174–205CrossRefGoogle Scholar
  3. Åström K, Wittenmark B (2008) Adaptive control, 2nd edn. Dover Publications, MineolaGoogle Scholar
  4. Bellman RE (1957) Dynamic Programming. Princeton University Press, PrincetonGoogle Scholar
  5. Bernstein DS, Givan R, Immerman N, Zilberstein S (2002) The complexity of decentralized control of Markov decision processes. Math Oper Res 27:819–840CrossRefGoogle Scholar
  6. Bertsekas DP (1995) Dynamic programming and optimal control vol 1, vol 2. Athena Scientific Belmont, MAGoogle Scholar
  7. Bonet B (2002) An epsilon-optimal grid-based algorithm for partially observable Markov decision processes. In: Proceedings of the 19th International Conference on Machine Learning (ICML-02), Sydney, Australia. Morgan Kaufman Publishers Inc., pp 51–58Google Scholar
  8. Boutilier C, Dearden R (1994) Using abstractions for decision-theoretic planning with time constraints. In: Proceedings of the Twelfth AAAI National Conference on Artificial Intelligence. AAAI Press, pp 1016–1022Google Scholar
  9. Boutilier C (1999) Sequential optimality and coordination in multiagent systems. In: IJCAI. pp 478–485Google Scholar
  10. Brafman R (1997) A heuristic variable grid solution method for POMDPs. In: Proceedings of the National Conference on Artificial Intelligence (AAAI-97), Providence, Rhode Island. pp 727–733Google Scholar
  11. Canessa S et al (2015) When do we need more data? A primer on calculating the value of information for applied ecologists. Methods Ecol Evol 6:1219–1228. doi:10.1111/2041-210x.12423 CrossRefGoogle Scholar
  12. Canessa S et al (2016) Adaptive management for improving species conservation across the captive-wild spectrum. Biol Conserv 199:123–131. doi:10.1016/j.biocon.2016.04.026 CrossRefGoogle Scholar
  13. Cassandra AR, Kaelbling LP (1995) Learning policies for partially observable environments: Scaling up. In: Machine Learning Proceedings 1995: Proceedings of the Twelfth International Conference on Machine Learning, Tahoe City, California. Morgan Kaufmann, p 362Google Scholar
  14. Chades I, Bouteiller B Solving multiagent Markov decision processes: a forest management example. In: Proceedings of the International Congress on Modelling and Simulation (MODSIM 2005), 2005. pp 1594–1600Google Scholar
  15. Chades I, Scherrer B, Charpillet F (2002) A heuristic approach for solving decentralized-pomdp: Assessment on the pursuit problem. In: Proceedings of the 2002 ACM symposium on Applied computing. ACM, pp 57–62Google Scholar
  16. Chadès I, McDonald-Madden E, McCarthy MA, Wintle B, Linkie M, Possingham HP (2008) When to stop managing or surveying cryptic threatened species. Proc Natl Acad Sci U S A 105:13936CrossRefPubMedPubMedCentralGoogle Scholar
  17. Chadès I, Martin TG, Nicol S, Burgman MA, Possingham HP, Buckley YM (2011) General rules for managing and surveying networks of pests, diseases, and endangered species. Proc Natl Acad Sci 108:8323–8328. doi:10.1073/pnas.1016846108 CrossRefPubMedPubMedCentralGoogle Scholar
  18. Chadès I, Carwardine J, Martin TG, Nicol S, Sabbadin R, Buffet O (2012) MOMDPs: a solution for modelling adaptive management problems. In: The Twenty-Sixth AAAI Conference on Artificial Intelligence (AAAI-12), Toronto, Canada. pp 267–273Google Scholar
  19. Chadès I, Chapron G, Cros M-J, Garcia F, Sabbadin R (2014) MDPtoolbox: a multi-platform toolbox to solve stochastic dynamic programming problems. Ecography 37:916–920CrossRefGoogle Scholar
  20. Charles AT (1992) Uncertainty and information in fishery management models: a Bayesian updating algorithm. Am J Math Manag Sci 12:191–225Google Scholar
  21. Dibangoye JS, Amato C, Buffet O, Charpillet F (2016) Optimally solving Dec-POMDPs as continuous-state MDPs. J Artif Intell Res 55:443–497Google Scholar
  22. Dujardin Y, Dietterich T, Chadès I (2015) alpha-min: a compact POMDP solver. In: International Joint Conference on Artificial Intelligence (IJCAI-2015), Buenos Aires, ArgentinaGoogle Scholar
  23. Ehrgott M (2005) Multicriteria optimization, 2nd edn. Springer, BerlinGoogle Scholar
  24. Fackler P (2013) MDPSOLVE Software for Dynamic OptimizationGoogle Scholar
  25. Fackler PL, Haight RG (2014) Monitoring as a partially observable decision problem. Resour Energy Econ 37:226–241CrossRefGoogle Scholar
  26. Fackler P, Pacifici K (2014) Addressing structural and observational uncertainty in resource management. J Environ Manag 133:27–36. doi:10.1016/j.jenvman.2013.11.004 CrossRefGoogle Scholar
  27. Filatov NM, Unbehauen H (2000) Survey of adaptive dual control methods. IEE Proc - Control Theory Appl 147:118–128. doi:10.1049/ip-cta:20000107 CrossRefGoogle Scholar
  28. Firn J, Rout T, Possingham H, Buckley YM (2008) Managing beyond the invader: manipulating disturbance of natives simplifies control efforts. J Appl Ecol 45:1143–1151. doi:10.1111/j.1365-2664.2008.01510.x Google Scholar
  29. Fisher RA (1922) On the Mathematical Foundations of Theoretical Statistics. Philos Trans R Soc Lond A: Math, Phys Eng Sci 222:309–368. doi:10.1098/rsta.1922.0009 CrossRefGoogle Scholar
  30. Frederick SW, Peterman RM (1995) Choosing fisheries harvest policies: when does uncertainty matter? Can J Fish Aquat Sci 52:291–306. doi:10.1139/f95-030 CrossRefGoogle Scholar
  31. Fulton EA, Smith ADM, Smith DC, van Putten IE (2011) Human behaviour: the key source of uncertainty in fisheries management. Fish Fish 12:2–17. doi:10.1111/j.1467-2979.2010.00371.x CrossRefGoogle Scholar
  32. Givan R, Leach S, Dean T (2000) Bounded-parameter Markov decision processes. Artif Intell 1:71–109CrossRefGoogle Scholar
  33. Gregory R, Ohlson D, Arvai J (2006) Deconstructing adaptive management: citeria for applications to environmental management. Ecol Appl 16:2411–2425CrossRefPubMedGoogle Scholar
  34. Grewal MS (2011) Kalman filtering. SpringerGoogle Scholar
  35. Haight RG, Polasky S (2010) Optimal control of an invasive species with imperfect information about the level of infestation. Resour Energy Econ 32:519–533CrossRefGoogle Scholar
  36. Hauser CE, Possingham HP (2008) Experimental or precautionary? Adaptive management over a range of time horizons. J Appl Ecol 45:72–81. doi:10.1111/j.1365-2664.2007.01395.x CrossRefGoogle Scholar
  37. Holling CS (1978) Adaptive environmental assessment and management. John Wiley & Sons, LondonGoogle Scholar
  38. Houston A, Clark C, McNamara J, Mangel M (1988) Dynamic models in behavioural and evolutionary ecology. Nature 332:29–34CrossRefGoogle Scholar
  39. Johnson FA, Clinton TM, Kendall WL, Dubovsky JA, Caithamer DF, Kelley JR Jr, Byron KW (1997) Uncertainty and the Management of Mallard Harvests. J Wildl Manag 61:202–216. doi:10.2307/3802429 CrossRefGoogle Scholar
  40. Johnson FA, Kendall WL, Dubovsky JA (2002) Conditions and limitations on learning in the adaptive management of mallard harvests. Wildl Soc Bull 176–185Google Scholar
  41. Kareiva P, Groves C, Marvier M (2014) REVIEW: The evolving linkage between conservation science and practice at The Nature Conservancy. J Appl Ecol 51:1137–1147. doi:10.1111/1365-2664.12259 CrossRefPubMedPubMedCentralGoogle Scholar
  42. Keith DA, Martin TG, McDonald-Madden E, Walters C (2011) Uncertainty and adaptive management for biodiversity conservation. Biol Conserv 144:1175–1178CrossRefGoogle Scholar
  43. Kurniawati H, Hsu D, Lee W-S (2008) SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces. In: Proceedings of Robotics: Science and Systems IV, Zurich, Switzerland. pp 65–72Google Scholar
  44. Littman ML (1994) Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of the eleventh international conference on machine learning. pp 157–163Google Scholar
  45. Lovejoy W (1991) Computationally feasible bounds for partially observed Markov decisions processes. Oper Res 39:162–175CrossRefGoogle Scholar
  46. Lubow BC (1997) Adaptive Stochastic Dynamic Programming (ASDP): Supplement to SFP User’s Guide, 20th edn. Colorado Cooperative Fish and Wildlife Research Unit, Colorado State University, Fort collinsGoogle Scholar
  47. Ludwig D, Walters CJ (1981) Measurement Errors and Uncertainty in Parameter Estimates for Stock and Recruitment. Can J Fish Aquat Sci 38:711–720. doi:10.1139/f81-094 CrossRefGoogle Scholar
  48. MacKenzie DI (2009) Getting the biggest bang for our conservation buck. Trends Ecol Evol (Personal Ed) 24:175–177CrossRefGoogle Scholar
  49. Madani O, Hanks S, Condon A (2003) On the undecidability of probabilistic planning and related stochastic optimization problems. Artif Intell 147:5–34CrossRefGoogle Scholar
  50. Mangel M, Clark CW (1983) Uncertainty, search, and information in fisheries. J Conseil 41:93–103. doi:10.1093/icesjms/41.1.93 CrossRefGoogle Scholar
  51. Marescot L, Chapron G, Chadès I, Fackler P, Duchamp C, Marboutin E, Gimenez O (2013) Complex decisions made simple: a primer on stochastic dynamic programming. Methods Ecol Evol 4:872–884CrossRefGoogle Scholar
  52. Martin J, Runge MC, Nichols JD, Lubow BC, Kendall WL (2009) Structured decision making as a conceptual framework to identify thresholds for conservation and management. Ecol Appl 19:1079–1090CrossRefPubMedGoogle Scholar
  53. Martin J et al (2011) Structured decision making as a proactive approach to dealing with sea level rise in Florida. Clim Chang 107:185–202CrossRefGoogle Scholar
  54. Martin TG, Camaclang AE, Possingham HP, Maguire LA, Chadès I (2016) Timing of Protection of Critical Habitat Matters. Conserv Lett:n/a-n/a. doi:10.1111/conl.12266 Google Scholar
  55. McCarthy MA (2007) Bayesian methods for ecology. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  56. McCarthy MA, Possingham HP (2007) Active adaptive management for conservation. Conserv Biol 21:956–963CrossRefPubMedGoogle Scholar
  57. McCarthy MA, Possingham HP, Gill AM (2001) Using stochastic dynamic programming to determine optimal fire management for Banksia ornata. J Appl Ecol 38:585–592CrossRefGoogle Scholar
  58. McCarthy MA, Armstrong DP, Runge MC (2012) Adaptive Management of Reintroduction. In: Reintroduction Biology. John Wiley & Sons, Ltd, pp 256–289. doi:10.1002/9781444355833.ch8
  59. McDonald-Madden E et al (2010a) Active adaptive conservation of threatened species in the face of uncertainty. Ecol Appl 20:1476–1489. doi:10.1890/09-0647.1 CrossRefPubMedGoogle Scholar
  60. McDonald-Madden E, Baxter PWJ, Fuller RA, Martin TG, Game ET, Montambault J, Possingham HP (2010b) Monitoring does not always count. Trends Ecol Evol 25:547–550. doi:10.1016/j.tree.2010.07.002 CrossRefPubMedGoogle Scholar
  61. McDonald-Madden E, Chadès I, McCarthy MA, Linkie M, Possingham HP (2011) Allocating conservation resources between areas where persistence of a species is uncertain. Ecol Appl 21:844–858. doi:10.1890/09-2075.1 CrossRefPubMedGoogle Scholar
  62. Mehta SV, Haight RG, Homans FR, Polasky S, Venette RC (2007) Optimal detection and control strategies for invasive species management. Ecol Econ 61:237–245. doi:10.1016/j.ecolecon.2006.10.024 CrossRefGoogle Scholar
  63. Monahan GE (1982) Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms. MGMT SCI 28:1–16CrossRefGoogle Scholar
  64. Moore CT, Conroy MJ (2006) Optimal regeneration planning for old-growth forest: addressing scientific uncertainty in endangered species recovery through adaptive management. For Sci 52:155–172Google Scholar
  65. Moore AL, McCarthy MA (2010) On Valuing Information in Adaptive-Management Models. Conserv Biol 24:984–993. doi:10.1111/j.1523-1739.2009.01443.x CrossRefPubMedGoogle Scholar
  66. Moore AL, Hauser CE, McCarthy MA (2008) How we value the future affects our desire to learn. Ecol Appl 18:1061–1069. doi:10.1890/07-0805.1 CrossRefPubMedGoogle Scholar
  67. Moore CT et al (2011) An Adaptive Decision Framework for the Conservation of a Threatened Plant. J Fish Wildl Manag 2:247–261. doi:10.3996/012011-jfwm-007 CrossRefGoogle Scholar
  68. Nichols JD, Johnson FA, Byron KW (1995) Managing North American Waterfowl in the Face of Uncertainty. Annu Rev Ecol Syst 26:177–199. doi:10.2307/2097204 CrossRefGoogle Scholar
  69. Nichols JD et al (2011) Climate change, uncertainty, and natural resource management. J Wildl Manag 75:6–18CrossRefGoogle Scholar
  70. Nicol S, Chadès I (2012) Which States Matter? An Application of an Intelligent Discretization Method to Solve a Continuous POMDP in Conservation Biology. PLoS ONE 7:e28993. doi:10.1371/journal.pone.0028993 CrossRefPubMedPubMedCentralGoogle Scholar
  71. Nicol SC, Possingham HP (2010) Should metapopulation restoration strategies increase patch area or number of patches? Ecol Appl 20:566–581CrossRefPubMedGoogle Scholar
  72. Nicol S, Buffet O, Iwamura T, Chadès I (2013) Adaptive Management of Migratory Birds Under Sea Level Rise. In: Proceedings of the 23rd International Joint Conference on Artificial Intelligence, Beijing. pp 2955–2957Google Scholar
  73. Nicol S, Griffith B, Austin J, Hunter CM (2014) Optimal water depth management on river-fed National Wildlife Refuges in a changing climate. Clim Chang 124:271–284CrossRefGoogle Scholar
  74. Nicol S, Fuller RA, Iwamura T, Chadès I (2015) Adapting environmental management to uncertain but inevitable change. Proc R Soc B 282 doi:10.1098/rspb.2014.2984Google Scholar
  75. Nilim A, El Ghaoui L (2005) Robust control of Markov decision processes with uncertain transition matrices. Oper Res 53:780–798CrossRefGoogle Scholar
  76. Ong SCW, Png SW, Hsu D, Lee S (2010) Planning under Uncertainty for Robotic Tasks with Mixed Observability. Int J Robot Res 29:1053–1068CrossRefGoogle Scholar
  77. Osogami T (2015) Robust partially observable Markov decision process. In: Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France. pp 106–115Google Scholar
  78. Papadimitriou CH, Tsitsiklis JN (1987) The complexity of Markov decision processes. Math Oper Res 12:441–450. doi:10.1287/moor.12.3.441 CrossRefGoogle Scholar
  79. Parma AM (1998) What can adaptive management do for our fish, forests, food, and biodiversity? Integr Biol: Issues, News, Rev 1:16–26CrossRefGoogle Scholar
  80. Perny P, Weng P (2010) On finding compromise solutions in multiobjective Markov decision processes. In: European Conference on Artificial Intelligence (ECAI-2010), Lisbonne, Portugal. pp 969–970Google Scholar
  81. Pichancourt JB, Chadès I, Firn J, van Klinken RD, Martin TG (2012) Simple rules to contain an invasive species with a complex life cycle and high dispersal capacity. J Appl Ecol 49:52–62CrossRefGoogle Scholar
  82. Pineau J, Gordon G, Thrun S (2003) Point-based value iteration: An anytime algorithm for POMDPs. In: International Joint Conference on Artificial Intelligence. Lawrence Erlbaum Associates LTD, pp 1025–1032Google Scholar
  83. Poupart P (2005) Exploiting structure to efficiently solve large scale partially observable Markov decision processes. University of TorontoGoogle Scholar
  84. Puterman ML (1994) Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc, New YorkCrossRefGoogle Scholar
  85. Regan HM, Colyvan M, Burgman MA (2002) A taxonomy and treatment of uncertainty for ecology and conservation biology. Ecol Appl 12:618–628. doi:10.1890/1051-0761(2002)012[0618:atatou]2.0.co;2 CrossRefGoogle Scholar
  86. Regan TJ, Chadès I, Possingham HP (2011) Optimal strategies for managing invasive plants in partially observable systems. J Appl Ecol 48:76–85CrossRefGoogle Scholar
  87. Roijers DM, Whiteson S, Oliehoek FA (2015) Point-based planning for multi-objective POMDPs. In: Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI-2015), Buenos Aires, Argentina.Google Scholar
  88. Rout TM, Hauser CE, Possingham HP (2009) Optimal adaptive management for the translocation of a threatened species. Ecol Appl 19:515–526. doi:10.1890/07-1989.1 CrossRefPubMedGoogle Scholar
  89. Runge MC (2011) An Introduction to Adaptive Management for Threatened and Endangered Species. J Fish Wildl Manag 2:220–233. doi:10.3996/082011-jfwm-045 CrossRefGoogle Scholar
  90. Runge MC (2013) Active adaptive management for reintroduction of an animal population. J Wildl Manag 77:1135–1144. doi:10.1002/jwmg.571 CrossRefGoogle Scholar
  91. Runge MC, Converse SJ, Lyons JE (2011) Which uncertainty? Using expert elicitation and expected value of information to design an adaptive program. Biol Conserv 144:1214–1223CrossRefGoogle Scholar
  92. Schlaifer R, Raiffa H (1961) Applied statistical decision theory. Clinton Press, Inc., BostonGoogle Scholar
  93. Sethi G, Costello C, Fisher A, Hanemann M, Karp L (2005) Fishery management under multiple uncertainty. J Environ Econ Manag 50:300–318. doi:10.1016/j.jeem.2004.11.005 CrossRefGoogle Scholar
  94. Sigaud O, Buffet O (2010) Markov decision processes in artificial intelligence: MDPs, beyond MDPs and applications. ISTE/Wiley, HobokenGoogle Scholar
  95. Silvert W (1978) The Price of Knowledge: Fisheries Management as a Research Tool. J Fish Res Board Can 35:208–212. doi:10.1139/f78-034 CrossRefGoogle Scholar
  96. Smith ADM, Walters CJ (1981) Adaptive Management of Stock–Recruitment Systems. Can J Fish Aquat Sci 38:690–703. doi:10.1139/f81-092 CrossRefGoogle Scholar
  97. Smith DR, McGowan CP, Daily JP, Nichols JD, Sweka JA, Lyons JE (2013) Evaluating a multispecies adaptive management framework: must uncertainty impede effective decision-making? J Appl Ecol 50:1431–1440. doi:10.1111/1365-2664.12145 CrossRefGoogle Scholar
  98. Southwell DM, Hauser CE, McCarthy MA (2016) Learning about colonization when managing metapopulations under an adaptive management framework. Ecol Appl 26:279–294. doi:10.1890/14-2430 CrossRefPubMedGoogle Scholar
  99. Spaan M, Vlassis N (2005) Perseus: Randomized Point-based Value Iteration for POMDPs. J Artif Intell Res 24:195–220Google Scholar
  100. Springborn M, Sanchirico JN (2013) A density projection approach for non-trivial information dynamics: adaptive management of stochastic natural resources. J Environ Econ Manag 66:609–624CrossRefGoogle Scholar
  101. Venner S, Chadès I, Bel-Venner M-C, Pasquet A, Charpillet F, Leborgne R (2006) Dynamic optimization over infinite-time horizon: Web-building strategy in an orb-weaving spider as a case study. J Theor Biol 241:725–733CrossRefPubMedGoogle Scholar
  102. Walters CJ (1975) Optimal Harvest Strategies for Salmon in Relation to Environmental Variability and Uncertain Production Parameters. J Fish Res Board Can 32:1777–1784. doi:10.1139/f75-211 CrossRefGoogle Scholar
  103. Walters CJ (1986) Adaptive management of renewable resources. McGraw Hill, New YorkGoogle Scholar
  104. Walters C (1997) Challenges in adaptive management of riparian and coastal ecosystems. Conserv Ecol 1:1CrossRefGoogle Scholar
  105. Walters CJ, Hilborn R (1976) Adaptive Control of Fishing Systems. J Fish Res Board Can 33:145–159. doi:10.1139/f76-017 CrossRefGoogle Scholar
  106. Walters CJ, Hilborn R (1978) Ecological optimization and adaptive management. Annu Rev Ecol Syst 9:157–188CrossRefGoogle Scholar
  107. Walters CJ, Ludwig D (1981) Effects of Measurement Errors on the Assessment of Stock–Recruitment Relationships. Can J Fish Aquat Sci 38:704–710. doi:10.1139/f81-093 CrossRefGoogle Scholar
  108. Walters CJ, Ludwig D (1987) Adaptive management of harvest rates in the presence of a risk averse utility function. Nat Resour Model 1:321–337Google Scholar
  109. Westgate MJ, Likens GE, Lindenmayer DB (2013) Adaptive management of biological systems: A review. Biol Conserv 158:128–139. doi:10.1016/j.biocon.2012.08.016 CrossRefGoogle Scholar
  110. White B (2005) An economic analysis of ecological monitoring. Ecol Model 189:241–250CrossRefGoogle Scholar
  111. Wiesemann W, Kuhn D, Rustem B (2013) Robust Markov Decision Processes. Math Oper Res 38:153–183. doi:10.1287/moor.1120.0566 CrossRefGoogle Scholar
  112. Williams BK (2009) Markov decision processes in natural resources management: Observability and uncertainty. Ecol Model 220:830–840. doi:10.1016/j.ecolmodel.2008.12.023 CrossRefGoogle Scholar
  113. Williams BK (2011a) Passive and active adaptive management: Approaches and an example. J Environ Manag 92:1371–1378. doi:10.1016/j.jenvman.2010.10.039 CrossRefGoogle Scholar
  114. Williams BK (2011b) Resolving structural uncertainty in natural resources management using POMDP approaches. Ecol Model 222:1092–1102. doi:10.1016/j.ecolmodel.2010.12.015 CrossRefGoogle Scholar
  115. Williams BK, Johnson FA (2015) Value of information in natural resource management: technical developments and application to pink-footed geese. Ecol Evol 5:466–474. doi:10.1002/ece3.1363 CrossRefPubMedPubMedCentralGoogle Scholar
  116. Williams BK, Johnson FA, Wilkins K (1996) Uncertainty and the adaptive management of waterfowl harvests. J Wildl Manag 60:223–232. doi:10.2307/3802220 CrossRefGoogle Scholar
  117. Williams B, Szaro R, Shapiro C (2009) Adaptive management: the U.S. Department of the Interior technical guide, 2 edn. U.S. Department of the Interior, Washington, D.C. doi:http://www.doi.gov/initiatives/AdaptiveManagement/TechGuide.pdf
  118. Williams BK, Eaton MJ, Breininger DR (2011) Adaptive resource management and the value of information. Ecol Model 222:3429–3436. doi:10.1016/j.ecolmodel.2011.07.003 CrossRefGoogle Scholar
  119. Wilson KA, McBride MF, Bode M, Possingham HP (2006) Prioritizing global conservation efforts. Nature 440:337–340CrossRefPubMedGoogle Scholar
  120. Wittenmark B (1995) Adaptive Dual Control Methods: An Overview. In: In 5th IFAC symposium on Adaptive Systems in Control and Signal ProcessingGoogle Scholar
  121. Zhou R, Hansen E (2001) An improved grid-based approximation algorithm for POMDPs. In: Proceedings of the 17th International Joint Conference on Artificial Intelligence (IJCAI-2001), Seattle, Washington, USAGoogle Scholar
  122. Zhou E, Fu MC, Marcus S (2010) Solving continuous-state POMDPs via density projection. IEEE Trans Autom Control 55:1101–1116CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  • Iadine Chadès
    • 1
  • Sam Nicol
    • 1
  • Tracy M. Rout
    • 2
  • Martin Péron
    • 1
    • 3
  • Yann Dujardin
    • 1
  • Jean-Baptiste Pichancourt
    • 1
  • Alan Hastings
    • 4
  • Cindy E. Hauser
    • 2
  1. 1.CSIROBrisbaneAustralia
  2. 2.School of BioSciencesUniversity of MelbourneParkville VicAustralia
  3. 3.School of Mathematical SciencesQueensland University of TechnologyBrisbaneAustralia
  4. 4.Department of Environmental Science and PolicyUniversity of CaliforniaDavisUSA

Personalised recommendations