Optimization methods to solve adaptive management problems

Abstract

Determining the best management actions is challenging when critical information is missing. However, urgency and limited resources require that decisions must be made despite this uncertainty. The best practice method for managing uncertain systems is adaptive management, or learning by doing. Adaptive management problems can be solved optimally using decision-theoretic methods; the challenge for these methods is to represent current and future knowledge using easy-to-optimize representations. Significant methodological advances have been made since the seminal adaptive management work was published in the 1980s, but despite recent advances, guidance for implementing these approaches has been piecemeal and study-specific. There is a need to collate and summarize new work. Here, we classify methods and update the literature with the latest optimal or near-optimal approaches for solving adaptive management problems. We review three mathematical concepts required to solve adaptive management problems: Markov decision processes, sufficient statistics, and Bayes’ theorem. We provide a decision tree to determine whether adaptive management is appropriate and then group adaptive management approaches based on whether they learn only from the past (passive) or anticipate future learning (active). We discuss the assumptions made when using existing models and provide solution algorithms for each approach. Finally, we propose new areas of development that could inspire future research. For a long time, limited by the efficiency of the solution methods, recent techniques to efficiently solve partially observable decision problems now allow us to solve more realistic adaptive management problems such as imperfect detection and non-stationarity in systems.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2

References

  1. Amato C, Oliehoek FA Scalable Planning and Learning for Multiagent POMDPs. In: Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015

  2. Åström KJ (1965) Optimal control of Markov decision processes with incomplete state estimation. J Math Anal Appl 10:174–205

    Article  Google Scholar 

  3. Åström K, Wittenmark B (2008) Adaptive control, 2nd edn. Dover Publications, Mineola

    Google Scholar 

  4. Bellman RE (1957) Dynamic Programming. Princeton University Press, Princeton

    Google Scholar 

  5. Bernstein DS, Givan R, Immerman N, Zilberstein S (2002) The complexity of decentralized control of Markov decision processes. Math Oper Res 27:819–840

    Article  Google Scholar 

  6. Bertsekas DP (1995) Dynamic programming and optimal control vol 1, vol 2. Athena Scientific Belmont, MA

    Google Scholar 

  7. Bonet B (2002) An epsilon-optimal grid-based algorithm for partially observable Markov decision processes. In: Proceedings of the 19th International Conference on Machine Learning (ICML-02), Sydney, Australia. Morgan Kaufman Publishers Inc., pp 51–58

  8. Boutilier C, Dearden R (1994) Using abstractions for decision-theoretic planning with time constraints. In: Proceedings of the Twelfth AAAI National Conference on Artificial Intelligence. AAAI Press, pp 1016–1022

  9. Boutilier C (1999) Sequential optimality and coordination in multiagent systems. In: IJCAI. pp 478–485

  10. Brafman R (1997) A heuristic variable grid solution method for POMDPs. In: Proceedings of the National Conference on Artificial Intelligence (AAAI-97), Providence, Rhode Island. pp 727–733

  11. Canessa S et al (2015) When do we need more data? A primer on calculating the value of information for applied ecologists. Methods Ecol Evol 6:1219–1228. doi:10.1111/2041-210x.12423

    Article  Google Scholar 

  12. Canessa S et al (2016) Adaptive management for improving species conservation across the captive-wild spectrum. Biol Conserv 199:123–131. doi:10.1016/j.biocon.2016.04.026

    Article  Google Scholar 

  13. Cassandra AR, Kaelbling LP (1995) Learning policies for partially observable environments: Scaling up. In: Machine Learning Proceedings 1995: Proceedings of the Twelfth International Conference on Machine Learning, Tahoe City, California. Morgan Kaufmann, p 362

  14. Chades I, Bouteiller B Solving multiagent Markov decision processes: a forest management example. In: Proceedings of the International Congress on Modelling and Simulation (MODSIM 2005), 2005. pp 1594–1600

  15. Chades I, Scherrer B, Charpillet F (2002) A heuristic approach for solving decentralized-pomdp: Assessment on the pursuit problem. In: Proceedings of the 2002 ACM symposium on Applied computing. ACM, pp 57–62

  16. Chadès I, McDonald-Madden E, McCarthy MA, Wintle B, Linkie M, Possingham HP (2008) When to stop managing or surveying cryptic threatened species. Proc Natl Acad Sci U S A 105:13936

    Article  PubMed  PubMed Central  Google Scholar 

  17. Chadès I, Martin TG, Nicol S, Burgman MA, Possingham HP, Buckley YM (2011) General rules for managing and surveying networks of pests, diseases, and endangered species. Proc Natl Acad Sci 108:8323–8328. doi:10.1073/pnas.1016846108

    Article  PubMed  PubMed Central  Google Scholar 

  18. Chadès I, Carwardine J, Martin TG, Nicol S, Sabbadin R, Buffet O (2012) MOMDPs: a solution for modelling adaptive management problems. In: The Twenty-Sixth AAAI Conference on Artificial Intelligence (AAAI-12), Toronto, Canada. pp 267–273

  19. Chadès I, Chapron G, Cros M-J, Garcia F, Sabbadin R (2014) MDPtoolbox: a multi-platform toolbox to solve stochastic dynamic programming problems. Ecography 37:916–920

    Article  Google Scholar 

  20. Charles AT (1992) Uncertainty and information in fishery management models: a Bayesian updating algorithm. Am J Math Manag Sci 12:191–225

    Google Scholar 

  21. Dibangoye JS, Amato C, Buffet O, Charpillet F (2016) Optimally solving Dec-POMDPs as continuous-state MDPs. J Artif Intell Res 55:443–497

    Google Scholar 

  22. Dujardin Y, Dietterich T, Chadès I (2015) alpha-min: a compact POMDP solver. In: International Joint Conference on Artificial Intelligence (IJCAI-2015), Buenos Aires, Argentina

  23. Ehrgott M (2005) Multicriteria optimization, 2nd edn. Springer, Berlin

    Google Scholar 

  24. Fackler P (2013) MDPSOLVE Software for Dynamic Optimization

  25. Fackler PL, Haight RG (2014) Monitoring as a partially observable decision problem. Resour Energy Econ 37:226–241

    Article  Google Scholar 

  26. Fackler P, Pacifici K (2014) Addressing structural and observational uncertainty in resource management. J Environ Manag 133:27–36. doi:10.1016/j.jenvman.2013.11.004

    Article  Google Scholar 

  27. Filatov NM, Unbehauen H (2000) Survey of adaptive dual control methods. IEE Proc - Control Theory Appl 147:118–128. doi:10.1049/ip-cta:20000107

    Article  Google Scholar 

  28. Firn J, Rout T, Possingham H, Buckley YM (2008) Managing beyond the invader: manipulating disturbance of natives simplifies control efforts. J Appl Ecol 45:1143–1151. doi:10.1111/j.1365-2664.2008.01510.x

    Google Scholar 

  29. Fisher RA (1922) On the Mathematical Foundations of Theoretical Statistics. Philos Trans R Soc Lond A: Math, Phys Eng Sci 222:309–368. doi:10.1098/rsta.1922.0009

    Article  Google Scholar 

  30. Frederick SW, Peterman RM (1995) Choosing fisheries harvest policies: when does uncertainty matter? Can J Fish Aquat Sci 52:291–306. doi:10.1139/f95-030

    Article  Google Scholar 

  31. Fulton EA, Smith ADM, Smith DC, van Putten IE (2011) Human behaviour: the key source of uncertainty in fisheries management. Fish Fish 12:2–17. doi:10.1111/j.1467-2979.2010.00371.x

    Article  Google Scholar 

  32. Givan R, Leach S, Dean T (2000) Bounded-parameter Markov decision processes. Artif Intell 1:71–109

    Article  Google Scholar 

  33. Gregory R, Ohlson D, Arvai J (2006) Deconstructing adaptive management: citeria for applications to environmental management. Ecol Appl 16:2411–2425

    CAS  Article  PubMed  Google Scholar 

  34. Grewal MS (2011) Kalman filtering. Springer

  35. Haight RG, Polasky S (2010) Optimal control of an invasive species with imperfect information about the level of infestation. Resour Energy Econ 32:519–533

    Article  Google Scholar 

  36. Hauser CE, Possingham HP (2008) Experimental or precautionary? Adaptive management over a range of time horizons. J Appl Ecol 45:72–81. doi:10.1111/j.1365-2664.2007.01395.x

    Article  Google Scholar 

  37. Holling CS (1978) Adaptive environmental assessment and management. John Wiley & Sons, London

    Google Scholar 

  38. Houston A, Clark C, McNamara J, Mangel M (1988) Dynamic models in behavioural and evolutionary ecology. Nature 332:29–34

    Article  Google Scholar 

  39. Johnson FA, Clinton TM, Kendall WL, Dubovsky JA, Caithamer DF, Kelley JR Jr, Byron KW (1997) Uncertainty and the Management of Mallard Harvests. J Wildl Manag 61:202–216. doi:10.2307/3802429

    Article  Google Scholar 

  40. Johnson FA, Kendall WL, Dubovsky JA (2002) Conditions and limitations on learning in the adaptive management of mallard harvests. Wildl Soc Bull 176–185

  41. Kareiva P, Groves C, Marvier M (2014) REVIEW: The evolving linkage between conservation science and practice at The Nature Conservancy. J Appl Ecol 51:1137–1147. doi:10.1111/1365-2664.12259

    Article  PubMed  PubMed Central  Google Scholar 

  42. Keith DA, Martin TG, McDonald-Madden E, Walters C (2011) Uncertainty and adaptive management for biodiversity conservation. Biol Conserv 144:1175–1178

    Article  Google Scholar 

  43. Kurniawati H, Hsu D, Lee W-S (2008) SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces. In: Proceedings of Robotics: Science and Systems IV, Zurich, Switzerland. pp 65–72

  44. Littman ML (1994) Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of the eleventh international conference on machine learning. pp 157–163

  45. Lovejoy W (1991) Computationally feasible bounds for partially observed Markov decisions processes. Oper Res 39:162–175

    Article  Google Scholar 

  46. Lubow BC (1997) Adaptive Stochastic Dynamic Programming (ASDP): Supplement to SFP User’s Guide, 20th edn. Colorado Cooperative Fish and Wildlife Research Unit, Colorado State University, Fort collins

    Google Scholar 

  47. Ludwig D, Walters CJ (1981) Measurement Errors and Uncertainty in Parameter Estimates for Stock and Recruitment. Can J Fish Aquat Sci 38:711–720. doi:10.1139/f81-094

    Article  Google Scholar 

  48. MacKenzie DI (2009) Getting the biggest bang for our conservation buck. Trends Ecol Evol (Personal Ed) 24:175–177

    Article  Google Scholar 

  49. Madani O, Hanks S, Condon A (2003) On the undecidability of probabilistic planning and related stochastic optimization problems. Artif Intell 147:5–34

    Article  Google Scholar 

  50. Mangel M, Clark CW (1983) Uncertainty, search, and information in fisheries. J Conseil 41:93–103. doi:10.1093/icesjms/41.1.93

    Article  Google Scholar 

  51. Marescot L, Chapron G, Chadès I, Fackler P, Duchamp C, Marboutin E, Gimenez O (2013) Complex decisions made simple: a primer on stochastic dynamic programming. Methods Ecol Evol 4:872–884

    Article  Google Scholar 

  52. Martin J, Runge MC, Nichols JD, Lubow BC, Kendall WL (2009) Structured decision making as a conceptual framework to identify thresholds for conservation and management. Ecol Appl 19:1079–1090

    Article  PubMed  Google Scholar 

  53. Martin J et al (2011) Structured decision making as a proactive approach to dealing with sea level rise in Florida. Clim Chang 107:185–202

    Article  Google Scholar 

  54. Martin TG, Camaclang AE, Possingham HP, Maguire LA, Chadès I (2016) Timing of Protection of Critical Habitat Matters. Conserv Lett:n/a-n/a. doi:10.1111/conl.12266

    Google Scholar 

  55. McCarthy MA (2007) Bayesian methods for ecology. Cambridge University Press, Cambridge

    Google Scholar 

  56. McCarthy MA, Possingham HP (2007) Active adaptive management for conservation. Conserv Biol 21:956–963

    Article  PubMed  Google Scholar 

  57. McCarthy MA, Possingham HP, Gill AM (2001) Using stochastic dynamic programming to determine optimal fire management for Banksia ornata. J Appl Ecol 38:585–592

    Article  Google Scholar 

  58. McCarthy MA, Armstrong DP, Runge MC (2012) Adaptive Management of Reintroduction. In: Reintroduction Biology. John Wiley & Sons, Ltd, pp 256–289. doi:10.1002/9781444355833.ch8

  59. McDonald-Madden E et al (2010a) Active adaptive conservation of threatened species in the face of uncertainty. Ecol Appl 20:1476–1489. doi:10.1890/09-0647.1

    Article  PubMed  Google Scholar 

  60. McDonald-Madden E, Baxter PWJ, Fuller RA, Martin TG, Game ET, Montambault J, Possingham HP (2010b) Monitoring does not always count. Trends Ecol Evol 25:547–550. doi:10.1016/j.tree.2010.07.002

    Article  PubMed  Google Scholar 

  61. McDonald-Madden E, Chadès I, McCarthy MA, Linkie M, Possingham HP (2011) Allocating conservation resources between areas where persistence of a species is uncertain. Ecol Appl 21:844–858. doi:10.1890/09-2075.1

    Article  PubMed  Google Scholar 

  62. Mehta SV, Haight RG, Homans FR, Polasky S, Venette RC (2007) Optimal detection and control strategies for invasive species management. Ecol Econ 61:237–245. doi:10.1016/j.ecolecon.2006.10.024

    Article  Google Scholar 

  63. Monahan GE (1982) Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms. MGMT SCI 28:1–16

    Article  Google Scholar 

  64. Moore CT, Conroy MJ (2006) Optimal regeneration planning for old-growth forest: addressing scientific uncertainty in endangered species recovery through adaptive management. For Sci 52:155–172

    Google Scholar 

  65. Moore AL, McCarthy MA (2010) On Valuing Information in Adaptive-Management Models. Conserv Biol 24:984–993. doi:10.1111/j.1523-1739.2009.01443.x

    Article  PubMed  Google Scholar 

  66. Moore AL, Hauser CE, McCarthy MA (2008) How we value the future affects our desire to learn. Ecol Appl 18:1061–1069. doi:10.1890/07-0805.1

    Article  PubMed  Google Scholar 

  67. Moore CT et al (2011) An Adaptive Decision Framework for the Conservation of a Threatened Plant. J Fish Wildl Manag 2:247–261. doi:10.3996/012011-jfwm-007

    Article  Google Scholar 

  68. Nichols JD, Johnson FA, Byron KW (1995) Managing North American Waterfowl in the Face of Uncertainty. Annu Rev Ecol Syst 26:177–199. doi:10.2307/2097204

    Article  Google Scholar 

  69. Nichols JD et al (2011) Climate change, uncertainty, and natural resource management. J Wildl Manag 75:6–18

    Article  Google Scholar 

  70. Nicol S, Chadès I (2012) Which States Matter? An Application of an Intelligent Discretization Method to Solve a Continuous POMDP in Conservation Biology. PLoS ONE 7:e28993. doi:10.1371/journal.pone.0028993

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  71. Nicol SC, Possingham HP (2010) Should metapopulation restoration strategies increase patch area or number of patches? Ecol Appl 20:566–581

    Article  PubMed  Google Scholar 

  72. Nicol S, Buffet O, Iwamura T, Chadès I (2013) Adaptive Management of Migratory Birds Under Sea Level Rise. In: Proceedings of the 23rd International Joint Conference on Artificial Intelligence, Beijing. pp 2955–2957

  73. Nicol S, Griffith B, Austin J, Hunter CM (2014) Optimal water depth management on river-fed National Wildlife Refuges in a changing climate. Clim Chang 124:271–284

    Article  Google Scholar 

  74. Nicol S, Fuller RA, Iwamura T, Chadès I (2015) Adapting environmental management to uncertain but inevitable change. Proc R Soc B 282 doi:10.1098/rspb.2014.2984

  75. Nilim A, El Ghaoui L (2005) Robust control of Markov decision processes with uncertain transition matrices. Oper Res 53:780–798

    Article  Google Scholar 

  76. Ong SCW, Png SW, Hsu D, Lee S (2010) Planning under Uncertainty for Robotic Tasks with Mixed Observability. Int J Robot Res 29:1053–1068

    Article  Google Scholar 

  77. Osogami T (2015) Robust partially observable Markov decision process. In: Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France. pp 106–115

  78. Papadimitriou CH, Tsitsiklis JN (1987) The complexity of Markov decision processes. Math Oper Res 12:441–450. doi:10.1287/moor.12.3.441

    Article  Google Scholar 

  79. Parma AM (1998) What can adaptive management do for our fish, forests, food, and biodiversity? Integr Biol: Issues, News, Rev 1:16–26

    Article  Google Scholar 

  80. Perny P, Weng P (2010) On finding compromise solutions in multiobjective Markov decision processes. In: European Conference on Artificial Intelligence (ECAI-2010), Lisbonne, Portugal. pp 969–970

  81. Pichancourt JB, Chadès I, Firn J, van Klinken RD, Martin TG (2012) Simple rules to contain an invasive species with a complex life cycle and high dispersal capacity. J Appl Ecol 49:52–62

    Article  Google Scholar 

  82. Pineau J, Gordon G, Thrun S (2003) Point-based value iteration: An anytime algorithm for POMDPs. In: International Joint Conference on Artificial Intelligence. Lawrence Erlbaum Associates LTD, pp 1025–1032

  83. Poupart P (2005) Exploiting structure to efficiently solve large scale partially observable Markov decision processes. University of Toronto

  84. Puterman ML (1994) Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc, New York

    Google Scholar 

  85. Regan HM, Colyvan M, Burgman MA (2002) A taxonomy and treatment of uncertainty for ecology and conservation biology. Ecol Appl 12:618–628. doi:10.1890/1051-0761(2002)012[0618:atatou]2.0.co;2

    Article  Google Scholar 

  86. Regan TJ, Chadès I, Possingham HP (2011) Optimal strategies for managing invasive plants in partially observable systems. J Appl Ecol 48:76–85

    Article  Google Scholar 

  87. Roijers DM, Whiteson S, Oliehoek FA (2015) Point-based planning for multi-objective POMDPs. In: Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI-2015), Buenos Aires, Argentina.

  88. Rout TM, Hauser CE, Possingham HP (2009) Optimal adaptive management for the translocation of a threatened species. Ecol Appl 19:515–526. doi:10.1890/07-1989.1

    Article  PubMed  Google Scholar 

  89. Runge MC (2011) An Introduction to Adaptive Management for Threatened and Endangered Species. J Fish Wildl Manag 2:220–233. doi:10.3996/082011-jfwm-045

    Article  Google Scholar 

  90. Runge MC (2013) Active adaptive management for reintroduction of an animal population. J Wildl Manag 77:1135–1144. doi:10.1002/jwmg.571

    Article  Google Scholar 

  91. Runge MC, Converse SJ, Lyons JE (2011) Which uncertainty? Using expert elicitation and expected value of information to design an adaptive program. Biol Conserv 144:1214–1223

    Article  Google Scholar 

  92. Schlaifer R, Raiffa H (1961) Applied statistical decision theory. Clinton Press, Inc., Boston

    Google Scholar 

  93. Sethi G, Costello C, Fisher A, Hanemann M, Karp L (2005) Fishery management under multiple uncertainty. J Environ Econ Manag 50:300–318. doi:10.1016/j.jeem.2004.11.005

    Article  Google Scholar 

  94. Sigaud O, Buffet O (2010) Markov decision processes in artificial intelligence: MDPs, beyond MDPs and applications. ISTE/Wiley, Hoboken

    Google Scholar 

  95. Silvert W (1978) The Price of Knowledge: Fisheries Management as a Research Tool. J Fish Res Board Can 35:208–212. doi:10.1139/f78-034

    Article  Google Scholar 

  96. Smith ADM, Walters CJ (1981) Adaptive Management of Stock–Recruitment Systems. Can J Fish Aquat Sci 38:690–703. doi:10.1139/f81-092

    Article  Google Scholar 

  97. Smith DR, McGowan CP, Daily JP, Nichols JD, Sweka JA, Lyons JE (2013) Evaluating a multispecies adaptive management framework: must uncertainty impede effective decision-making? J Appl Ecol 50:1431–1440. doi:10.1111/1365-2664.12145

    Article  Google Scholar 

  98. Southwell DM, Hauser CE, McCarthy MA (2016) Learning about colonization when managing metapopulations under an adaptive management framework. Ecol Appl 26:279–294. doi:10.1890/14-2430

    Article  PubMed  Google Scholar 

  99. Spaan M, Vlassis N (2005) Perseus: Randomized Point-based Value Iteration for POMDPs. J Artif Intell Res 24:195–220

    Google Scholar 

  100. Springborn M, Sanchirico JN (2013) A density projection approach for non-trivial information dynamics: adaptive management of stochastic natural resources. J Environ Econ Manag 66:609–624

    Article  Google Scholar 

  101. Venner S, Chadès I, Bel-Venner M-C, Pasquet A, Charpillet F, Leborgne R (2006) Dynamic optimization over infinite-time horizon: Web-building strategy in an orb-weaving spider as a case study. J Theor Biol 241:725–733

    Article  PubMed  Google Scholar 

  102. Walters CJ (1975) Optimal Harvest Strategies for Salmon in Relation to Environmental Variability and Uncertain Production Parameters. J Fish Res Board Can 32:1777–1784. doi:10.1139/f75-211

    Article  Google Scholar 

  103. Walters CJ (1986) Adaptive management of renewable resources. McGraw Hill, New York

    Google Scholar 

  104. Walters C (1997) Challenges in adaptive management of riparian and coastal ecosystems. Conserv Ecol 1:1

    Article  Google Scholar 

  105. Walters CJ, Hilborn R (1976) Adaptive Control of Fishing Systems. J Fish Res Board Can 33:145–159. doi:10.1139/f76-017

    Article  Google Scholar 

  106. Walters CJ, Hilborn R (1978) Ecological optimization and adaptive management. Annu Rev Ecol Syst 9:157–188

    Article  Google Scholar 

  107. Walters CJ, Ludwig D (1981) Effects of Measurement Errors on the Assessment of Stock–Recruitment Relationships. Can J Fish Aquat Sci 38:704–710. doi:10.1139/f81-093

    Article  Google Scholar 

  108. Walters CJ, Ludwig D (1987) Adaptive management of harvest rates in the presence of a risk averse utility function. Nat Resour Model 1:321–337

    Google Scholar 

  109. Westgate MJ, Likens GE, Lindenmayer DB (2013) Adaptive management of biological systems: A review. Biol Conserv 158:128–139. doi:10.1016/j.biocon.2012.08.016

    Article  Google Scholar 

  110. White B (2005) An economic analysis of ecological monitoring. Ecol Model 189:241–250

    Article  Google Scholar 

  111. Wiesemann W, Kuhn D, Rustem B (2013) Robust Markov Decision Processes. Math Oper Res 38:153–183. doi:10.1287/moor.1120.0566

    Article  Google Scholar 

  112. Williams BK (2009) Markov decision processes in natural resources management: Observability and uncertainty. Ecol Model 220:830–840. doi:10.1016/j.ecolmodel.2008.12.023

    Article  Google Scholar 

  113. Williams BK (2011a) Passive and active adaptive management: Approaches and an example. J Environ Manag 92:1371–1378. doi:10.1016/j.jenvman.2010.10.039

    Article  Google Scholar 

  114. Williams BK (2011b) Resolving structural uncertainty in natural resources management using POMDP approaches. Ecol Model 222:1092–1102. doi:10.1016/j.ecolmodel.2010.12.015

    Article  Google Scholar 

  115. Williams BK, Johnson FA (2015) Value of information in natural resource management: technical developments and application to pink-footed geese. Ecol Evol 5:466–474. doi:10.1002/ece3.1363

    Article  PubMed  PubMed Central  Google Scholar 

  116. Williams BK, Johnson FA, Wilkins K (1996) Uncertainty and the adaptive management of waterfowl harvests. J Wildl Manag 60:223–232. doi:10.2307/3802220

    Article  Google Scholar 

  117. Williams B, Szaro R, Shapiro C (2009) Adaptive management: the U.S. Department of the Interior technical guide, 2 edn. U.S. Department of the Interior, Washington, D.C. doi:http://www.doi.gov/initiatives/AdaptiveManagement/TechGuide.pdf

  118. Williams BK, Eaton MJ, Breininger DR (2011) Adaptive resource management and the value of information. Ecol Model 222:3429–3436. doi:10.1016/j.ecolmodel.2011.07.003

    Article  Google Scholar 

  119. Wilson KA, McBride MF, Bode M, Possingham HP (2006) Prioritizing global conservation efforts. Nature 440:337–340

    CAS  Article  PubMed  Google Scholar 

  120. Wittenmark B (1995) Adaptive Dual Control Methods: An Overview. In: In 5th IFAC symposium on Adaptive Systems in Control and Signal Processing

  121. Zhou R, Hansen E (2001) An improved grid-based approximation algorithm for POMDPs. In: Proceedings of the 17th International Joint Conference on Artificial Intelligence (IJCAI-2001), Seattle, Washington, USA

  122. Zhou E, Fu MC, Marcus S (2010) Solving continuous-state POMDPs via density projection. IEEE Trans Autom Control 55:1101–1116

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Gwen Iacona and Ayesha Tulloch for commenting on earlier versions of this manuscript. The idea of this review paper emerged at the “Natural resource management” workshop organized by the Mathematical Biosciences Institute, Columbus (2013) and an adaptive management workshop supported by a CSIRO Julius Career Award (IC). TMR was supported by an Australian Research Council Discovery Grant (DP110101499). CEH was supported by the National Environmental Research Program Environmental Decisions Hub.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Iadine Chadès.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Fig. S1

(DOCX 27 kb)

Table S1

(DOCX 23 kb)

Table S2

(DOCX 34 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chadès, I., Nicol, S., Rout, T.M. et al. Optimization methods to solve adaptive management problems. Theor Ecol 10, 1–20 (2017). https://doi.org/10.1007/s12080-016-0313-0

Download citation

Keywords

  • Adaptive management
  • Markov decision process
  • MDP
  • Partially observable Markov decision process
  • POMDP
  • Stochastic dynamic programming
  • Value of information
  • Hidden Markov models
  • Natural resource management
  • Conservation