What Is Expert Knowledge, How Is Such Knowledge Gathered, and How Do We Use It to Address Questions in Landscape Ecology?



Expert knowledge plays an integral role in applied ecology and conservation (Burgman 2005). Environmental systems are characterized by complex dynamics, multiple drivers, and a paucity of data (Carpenter 2002). Action is often required before uncertainties can be resolved. Where empirical data are scarce or unavailable, expert knowledge is often regarded as the best or only source of information (Sutherland 2006; Kuhnert et al. 2010). Experts may be called upon to provide input for all stages of the modeling and management process, and specifically to inform the definition and structuring of the problem (Cowling and Pressey 2003; Sutherland et al. 2008), to inform the selection of data or variables, model structures, and assumptions about functional relationships between variables (Pearce et al. 2001; Czembor and Vesk 2009), and to inform the analysis of data, estimation of parameters, interpretation of results, and the characterization of uncertainty (Alho and Kangas 1997; Martin et al. 2005).


Baran Beach Alho 


  1. Al-Awadhi SA, Garthwaite PH (1998) An elicitation method for multivariate normal distributions. Commun Stat A-Theor 27:1123–1142CrossRefGoogle Scholar
  2. Alho JM, Kangas J (1997) Analyzing uncertainties in experts’ opinions of forest plan performance. For Sci 43:521–528Google Scholar
  3. Anderson EL, Hattis D (1999) A. Uncertainty and variability. Risk Anal 19:47–49Google Scholar
  4. Anderson JL (1998) Embracing uncertainty: the interface of Bayesian statistics and cognitive psychology. Ecol Soc 2(1), article 2. Available from http://www.consecol.org/vol2/iss1/art2/ (accessed May 2011)Google Scholar
  5. Armstrong JS (ed) (2001) Principles of forecasting: a handbook for researchers and practitioners. Kluwer Academic Publishers, NorwellGoogle Scholar
  6. Armstrong JS (2006) Findings from evidence-based forecasting: methods for reducing forecast error. Int J Forecasting 22:583–598CrossRefGoogle Scholar
  7. Arnott D (2006) Cognitive biases and decision support systems development: a design science approach. Inform Syst J 16:55–78CrossRefGoogle Scholar
  8. Ausden M, Sutherland WJ, James R (2001) The effects of flooding lowland wet grassland on soil macroinvertebrate prey of breeding wading birds. J Appl Ecol 38:320–338CrossRefGoogle Scholar
  9. Ayyub BM (2001) Elicitation of expert opinions for uncertainty and risks. CRC Press, Boca RatonCrossRefGoogle Scholar
  10. Baran N (2000) Effective survey methods for detecting plants. MSc Thesis. University of Melbourne, MelbourneGoogle Scholar
  11. Bates JM, Granger CWJ (1969) The combination of forecasts. Oper Res Q 20:451–468CrossRefGoogle Scholar
  12. Beyth-Marom R (1982) How probable is probable? A numerical translation of verbal probability expressions. J Forecasting 1:257–269CrossRefGoogle Scholar
  13. Booker JM, McNamara LA (2004) Solving black box computation problems using expert knowledge theory and methods. Reliab Eng Syst Safe 85:331–340CrossRefGoogle Scholar
  14. Bransford JD, Brown AL, Cocking RR (2000) How people learn: brain, mind, experience and school. National Academy Press, WashingtonGoogle Scholar
  15. Brun W, Teigen KH (1988) Verbal probabilities: ambiguous, context-dependent, or both. Organ Behav Hum Dec 41:390–404CrossRefGoogle Scholar
  16. Budescu, DV, Wallsten TS (1985) Consistency in interpretation of probabilistic phrases. Organ Behav Hum Dec 36:391–405CrossRefGoogle Scholar
  17. Burgman MA (2000) Population viability analysis for bird conservation: prediction, heuristics, monitoring and psychology. Emu 100:347–353CrossRefGoogle Scholar
  18. Burgman MA (2005) Risks and decisions for conservation and environmental management. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  19. Burgman MA, Carr A, Godden L et al (2011) Redefining expertise and improving ecological judgement. Conserv Lett 4:81–87CrossRefGoogle Scholar
  20. Camerer CF, Johnson EJ (1997) The process-performance paradox in expert judgment: how can experts know so much and predict so badly? In: Goldstein WM, Hogarth RM (eds) Research on judgment and decision making: currents, connections and controversies. Cambridge University Press, Cambridge, pp 342–364Google Scholar
  21. Campbell LM (2002) Science and sustainable use: views of marine turtle conservation experts. Ecol Appl 12:1229–1246CrossRefGoogle Scholar
  22. Carpenter SR (2002) Ecological futures: building an ecology of the long now. Ecology 83:2069–2083Google Scholar
  23. Chase WG, Simon HA (1973) The mind’s eye in chess. In: Chase WG (ed) Visual information processing. Academic Press, New York, pp 215–281Google Scholar
  24. Chi MTH (2006) Two approaches to the study of experts’ characteristics. In: Ericsson KA, Charness N, Feltovich PJ, Hoffman, RR (eds) The Cambridge handbook of expertise and expert performance. Cambridge University Press, New York, pp 21–30Google Scholar
  25. Christen JA, Nakamura M (2000) On the analysis of accumulation curves. Biometrics 56:748–754PubMedCrossRefGoogle Scholar
  26. Christensen-Szalanski JJJ, Bushyhead JB (1981) Physicians’ use of probabilistic information in a real clinical setting. J Exp Psychol Human Percept Perform 7:125–126CrossRefGoogle Scholar
  27. Chuenpagdee R, Morgan LE, Maxwell SM et al (2003) Shifting gears: assessing collateral impacts of fishing methods in the U.S. waters.  Front Ecol Environ 10:517–524CrossRefGoogle Scholar
  28. Clemen RT (1989) Combining forecasts: a review and annotated bibliography. Int J Forecasting 5:559–583CrossRefGoogle Scholar
  29. Clemen RT, Winkler RL (1985) Limits for the precision and value of information from dependent sources. Oper Res 33:427–442CrossRefGoogle Scholar
  30. Clemen RT, Winkler RL (1999) Combining probability distributions from experts in risk analysis. Risk Anal 19:187–203Google Scholar
  31. Cohen MJ, Carstenn S, Lane CR (2004) Floristic quality indices for biotic assessment of depressional marsh condition in Florida. Ecol Appl 14:784–794CrossRefGoogle Scholar
  32. Collins, HM, Evans R (2007) Rethinking expertise. University of Chicago Press, ChicagoGoogle Scholar
  33. Cooke RM (1991) Experts in uncertainty: opinion and subjective probability in science. Oxford University Press, New YorkGoogle Scholar
  34. Cooke RM, Goossens LHJ (2000) Procedures guide for structured expert judgement in accident consequence modelling. Radiat Prot Dosim 90:303–309Google Scholar
  35. Cosmides L, Tooby J (1996) Are humans good intuitive statisticians after all? Rethinking some conclusions from the literature on judgment under uncertainty. Cognition 58:1–73CrossRefGoogle Scholar
  36. Cowling RM, Pressey RL (2003) Introduction to systematic conservation planning in the Cape Floristic Region. Biol Conserv 112:1–13CrossRefGoogle Scholar
  37. Crance JHBR (1987) Guidelines for using the Delphi technique to develop habitat suitability index curves. U.S. Fish Wildl Serv., Washington. Biological Report#82(10.134)Google Scholar
  38. Crome FHJ, Thomas MR, Moore LA (1996) A novel Bayesian approach to assessing impacts of rain forest logging. Ecol Appl 6:1104–1123CrossRefGoogle Scholar
  39. Currie F, Elliott G (1997) Forests and birds: a guide to managing forests for rare birds. Forestry Authority, Cambridge, and Royal Society for the Protection of Birds, SandyGoogle Scholar
  40. Czembor CA, Vesk PA (2009) Incorporating between-expert uncertainty into state-and-transition simulation models for forest restoration. For Ecol Manage 259:165–175CrossRefGoogle Scholar
  41. Dawes RM, Kagan J (1988) Rational choice in an uncertain world. Harcourt Brace Jovanovich, San DiegoGoogle Scholar
  42. Dickinson JP (1973) Some statistical results in combination of forecasts. Oper Res Q 24:253–260CrossRefGoogle Scholar
  43. Dickinson JP (1975) Some comments on combination of forecasts. Oper Res Q 26:205–210CrossRefGoogle Scholar
  44. Drescher, MA. Perera AH, Buse LJ et al (2008) Uncertainty in expert knowledge of forest succession: a case study from boreal Ontario. For Chron 84:194–209Google Scholar
  45. Ericsson KA (2004) Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 79:S70–S81PubMedCrossRefGoogle Scholar
  46. Ericsson KA, Charness N (1994) Expert performance: its structure and acquisition. Am Psychol 49:725–747CrossRefGoogle Scholar
  47. Ericsson KA, Charness N, Feltovich PJ et al (eds) (2006) The Cambridge handbook of expertise and expert performance. Cambridge University Press, New YorkGoogle Scholar
  48. Ericsson KA, Kintsch W (1995) Long-term working memory. Psychol Rev 102:211–245PubMedCrossRefGoogle Scholar
  49. Ericsson KA, Lehmann AC (1996) Expert and exceptional performance: evidence of maximal adaptation to task constraints. Annu Rev Psychol 47:273–305PubMedCrossRefGoogle Scholar
  50. Ericsson KA, Towne TJ  (2010)  Expertise.  Wiley Interdisciplinary Reviews: Cognitive Science 1:404–416CrossRefGoogle Scholar
  51. Fazey I, Fazey JA, Fazey DMA (2005) Learning more effectively from experience. Ecol Soc 10(2), article 4. Available from http://www.ecologyandsociety.org/vol10/iss2/art4/ (accessed May 2011)Google Scholar
  52. Ferrell WR (1994) Discrete subjective probabilities and decision analysis: elicitation, calibration and combination. In: Wright G, Ayton P (eds) Subjective probability. Wiley, New YorkGoogle Scholar
  53. Ferson S, Ginzburg LR (1996) Different methods are needed to propagate ignorance and variability. Reliab Eng Syst Safe 54:133–144CrossRefGoogle Scholar
  54. Fischhoff B, Slovic P, Lichtenstein S (1982) Lay foibles and expert fables in judgments about risk. Am Stat 36:240–255CrossRefGoogle Scholar
  55. Fisher L (2009) The perfect swarm: the science of complexity in everyday life. Basic Books, New YorkGoogle Scholar
  56. Garthwaite PH, Kadane JB, O’Hagan A (2005) Statistical methods for eliciting probability distributions. J Am Stat Assoc 100:680–700CrossRefGoogle Scholar
  57. Genest C, McConway KJ (1990) Allocating the weights in the linear opinion pool. J Forecasting 9:53–73CrossRefGoogle Scholar
  58. Genest C, Zidek JV (1986) Combining probability distributions: a critique and an annotated bibliography. Stat Sci 1:114–148CrossRefGoogle Scholar
  59. Gigerenzer G (1999) Simple heuristics that make us smart. Oxford University Press, New YorkGoogle Scholar
  60. Gigerenzer G (2002) Calculated risks: how to know when the numbers deceive you. Simon and Schuster, New YorkGoogle Scholar
  61. Gigerenzer G (2008) Rationality for mortals: how people cope with uncertainty. Oxford University Press, New YorkGoogle Scholar
  62. Gigerenzer G, Hoffrage U (1995) How to improve Bayesian reasoning without instruction: frequency formats. Psychol Rev 102:684–704CrossRefGoogle Scholar
  63. Gilovich T, Griffin D, Kahneman D (eds) (2002) Heuristics and biases: the psychology of intuitive judgement. Cambridge University Press, CambridgeGoogle Scholar
  64. Grier JW, Elder JB, Gramlich FJ et al (1993) The bald eagle in the northern United States. Bird Conserv 1:41–66Google Scholar
  65. Griffiths SP, Kuhnert PM, Venables WN, Blaber SJM (2007) Estimating abundance of pelagic fishes using gillnet catch data in data-limited fisheries: a Bayesian approach. Can J Fish Aquat Sci 64:1019–1033CrossRefGoogle Scholar
  66. Helander B, Marquiss M, Bowerman W (eds) (2003) Sea Eagle 2000. In: Proceedings from an International Conference at Bjökö, Sweden, 13–17 September 2000. Swedish Society for Nature Conservation, Stockholm, pp 129–132Google Scholar
  67. Helander B, Stjernberg,T (2003) Action plan for the conservation of white-tailed Sea Eagle (Haliaeetus albicilla). The Convention on the Conservation of European Wildlife and Natural Habitats, StrasbourgGoogle Scholar
  68. Hertwig R, Gigerenzer G (1999) The ‘conjunction fallacy’ revisited: how intelligent inferences look like reasoning errors. J Behav Dec Making 12:275–305CrossRefGoogle Scholar
  69. Hofer E (1996) When to separate uncertainties and when not to separate. Reliab Eng Syst Safe 54:113–118CrossRefGoogle Scholar
  70. Hoffman FO, Kaplan S (1999) Beyond the domain of direct observation: how to specify a probability distribution that represents the “state of knowledge” about uncertain inputs. Risk Anal 19:131–134Google Scholar
  71. Hoffman RR (1998) How can expertise be defined? Implications of research from cognitive psychology. In: Williams R, Faulkner W, Fleck J (eds) Exploring expertise. Macmillan, New York, pp 81–100Google Scholar
  72. Hogarth RM (1977) Methods for aggregating opinions. In: Jungermann H, DeZeeuw G (eds) Decision making and change in human affairs. Reidel, Dordrecht, pp 231–255CrossRefGoogle Scholar
  73. Hogarth RM (1987) Judgment and choice: the psychology of decision. Wiley, New YorkGoogle Scholar
  74. Hogarth RM (2001) Educating intuition. The University of Chicago Press, ChicagoGoogle Scholar
  75. Hokstad P, Oien K, Reinertsen R (1998) Recommendations on the use of expert judgment in safety and reliability engineering studies: two offshore case studies. Reliab Eng Syst Safe 61:65–76CrossRefGoogle Scholar
  76. Hora SC (1992) Acquisition of expert judgment: examples from risk assessment. J Energy Dev 118:136–148Google Scholar
  77. Hora SC (2004) Probability judgments for continuous quantities: linear combinations and calibration. Manage Sci 50:597–604CrossRefGoogle Scholar
  78. Jacobs RA (1995) Methods for combining experts probability assessments. Neural Comput 7:867–888PubMedCrossRefGoogle Scholar
  79. Johnson CJ, Gillingham MP (2004) Mapping uncertainty: sensitivity of wildlife habitat ratings to expert opinion. J Appl Ecol 41:1032–1041CrossRefGoogle Scholar
  80. Kadane JB, Dickey JM, Winkler RL et al (1980) Interactive elicitation of opinion for a normal linear model. J Am Stat Assoc 75:845–854CrossRefGoogle Scholar
  81. Kadane JB, Wolfson LJ (1998) Experiences in elicitation. J Roy Stat Soc D-Sta 47:3–19CrossRefGoogle Scholar
  82. Kahneman D (1991) Judgment and decision making: a personal view. Psychol Sci 2:142–145CrossRefGoogle Scholar
  83. Kahneman D, Tversky A (eds) (1982) Judgment under uncertainty: heuristics and biases. Cambridge University Press, CambridgeGoogle Scholar
  84. Kangas AS, Kangas J (2004) Probability, possibility and evidence: approaches to consider risk and uncertainty in forestry decision analysis. For Policy Econ 6:169–188CrossRefGoogle Scholar
  85. Kaplan S (1992) ‘Expert information’ versus ‘expert opinions’. Another approach to the problem of eliciting/combining/using expert knowledge in PRA. Reliab Eng Syst Safe 35:61–72Google Scholar
  86. Kardes FR (2006) When should consumers and managers trust their intuition? J Consum Psychol 16:20–24CrossRefGoogle Scholar
  87. Keeney RL, von Winterfeldt D (1991) Eliciting probabilities from experts in complex technical problems. IEEE Trans Eng Manage 38:191–201CrossRefGoogle Scholar
  88. Keith DW (1996) When is it appropriate to combine expert judgments? Climatic Change 33:139–143CrossRefGoogle Scholar
  89. Kidd A, Welbank M (1984) Knowledge acquisition. In: Fox J (ed) Infotech state of the art report on expert systems. Pergamon, LondonGoogle Scholar
  90. Kuhnert PM, Martin TG, Griffiths SP (2010) A guide to eliciting and using expert knowledge in Bayesian ecological models. Ecol Lett 7:900–914CrossRefGoogle Scholar
  91. Kunda Z (1990) The case for motivated reasoning. Psychol Bull 108:480–498PubMedCrossRefGoogle Scholar
  92. Kynn M (2004) Eliciting expert knowledge for Bayesian logistic regression in species habitat modelling. Department of statistics, Queensland University of Technology, BrisbaneGoogle Scholar
  93. Kynn M (2008) The ‘heuristics and biases’ bias in expert elicitation. J Roy Stat Soc A-Sta 171:239–264Google Scholar
  94. Larkin J, McDermott J, Simon DP, Simon, HA (1980) Expert and novice performance in solving physics problems. Science 208:1335–1342PubMedCrossRefGoogle Scholar
  95. Lock A (1987) Integrating group judgments in subjective forecasts. In: Wright G, Ayton P (eds) Judgmental forecasting. Wiley, Chichester, pp 109–128Google Scholar
  96. Low-Choy S, O’Leary R, Mengersen K (2009) Elicitation by design in ecology: using expert opinion to inform priors for Bayesian statistical models. Ecology 90:265–277CrossRefGoogle Scholar
  97. Ludwig D, Mangel M, Haddad B (2001) Ecology, conservation, and public policy. Annu Rev Ecol Syst 32:481–517CrossRefGoogle Scholar
  98. MacMillan DC, Marshall K (2006) The Delphi process: an expert-based approach to ecological modelling in data-poor environments. Anim Conserv 9:11–19CrossRefGoogle Scholar
  99. MacNally, R (2007) Consensus weightings of evidence for inferring breeding success in broad-scale bird studies. Austral Ecol 32:479–484CrossRefGoogle Scholar
  100. Marsh H, Dennis A, Hines H et al (2007) Optimizing allocation of management resources for wildlife. Conserv Biol 21:387–399PubMedCrossRefGoogle Scholar
  101. Martin TG, Kuhnert PM, Mengersen K, Possingham, HP (2005) The power of expert opinion in ecological models using Bayesian methods: impact of grazing on birds. Ecol Appl 15:266–280CrossRefGoogle Scholar
  102. McCoy ED, Sutton PE, Mushinsky HR (1999) The role of guesswork in conserving the threatened sand skink. Conserv Biol 13:190–194CrossRefGoogle Scholar
  103. Meyer M, Booker J (1991) Eliciting and analyzing expert judgment: a practical guide. Academic Press, New YorkGoogle Scholar
  104. Morgan MG, Henrion M (1990) Uncertainty: a guide to dealing with uncertainty in quantitative risk and policy analysis. Cambridge University Press, New YorkGoogle Scholar
  105. Morris PA (1974) Decision analysis expert use. Manage Sci 20:1233–1241CrossRefGoogle Scholar
  106. Morris PA (1977) Combining expert judgments: a Bayesian approach. Manage Sci 23:679–693CrossRefGoogle Scholar
  107. Moss R, Schneider, SH (2000) Uncertainties in the IPCC TAR: Recommendations to lead authors for more consistent assessment and reporting. In: Pachauri R, Taniguchi R, Tanaka K (eds) Guidance papers on the cross cutting issues of the third assessment report of the IPCC. World Meteorological Organisation, Geneva, pp 33–51Google Scholar
  108. Murphy AH, Winkler RL (1984) Probability forecasting in meteorology. J Am Stat Assoc 79:489–500CrossRefGoogle Scholar
  109. O’Hagan A (1998) Eliciting expert beliefs in substantial practical applications. J Roy Stat Soc D–Statistics 47:21–35Google Scholar
  110. O’Hagan A, Buck CE, Daneshkhah AR et al (2006). Uncertain judgments: eliciting expert probabilities. John Wiley, West SussexCrossRefGoogle Scholar
  111. O’Neill SJ, Osborn TJ, Hulme M et al (2008) Using expert knowledge to assess uncertainties in future polar bear populations under climate change. J Appl Ecol 45:1649–1659CrossRefGoogle Scholar
  112. Otway H, von Winterfeldt D (1992) Expert judgment in risk analysis and management: process, context, and pitfalls. Risk Anal 12:83–93PubMedCrossRefGoogle Scholar
  113. Pate-Cornell ME (1996) Uncertainties in risk analysis: six levels of treatment. Reliab Eng Syst Safe 54:95–111CrossRefGoogle Scholar
  114. Payne S (1951) The art of asking questions. Princeton University Press, PrincetonGoogle Scholar
  115. Pearce JL, Cherry K, Drielsma M et al (2001) Incorporating expert opinion and fine-scale vegetation mapping into statistical models of faunal distribution. J Appl Ecol 38:412–424CrossRefGoogle Scholar
  116. Pellikka J, Kuikka S, Lindén H, Varis O (2005) The role of game management in wildlife populations: uncertainty analysis of expert knowledge. Eur J  Wildlife Res 51:48–59CrossRefGoogle Scholar
  117. Peterson CR, Beach LF (1967) Man as an intuitive statistician. Psychol Bull 68:29–46PubMedCrossRefGoogle Scholar
  118. Price PC (1998) Effects of a relative-frequency elicitation question on likelihood judgment accuracy: the case of external correspondence. Organ Behav Hum Dec 76:277–297CrossRefGoogle Scholar
  119. Reading RP, Clark TW, Seebeck JH, Pearce J (1996) Habitat suitability index model for the eastern barred bandicoot, Perameles gunnii. Wildlife Res 23:221–235CrossRefGoogle Scholar
  120. Regan HM, Colyvan M, Burgman MA (2002) A taxonomy and treatment of uncertainty for ecology and conservation biology. Ecol Appl 12:618–628CrossRefGoogle Scholar
  121. Renooij S (2001) Probability elicitation for belief networks: issues to consider. Knowl Eng Rev 16:255–269CrossRefGoogle Scholar
  122. Richman HB, Gobet F, Staszewski JJ, Simon HA (1995) Simulation of expert memory using EPAM IV. Psychol Rev 102:305–333PubMedCrossRefGoogle Scholar
  123. Roloff GJ, Kernohan BJ (1999) Evaluating reliability of habitat suitability index models. Wildlife Soc Bull 27:973–985Google Scholar
  124. Rosqvist T, Tuominen R (2004) Qualification of formal safety assessment: an exploratory study. Safety Sci 42:99–120CrossRefGoogle Scholar
  125. Rothlisberger JD, Lodge DM, Cooke RM, Finnoff DC (2010) Future declines of the binational Laurentian Great Lakes fisheries: the importance of environmental and cultural change. Front Ecol Environ 8:239–244CrossRefGoogle Scholar
  126. Saati TL (1980) The analytic hierarchy process. New York, McGraw-HillGoogle Scholar
  127. Sanderson EW, Redford KH, Chetkiewicz CLB et al (2002) Planning to save a species: the jaguar as a model. Conserv Biol 16:58–72CrossRefGoogle Scholar
  128. Seaver DA (1978) Assessing probability with multiple individuals: group interaction versus mathematical aggregation. Social Science Research Institute, University of Southern California, Los Angeles. Report# SSRI-78-3Google Scholar
  129. Shanteau J (1992) Competence in experts: the role of task characteristics. Organ Behav Hum Dec 53:252–266CrossRefGoogle Scholar
  130. Shanteau J, Stewart TR (1992) Why study expert decision-making: some historical perspectives and comments. Organ Behav Hum Dec 53:95–106CrossRefGoogle Scholar
  131. Shephard GG, Kirkwood CW (1994) Managing the judgmental probability elicitation process: a case study of analyst/manager interaction. IEEE Trans Eng Manage 41:414–425CrossRefGoogle Scholar
  132. Shrader-Frechette K (1996) Value judgments in verifying and validating risk assessment models. In: Cothern CR (ed) Handbook for environmental risk decision making: values, perception and ethics. CRC Lewis Publishers, Boca Raton, pp 291–309Google Scholar
  133. Slottje P, van der Sluijs JP, Knol AB (2008) Expert elicitation: methodological suggestions for its use in environmental health impact assessments. RIVM, Copernicus Institute for Sustainable Development and Innovation., Bilthoven. Report 630004001/2008Google Scholar
  134. Slovic P (1999) Trust, emotion, sex, politics and science: surveying the risk-assessment battlefield. Risk Anal 19:689–701PubMedGoogle Scholar
  135. Slovic P, Finucane ML, Peters E, MacGregor DG (2004) Risk as analysis and risk as feelings: some thoughts about affect, reason, risk, and rationality. Risk Anal 24:311–322PubMedCrossRefGoogle Scholar
  136. Slovic P, Monahan J, MacGregor DG (2000) Violence risk assessment and risk communication: the effects of using actual cases, providing instruction, and employing probability versus frequency formats. Law Human Behav 24:271–296CrossRefGoogle Scholar
  137. Speirs-Bridge A, Fidler F, McBride M et al (2010) Reducing overconfidence in the interval judgments of experts. Risk Anal 30:512–523PubMedCrossRefGoogle Scholar
  138. Spetzler CS, Stael Von Holstein CAS (1975) Probability encoding in decision analysis. Manage Sci 22:340–358CrossRefGoogle Scholar
  139. Stern PC, Fineberg HV (eds) (1996) Understanding risk: informing decisions in a democratic society. National Academies Press, WashingtonGoogle Scholar
  140. Sutherland WJ (2006) Predicting the ecological consequences of environmental change: a review of the methods. J Appl Ecol 43:599–616CrossRefGoogle Scholar
  141. Sutherland WJ, Bailey MJ, Bainbridge IP et al (2008) Future novel threats and opportunities facing UK biodiversity identified by horizon scanning. J Appl Ecol 45:821–833CrossRefGoogle Scholar
  142. Sutherland WJ, Pullin AS, Dolman PM, Knight TM (2004) The need for evidence-based conservation. Trends Ecol Evol 19:305–308PubMedCrossRefGoogle Scholar
  143. Tallman I, Leik RK, Gray LN, Stafford MC (1993) A theory of problem-solving behavior. Soc Psychol Quart 56:157–177CrossRefGoogle Scholar
  144. Tavana M, Kennedy DT, Mohebbi B (1997) An applied study using the analytic hierarchy process to translate common verbal phrases to numerical probabilities. J Behav Dec Making 10:133–150CrossRefGoogle Scholar
  145. Teck SJ, Halpern BS, Kappel CV et al (2010) Using expert judgment to estimate marine ecosystem vulnerability in the California Current. Ecol Appl 20:1402–1416PubMedCrossRefGoogle Scholar
  146. Tversky A, Kahneman D (1974) Judgment under uncertainty: heuristics and biases. Science 185:1124–1131PubMedCrossRefGoogle Scholar
  147. Tversky A, Kahneman D (1983) Extensional versus intuitive reasoning: the conjunction fallacy in probability judgment. Psychol Rev 90:293–315CrossRefGoogle Scholar
  148. Tversky A, Koehler DJ (1994) Support theory: a nonextensional representation of subjective-probability. Psychol Rev 101:547–567CrossRefGoogle Scholar
  149. van der Gaag LC, Renooij S, Witteman CLM et al (1999) How to elicit many probabilities. In: Laskey KB, Prade H (eds) Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence, Stockholm, July–August 1999. Morgan Kaufmann, San FranciscoGoogle Scholar
  150. van der Gaag LC, Renooij S, Witteman CLM et al (2002) Probabilities for a probabilistic network: a case study in oesophageal cancer. Artif Intell Med 25:123–148PubMedCrossRefGoogle Scholar
  151. van Steen JFJ (1992) A perspective on structured expert judgment. J Hazard Mater 29:365–385CrossRefGoogle Scholar
  152. von Winterfeldt D, Edwards W (1986) Decision analysis and behavioral research. Cambridge University Press, CambridgeGoogle Scholar
  153. Walls L, Quigley J (2001) Building prior distributions to support Bayesian reliability growth modelling using expert judgement. Reliab Eng Syst Safe 74:117–128CrossRefGoogle Scholar
  154. Wallsten TS, Budescu DV (1995) A review of human linguistic probability processing: general principles and empirical evidence. Knowl Eng Rev 10:43–62CrossRefGoogle Scholar
  155. Wallsten TS, Budescu DV, Erev I, Diederich A (1997) Evaluating and combining subjective probability estimates. J Behav Dec Making 10:243–268CrossRefGoogle Scholar
  156. Wallsten TS, Budescu DV, Rapoport A et al (1986) Measuring the vague meanings of probability terms. J Exp Psychol Gen 115:348–365CrossRefGoogle Scholar
  157. Whitfield DP, Ruddock M, Bullman R (2008) Expert opinion as a tool for quantifying bird tolerance to human disturbance. Biol Conserv 141:2708–2717CrossRefGoogle Scholar
  158. Wilson AG (1994) Cognitive factors affecting subjective probability assessment. Duke University, Institute of Statistics and Decision Sciences, Durham. Report #94–02Google Scholar
  159. Windschitl PD, Wells GL (1996) Measuring psychological uncertainty: verbal versus numeric methods. J Exp Psychol-Appl, 2:343–364CrossRefGoogle Scholar
  160. Winkler RL, Makridakis S (1983) The combination of forecasts. J Roy Stat Soc A-Sta 146:150–157CrossRefGoogle Scholar
  161. Yamada K, Elith J, McCarthy M, Zerger A (2003) Eliciting and integrating expert knowledge for wildlife habitat modelling. Ecol Model 165:251–264CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.School of Botany, University of MelbourneParkvilleAustralia
  2. 2.Australian Centre of Excellence for Risk AnalysisSchool of Botany, University of MelbourneParkvilleAustralia

Personalised recommendations