, Volume 20, Issue 2, pp 331–339 | Cite as

The Power and the Pitfalls of Large-scale, Unreplicated Natural Experiments

  • Shanta C. Barley
  • Jessica J. Meeuwig


Large-scale, unreplicated natural experiments (LUNEs) have a unique power to test hypotheses at ecologically realistic scales and have delivered insights of great power into cosmology, evolution and geology. Yet, LUNEs are relatively rare in the field of ecology and continue to meet resistance due to their lack of replication. However, in the vast majority of cases, large-scale experiments cannot be replicated for practical and ethical reasons. Here, we make the case that LUNEs have had a disproportionately positive effect on conservation policy and are a crucial next step in the extrapolation of our understanding of ecological processes from small-scale experiments to relevant scales, particularly in the context of the current “replication crisis” affecting many sciences. Greater inclusion of LUNEs in mainstream ecology will help humanity to solve global problems as human transformation of the planet accelerates in coming decades.


ecosystem scale ecological processes large-scale experiment natural experiment pseudo-experiment replication 



We would like to thank the University of Western Australia, Perth, which partly funded this research via an International Postgraduate Research Scholarship to SB.


  1. Allison GW, Lubchenco J, Carr MH. 1998. Marine reserves are necessary but not sufficient for marine conservation. Ecol Appl 8:79–92.CrossRefGoogle Scholar
  2. Atwood TB, Connolly RM, Ritchie EG, Lovelock CE, Heithaus MR, Hays GC, Fourqurean JW, Macreadie PI. 2015. Predators help protect carbon stocks in blue carbon ecosystems. Nat Clim Chang 5:1038–45.CrossRefGoogle Scholar
  3. Bell JD, Westoby M. 1986. Variation in seagrass height and density over a wide spatial scale: effects on common fish and decapods. J Exp Mar Biol Ecol 104:275–95.CrossRefGoogle Scholar
  4. Bence JR, Stewart-Oaten A, Schroeter SC. 1996. Estimating the size of an effect from a before-after-control impact paired series design. In: Russell J, Schmitt CWO, Eds. Detecting ecological impacts: concepts and applications in coastal habitats. San Diego: Academic Press.Google Scholar
  5. Bennett S, Bellwood DR. 2011. Latitudinal variation in macroalgal consumption by fishes on the Great Barrier Reef. Mar Ecol Prog Ser 426:241–52.CrossRefGoogle Scholar
  6. Brown GP, Phillips BL, Shine R. 2011. The ecological impact of invasive cane toads on tropical snakes: field data do not support laboratory-based predictions. Ecology 92:422–31.CrossRefPubMedGoogle Scholar
  7. Carpenter RC. 1990. Mass mortality of Diadema antillarum. I. Long-term effects on sea urchin population-dynamics and coral reef algal communities. Mar Biol 104:67–77.CrossRefGoogle Scholar
  8. Carpenter SR, Chisholm SW, Krebs CJ, Schindler DW, Wright RF. 1995. Ecosystem experiments. Science 269:324–7.CrossRefPubMedGoogle Scholar
  9. Carpenter SR. 1989. Replication and treatment strength in whole-lake experiments. Ecology 70:453–63.CrossRefGoogle Scholar
  10. Chave J. 2013. The problem of pattern and scale in ecology: what have we learned in 20 years? Ecol Lett 16:4–16.CrossRefPubMedGoogle Scholar
  11. Chown SL, Gaston KJ, Robinson D. 2004. Macrophysiology: large-scale patterns in physiological traits and their ecological implications. Funct Ecol 18:159–67.CrossRefGoogle Scholar
  12. Clark JS, Gelfand AE. 2006. A future for models and data in environmental science. Trends Ecol Evol 21:375–80.CrossRefPubMedGoogle Scholar
  13. Cottenie K, De Meester L. 2003. Comment to Oksanen (2001): reconciling Oksanen (2001) and Hurlbert (1984). Oikos 100:394–6.CrossRefGoogle Scholar
  14. Darimont CT, Fox CH, Bryan HM, Reimchen TE. 2015. The unique ecology of human predators. Science 349:858–60.CrossRefPubMedGoogle Scholar
  15. Edgar G, Stuart-Smith R, Willis T, Kininmonth S, Baker S, Banks S, Barrett N, Becerro M, Bernard A, Berkhout J, Buxton C, Campbell S, Cooper A, Davey M, Edgar S, Försterra G, Galván D, Irigoyen A, Kushner D, Moura R, Parnell P, Shears N, Soler G, Strain E, Thomson R. 2014. Global conservation outcomes depend on marine protected areas with five key features. Nature 506:216–20.CrossRefPubMedGoogle Scholar
  16. Ehrlich PR, Ehrlich AH. 2013. Can a collapse of global civilization be avoided? Proc Biol Sci 280:1–9.Google Scholar
  17. Estes JA, Palmisano JF. 1974. Sea otters: their role in structuring nearshore communities. Science 185:1058–60.CrossRefPubMedGoogle Scholar
  18. Estes JA, Terborgh J, Brashares JS, Power ME, Berger J, Bond WJ, Carpenter SR, Essington TE, Holt RD, Jackson JBC, Marquis RJ, Oksanen L, Oksanen T, Paine RT, Pikitch EK, Ripple WJ, Sandin SA, Scheffer M, Schoener TW, Shurin JB, Sinclair ARE, Soulé ME, Virtanen R, Wardle DA. 2011. Trophic downgrading of planet Earth. Science 333:301–6.CrossRefPubMedGoogle Scholar
  19. Ewers R, Didham R, Fahrig L, Ferraz G, Hector A, Holt R, Kapos V, Reynolds G, Sinun W, Snaddon J, Turner E. 2011. A large-scale forest fragmentation experiment: the stability of altered forest ecosystems project. Philos Trans Royal Soc B: Biol Sci 366:3292–302.CrossRefGoogle Scholar
  20. Fisher RA. 1926. The arrangement of field experiments. J Minist Agri 33:503–13.Google Scholar
  21. Folt CL, Nislow KH, Power ME. 1998. Implications of temporal and spatial scale for Atlantic salmon (Salmo salar) research. Can J Fish Aquat Sci 55:9–21.CrossRefGoogle Scholar
  22. Fraser DF, Gilliam JF. 1987. Feeding under predation hazard: response of the guppy and Hart’s rivulus from sites with contrasting predation hazard. Behav Ecol Sociobiol 21:203–9.CrossRefGoogle Scholar
  23. Gabric AJ, Cropp R, McTainsh G, Butler H, Johnston BM, O’Loingsigh T, Van Tran D. 2015. Tasman Sea biological response to dust storm events during the austral spring of 2009. Marine and Freshwater Research 9999.Google Scholar
  24. Gadgil M, Bossert WH. 1970. Life historical consequences of natural selection. Am Naturalist 104:1–24.CrossRefGoogle Scholar
  25. Grossman J, Mackenzie FJ. 2005. The randomized controlled trial: gold standard, or merely standard? Perspect Biol Med 48:516–34.CrossRefPubMedGoogle Scholar
  26. Grubbs RD, Carlson JK, Romine JG, Curtis TH, McElroy WD, McCandless CT, Musick CFC& JA. 2016. Critical assessment and ramifications of a purported marine trophic cascade. Sci Rep:20970.Google Scholar
  27. Hale M, Rivkin RB. 2007. Interpreting the results of oceanic mesoscale enrichment experiments: caveats and lessons from limnology and coastal ecology. Limnol Oceanogr 52:912–16.CrossRefGoogle Scholar
  28. Hargrove WW, Pickering J. 1992. Pseudoreplication: a sine qua non for regional ecology. Landsc Ecol 6:251–8.CrossRefGoogle Scholar
  29. Hildrew AG, Woodward G, Winterbottom JH, Orton S. 2004. Strong density dependence in a predatory insect: large-scale experiments in a stream. J Anim Ecol 73:448–58.CrossRefGoogle Scholar
  30. Hillerislambers J, Ettinger AK, Ford KR, Haak DC, Horwith M, Miner BE, Rogers HS, Sheldon KS, Tewksbury JJ, Waters SM, Yang S. 2013. Accidental experiments: ecological and evolutionary insights and opportunities derived from global change. Oikos 122:1649–61.CrossRefGoogle Scholar
  31. Hixon MA, Anderson TW, Buch KL, Johnson DW, Mcleod JB, Stallings CD. 2012. Density dependence and population regulation in marine fish: a large-scale, long-term field manipulation. Ecol Monogr 82:467–89.CrossRefGoogle Scholar
  32. Högberg P, Nordgren A, Buchmann N, Taylor AF, Ekblad A, Högberg MN, Nyberg G, Ottosson-Löfvenius M, Read DJ. 2001. Large-scale forest girdling shows that current photosynthesis drives soil respiration. Nature 411:789–92.CrossRefPubMedGoogle Scholar
  33. Hüffmeiera J, Mazeia J, Schultze T. 2016. Reconceptualizing replication as a sequence of different studies: a replication typology. J Exp Soc Psychol. doi: 10.1016/j.jesp.2015.09.009.Google Scholar
  34. Hurlbert SH. 1984. Pseudoreplication and the design of ecological field experiments. Ecol Monogr 54:187.CrossRefGoogle Scholar
  35. Hurlbert SH. 2004. On misinterpretations of pseudoreplication and related matters: a reply to Oksanen. Oikos 104:591–7.CrossRefGoogle Scholar
  36. Ioannidis JPA. 2005. Why most published research findings are false. PLoS Med 2:0696–701.Google Scholar
  37. Johnson DH. 2002. The importance of replication in wildlife research. J Wildl Manag 66:919–32.CrossRefGoogle Scholar
  38. Johnson DH. 2006. The many faces of replication. In: Crop science. Vol. 46. pp 2486–91.Google Scholar
  39. Kates RW, Clark WC. 1996. Environmental Surprise: expecting the Unexpected? Environ: Sci Policy Sustain Dev 38:6–34.CrossRefGoogle Scholar
  40. Kessler JD. 2011. A persistent oxygen anomaly reveals the fate of spilled methane in the deep gulf of Mexico. Science 331:312.CrossRefPubMedGoogle Scholar
  41. Knowlton N. 1992. Thresholds and multiple stable states in coral reef community dynamics. Am Zool 32:674–82.
  42. Kreyling J, Jentsch A, Beier C. 2014. Beyond realism in climate change experiments: gradient approaches identify thresholds and tipping points. Ecol Lett 17:125–8.
  43. Lennon JT. 2011. Replication, lies and lesser-known truths regarding experimental design in environmental microbiology. Environ Microbiol 13:1383–6.CrossRefPubMedGoogle Scholar
  44. Lester SE, Halpern BS. 2008. Biological responses in marine no-take reserves versus partially protected areas. Mar Ecol Prog Ser 367:49–56.CrossRefGoogle Scholar
  45. Leuzinger S, Luo Y, Beier C, Dieleman W, Vicca S, Körner C. 2011. Do global change experiments overestimate impacts on terrestrial ecosystems? Trends Ecol Evol 26:236–41.CrossRefPubMedGoogle Scholar
  46. Lezama F, Baeza S, Altesor A, Cesa A, Chaneton EJ, Paruelo JM. 2014. Variation of grazing-induced vegetation changes across a large-scale productivity gradient. J Veg Sci 25:8–21.CrossRefGoogle Scholar
  47. Likens GE, Bormann FH, Johnson NM, Fisher DW, Pierce RS. 1970. Effects of forest cutting and herbicide treatment on nutrient budgets in the hubbard brook watershed-ecosystem. Ecol Monogr 40:23–47.CrossRefGoogle Scholar
  48. Madin EMP, Gaines SD, Warner RR. 2010. Field evidence for pervasive indirect effects of fishing on prey foraging behavior. Ecology 91:3563–71.CrossRefPubMedGoogle Scholar
  49. McArdle BH. 1996. Levels of evidence in studies of competition, predation, and disease. N. Z. J Ecol 20:7–15.Google Scholar
  50. McNutt M. 2014. Reproducibility. Science (New York) 343:229.
  51. Miao S, Carstenn S. 2006. A new direction for large-scale experimental design and analysis. Front Ecol Environ 4:227.CrossRefGoogle Scholar
  52. Micheli F. 1999. Eutrophication, fisheries, and consumer-resource dynamics in marine pelagic ecosystems. Science (New York, NY) 285:1396–8Google Scholar
  53. Morris SC. 1989. Burgess Shale faunas and the Cambrian explosion. Science 246:339–46.CrossRefPubMedGoogle Scholar
  54. Morrisette PM. 1989. The Evolution of Policy Responses to Stratospheric Ozone Depletion. Nat Res J 29:793–820.Google Scholar
  55. Moss R, Watson A, Parr R. 1996. Experimental prevention of a population cycle in Red Grouse. Ecology 77:1512–30.CrossRefGoogle Scholar
  56. Myers RA, Baum JK, Shepherd TD, Powers SP, Peterson CH. 2007. Cascading effects of the loss of apex predatory sharks from a coastal ocean. Science 315:1846–50.CrossRefPubMedGoogle Scholar
  57. Naeem S, Thompson LJ, Lawler SP, Lawton JH, Woodfin RM. 1994. Declining biodiversity can alter the performance of ecosystems. Nature 368:734–7.CrossRefGoogle Scholar
  58. Nosek BA, Alter G, Banks GC, Borsboom D, Bowman SD, Breckler SJ, Buck S, Chambers CD, Chin G, Christensen G, Contestabile M, Dafoe A, Eich E, Freese J, Glennerster R, Goroff D, Green DP, Hesse B, Humphreys M, Ishiyama J, Karlan D, Kraut A, Lupia A, Mabry P, Madon TA, Malhotra N, Mayo-Wilson E, McNutt M, Miguel E, Paluck EL, Simonsohn U, Soderberg C, Spellman BA, Turitto J, VandenBos G, Vazire S, Wagenmakers EJ, Wilson R, Yarkoni T. 2015. Promoting an open research culture. Science 348:1422–5.CrossRefPubMedPubMedCentralGoogle Scholar
  59. O’Dowd DJ, Green PT, Lake PS. 2003. Invasional ‘meltdown’ on an oceanic island. Ecol Lett 6:812–17.CrossRefGoogle Scholar
  60. Pace ML, Cole JJ, Carpenter SR. 1998. Trophic cascades and compensation: differential responses of microzooplankton in whole-lake experiments. Ecology 79:138–52.CrossRefGoogle Scholar
  61. Pauly D, Christensen V, Dalsgaard J, Froese R, Torres F. 1998. Fishing down marine food webs. Science (New York, NY) 279:860–3.
  62. Power ME, Dietrich WE, Sullivan KO. 1998. Experiment, observation, and inference in river and watershed investigations. In: Bernardo J, Resetarits WJ, Eds. Experimental ecology: issues and perspectives. Oxford: Oxford University Press. p 113–32.Google Scholar
  63. Prevedello J, Dickman C, Vieira M, Vieira E. 2013. Population responses of small mammals to food supply and predators: a global meta-analysis. J Anim Ecol 82:927–36.CrossRefPubMedGoogle Scholar
  64. Raffaelli D, Moller H. 1999. Manipulative field experiments in animal ecology: do they promise more than they can deliver? Adv Ecol Res 30:299–338.CrossRefGoogle Scholar
  65. Ritsema J, Van Heijst HJ. 2000. Seismic imaging of structural heterogeneity in Earth’s mantle: evidence for large-scale mantle flow. Science progress 83.Google Scholar
  66. Rogers H, HilleRisLambers J, Miller R, Tewksbury J. 2012. ‘Natural experiment’ demonstrates top-down control of spiders by birds on a landscape level. PLoS One 7:1–8.Google Scholar
  67. Rose GA, Leggett WC. 1990. The importance of scale to predator-prey spatial correlations: an example of Atlantic fishes. Ecology 71:33–43.CrossRefGoogle Scholar
  68. Rowan R, Knowlton N, Baker A, Jara J. 1997. Landscape ecology of algal symbionts creates variation in episodes of coral bleaching. Nature 388:265–9.CrossRefPubMedGoogle Scholar
  69. Ruppert JLW, Travers MJ, Smith LL, Fortin MJ, Meekan MG. 2013. Caught in the middle: combined impacts of shark removal and coral loss on the fish communities of coral reefs. PLoS One 8:e74648.CrossRefPubMedPubMedCentralGoogle Scholar
  70. Sarnelle O. 1997. Daphnia effects on microzooplankton: comparisons of enclosure and whole-lake responses. Ecology 78:913–28.Google Scholar
  71. Schindler DW, Armstrong FAJ, Holmgren SK, Brunskill GJ. 1971. Eutrophication of lake 227, experimental lakes area, northwestern Ontario, by addition of phosphate and nitrate. J Fish Res Board Can 28:1763–82.CrossRefGoogle Scholar
  72. Schindler DW, Fee EJ, Ruszczynski T. 1978. Phosphorus input and its consequences for phytoplankton standing crop and production in the experimental lakes area and in similar lakes. J Fish Res Board Can 35:190–6.CrossRefGoogle Scholar
  73. Schindler DW, Hecky RE, Findlay DL, Stainton MP, Parker BR, Paterson MJ, Beaty KG, Lyng M, Kasian SEM. 2008. Eutrophication of lakes cannot be controlled by reducing nitrogen input: results of a 37-year whole-ecosystem experiment. Proc Natl Acad Sci U. S. A. 105:11254–8.CrossRefPubMedPubMedCentralGoogle Scholar
  74. Schindler DW. 1998. Replication versus realism: the need for ecosystem-scale experiments. Ecosystems 1:323–34.CrossRefGoogle Scholar
  75. Schindler DW. 2012. The dilemma of controlling cultural eutrophication of lakes. Proc Royal Soc B: Biol Sci 279:4322–33.CrossRefGoogle Scholar
  76. Shurin JB, Borer ET, Seabloom EW, Anderson K, Blanchette CA, Broitman B, Cooper SD, Halpern BS. 2002. A cross-ecosystem comparison of the strength of trophic cascades. Ecol Lett 5:785–91.CrossRefGoogle Scholar
  77. Smith SA, Bell G, Bermingham E. 2004. Cross-cordillera exchange mediated by the panama canal increased the species richness of local freshwater fish assemblages. Proc Biol Sci 271:1889–96.CrossRefPubMedPubMedCentralGoogle Scholar
  78. Stewart-Oaten A, Bence JR, Osenberg CW. 1992. Assessing effects of unreplicated perturbations: no simple solutions. Ecology 73:1396–404.CrossRefGoogle Scholar
  79. Stewart-Oaten A, Murdoch WW, Parker KR. 1986. Environmental impact assessment: ‘pseudoreplication’ in time? Ecology 67:929–40.CrossRefGoogle Scholar
  80. Stroebe W, Strack F. 2014. The alleged crisis and the illusion of exact replication. Perspect Psychol Sci 9:59–71.CrossRefPubMedGoogle Scholar
  81. Vanni MJ, Luecke C, Kitchell JF, Allen Y, Temte J, Magnuson JJ. 1990. Effects on lower trophic levels of massive fish mortality. Nature 344:333–5.CrossRefGoogle Scholar
  82. Walsh SM, Hamilton SL, Ruttenberg BI, Donovan MK, Sandin SA. 2012. Fishing top predators indirectly affects condition and reproduction in a reef-fish community. J Fish Biol 80:519–37.CrossRefPubMedGoogle Scholar
  83. Wardle DA, Barker GM. 1997. Competition and herbivory in establishing grassland communities: implications for plant biomass, species diversity and soil microbial activity. Oikos 80:470–80.CrossRefGoogle Scholar
  84. Weimin X, Peet RK. 2011. The complexity of catastrophic wind impacts on temperate forest. In: Lupo A, Ed. Recent hurricane research - climate, dynamics, and societal impacts. InTech. pp 503–34.Google Scholar
  85. Worm B, Paine RT. 2016. Humans as a hyperkeystone species. Trends in Ecology and Evolution2.Google Scholar
  86. Zoback MD, Gorelick SM. 2012. Earthquake triggering and large-scale geologic storage of carbon dioxide. Proc Natl Acad Sci 109:10164–8.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.School of Animal Biology and the Oceans InstituteUniversity of Western AustraliaPerthAustralia

Personalised recommendations