Improving basic and translational science by accounting for litter-to-litter variation in animal models
Animals from the same litter are often more alike compared with animals from different litters. This litter-to-litter variation, or “litter effects”, can influence the results in addition to the experimental factors of interest. Furthermore, sometimes an experimental treatment can only be applied to whole litters rather than to individual offspring. An example is the valproic acid (VPA) model of autism, where VPA is administered to pregnant females thereby inducing the disease phenotype in the offspring. With this type of experiment the sample size is the number of litters and not the total number of offspring. If such experiments are not appropriately designed and analysed, the results can be severely biased as well as extremely underpowered.
A review of the VPA literature showed that only 9% (3/34) of studies correctly determined that the experimental unit (n) was the litter and therefore made valid statistical inferences. In addition, litter effects accounted for up to 61% (p <0.001) of the variation in behavioural outcomes, which was larger than the treatment effects. In addition, few studies reported using randomisation (12%) or blinding (18%), and none indicated that a sample size calculation or power analysis had been conducted.
Litter effects are common, large, and ignoring them can make replication of findings difficult and can contribute to the low rate of translating preclinical in vivo studies into successful therapies. Only a minority of studies reported using rigorous experimental methods, which is consistent with much of the preclinical in vivo literature.
- Geerts H: Of mice and men: bridging the translational disconnect in CNS drug discovery. CNS Drugs 2009,23(11):915–926. [http://dx.doi.org/10.2165/11310890-000000000-00000] CrossRef
- Haseman JK, Hogan MD: Selection of the experimental unit in teratology studies. Teratol 1975,12(2):165–171. CrossRef
- Hughes CW: Outcome of early experience studies as affected by between-litter variance. J Nutr 1979,109(4):642–645.
- Holson RR, Pearce B: Principles and pitfalls in the analysis of prenatal treatment effects in multiparous species. Neurotoxicol Teratol 1992,14(3):221–228. CrossRef
- Zorrilla EP: Multiparous species present problems (and possibilities) to developmentalists. Dev Psychobiol 1997,30(2):141–150. CrossRef
- Wainwright PE: Issues of design and analysis relating to the use of multiparous species in developmental nutritional studies. J Nutr 1998,128(3):661–663.
- Festing MFW, Altman DG: Guidelines for the design and statistical analysis of experiments using laboratory animals. ILAR J 2002,43(4):244–258. CrossRef
- Festing MFW: Principles: the need for better experimental design. Trends Pharmacol Sci 2003,24(7):341–345. CrossRef
- Festing MFW: Design and statistical methods in studies using animal models of development. ILAR J 2006, 47:5–14. CrossRef
- Casella G: Statistical Design. New York: Springer; 2008. CrossRef
- Maurissen J: Practical considerations on the design, execution and analysis of developmental neurotoxicity studies to be published in Neurotoxicology and Teratology. Neurotoxicol Teratol 2010,32(2):121–123. [http://dx.doi.org/10.1016/j.ntt.2009.09.002] CrossRef
- Lazic SE: The problem of pseudoreplication in neuroscientific studies is it affecting your analysis? BMC Neurosci 2010, 11:5. [http://dx.doi.org/10.1186/1471-2202-11-5] CrossRef
- International Conference on Harmonisation: Detection of toxicity to reproduction for medicinal products and toxicity to male fertility. 1993, S5(R2):1–23.
- OECD: Guideline for the testing of chemicals: developmental neurotoxicity study. 2007, 426:1–26. CrossRef
- Hurlbert SH: Pseudoreplication and the design of ecological field experiments. Ecol Monogr 1984,54(2):187–211. CrossRef
- Schnabel J: Neuroscience: Standard model. Nature 2008,454(7205):682–685. [http://dx.doi.org/10.1038/454682a] CrossRef
- Scott S, Kranz JE, Cole J, Lincecum JM, Thompson K, Kelly N, Bostrom A, Theodoss J, Al-Nakhala BM, Vieira FG, Ramasubbu J, Heywood JA: Design, power, and interpretation of studies in the standard murine model of ALS. Amyotroph Lateral Scler 2008, 9:4–15. [http://dx.doi.org/10.1080/17482960701856300] CrossRef
- Benatar M: Lost in translation: treatment trials in the SOD1 mouse and in human ALS. Neurobiol Dis 2007, 26:1–13. [http://dx.doi.org/10.1016/j.nbd.2006.12.015] CrossRef
- Bebarta V, Luyten D, Heard K: Emergency medicine animal research: does use of randomization and blinding affect the results? Acad Emerg Med 2003,10(6):684–687. CrossRef
- Crossley NA, Sena E, Goehler J, Horn J, van der Worp B, Bath PMW, Macleod M, Dirnagl U: Empirical evidence of bias in the design of experimental stroke studies: a metaepidemiologic approach. Stroke 2008,39(3):929–934. [http://dx.doi.org/10.1161/STROKEAHA.107.498725] CrossRef
- Kilkenny C, Parsons N, Kadyszewski E, Festing MFW, Cuthill IC, Fry D, Hutton J, Altman DG: Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PLoS One 2009,4(11):e7824. [http://dx.doi.org/10.1371/journal.pone.0007824] CrossRef
- Sena ES, van der Worp HB, Bath PMW, Howells DW, Macleod MR: Publication bias in reports of animal stroke studies leads to major overstatement of efficacy. PLoS Biol 2010,8(3):e1000344. [http://dx.doi.org/10.1371/journal.pbio.1000344] CrossRef
- Vesterinen HM, Sena ES, ffrench Constant C, Williams A, Chandran S, Macleod MR: Improving the translational hit of experimental treatments in multiple sclerosis. Mult Scler 2010,16(9):1044–1055. [http://dx.doi.org/10.1177/1352458510379612] CrossRef
- Hackam DG, Redelmeier DA: Translation of research evidence from animals to humans. JAMA 2006,296(14):1731–1732. [http://dx.doi.org/10.1001/jama.296.14.1731] CrossRef
- Dirnagl U: Bench to bedside: the quest for quality in experimental stroke research. J Cereb Blood Flow Metab 2006,26(12):1465–1478. [http://dx.doi.org/10.1038/sj.jcbfm.9600298] CrossRef
- Philip M, Benatar M, Fisher M, Savitz SI: Methodological quality of animal studies of neuroprotective agents currently in phase II/III, acute ischemic stroke trials. Stroke 2009,40(2):577–581. [http://dx.doi.org/10.1161/STROKEAHA.108.524330] CrossRef
- Bart van der Worp H, Howells DW, Sena ES, Porritt MJ, Rewell S, O’Collins V, Macleod MR: Can animal models of disease reliably inform human studies? PLoS Med 2010,7(3):e1000245. [http://dx.doi.org/10.1371/journal.pmed.1000245] CrossRef
- Shineman DW, Basi GS, Bizon JL, Colton CA, Greenberg BD, Hollister BA, Lincecum J, Leblanc GG, Lee LBH, Luo F, Morgan D, Morse I, Refolo LM, Riddell DR, Scearce-Levie K, Sweeney P, Yrjanheikki J, Fillit HM: Accelerating drug discovery for Alzheimer’s disease: best practices for preclinical animal studies. Alzheimers Res Ther 2011,3(5):28. http://dx.doi.org/10.1186/alzrt90 CrossRef
- Rooke EDM, Vesterinen HM, Sena ES, Egan KJ, Macleod MR: Dopamine agonists in animal models of Parkinson’s disease: a systematic review and meta-analysis. Parkinsonism Relat Disord 2011,17(5):313–320. [http://dx.doi.org/10.1016/j.parkreldis.2011.02.010] CrossRef
- Sarewitz D: Beware the creeping cracks of bias. Nature 2012,485(7397):149. [http://dx.doi.org/10.1038/485149a] CrossRef
- Nieuwenhuis S, Forstmann BU, Wagenmakers EJ: Erroneous analyses of interactions in neuroscience: a problem of significance. Nat Neurosci 2011,14(9):1105–1107. [http://dx.doi.org/10.1038/nn.2886] CrossRef
- Cumming G, Fidler F, Vaux DL: Error bars in experimental biology. J Cell Biol 2007, 177:7–11. CrossRef
- Lazic SE: Relating hippocampal neurogenesis to behavior: the dangers of ignoring confounding variables. Neurobiol Aging 2010,31(12):2169–2171. [http://dx.doi.org/10.1016/j.neurobiolaging.2010.04.037] CrossRef
- Lazic SE: Using causal models to distinguish between neurogenesis-dependent and -independent effects on behaviour. J R Soc Interface 2011. e-pub ahead of print 28 Sep 2011 [http://dx.doi.org/10.1098/rsif.2011.0510]
- Simmons JP, Nelson LD, Simonsohn U: False-positive psychology undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci 2011,22(11):1359–1366. [http://dx.doi.org/10.1177/0956797611417632] CrossRef
- Mullard A: Reliability of ’new drug target’ claims called into question. Nat Rev Drug Discov 2011,10(9):643–644. [http://dx.doi.org/10.1038/nrd3545] CrossRef
- Prinz F, Schlange T, Asadullah K: Believe it or not: how much can we rely on published data on potential drug targets? Nat Rev Drug Discov 2011,10(9):712. [http://dx.doi.org/10.1038/nrd3439-c1] CrossRef
- Ledford H: Drug candidates derailed in case of mistaken identity. Nature 2012,483(7391):519. [http://dx.doi.org/10.1038/483519a] CrossRef
- Begley CG, Ellis LM: Drug development: Raise standards for preclinical cancer research. Nature 2012,483(7391):531–533. [http://dx.doi.org/10.1038/483531a] CrossRef
- Rinaldi T, Silberberg G, Markram H: Hyperconnectivity of local neocortical microcircuitry induced by prenatal exposure to valproic acid. Cereb Cortex 2008,18(4):763–770. [http://dx.doi.org/10.1093/cercor/bhm117] CrossRef
- Mehta MV, Gandal MJ, Siegel SJ: mGluR5-antagonist mediated reversal of elevated stereotyped, repetitive behaviors in the VPA model of autism. PLoS One 2011,6(10):e26077. [http://dx.doi.org/10.1371/journal.pone.0026077] CrossRef
- Zuur AF, Ieno EN, Walker NJ, Saveliev AA, Smith GM: Mixed Effects Models and Extensions in Ecology with R. New York: Springer; 2009. CrossRef
- Gelman A, Hill J: Data Analysis using Regression and Multilevel/Hierarchical Models. Cambridge, UK: Cambridge University Press; 2007.
- Stodgell CJ, Ingram JL, O’Bara M, Tisdale BK, Nau H, Rodier PM: Induction of the homeotic gene Hoxa1 through valproic acid’s teratogenic mechanism of action. Neurotoxicol Teratol 2006,28(5):617–624. [http://dx.doi.org/10.1016/j.ntt.2006.08.004] CrossRef
- Kuwagata M, Ogawa T, Shioda S, Nagata T: Observation of fetal brain in a rat valproate-induced autism model: a developmental neurotoxicity study. Int J Dev Neurosci 2009,27(4):399–405. [http://dx.doi.org/10.1016/j.ijdevneu.2009.01.006] CrossRef
- Murawski NJ, Brown KL, Stanton ME: Interstimulus interval (ISI) discrimination of the conditioned eyeblink response in a rodent model of autism. Behav Brain Res 2009,196(2):297–303. [http://dx.doi.org/10.1016/j.bbr.2008.09.020] CrossRef
- Flood ZC, Engel DLJ, Simon CC, Negherbon KR, Murphy LJ, Tamavimok W, Anderson GM, Janusonis S: Brain growth trajectories in mouse strains with central and peripheral serotonin differences: relevance to autism models. Neurosci 2012, 210:286–295. [http://dx.doi.org/10.1016/j.neuroscience.2012.03.010] CrossRef
- Peers IS, Ceuppens PR, Harbron C: In search of preclinical robustness. Nat Rev Drug Discov 2012,11(10):733–734. [http://dx.doi.org/10.1038/nrd3849] CrossRef
- Landis SC, Amara SG, Asadullah K, Austin CP, Blumenstein R, Bradley EW, Crystal RG, Darnell RB, Ferrante RJ, Fillit H, Finkelstein R, Fisher M, Gendelman HE, Golub RM, Goudreau JL, Gross RA, Gubitz AK, Hesterlee SE, Howells DW, Huguenard J, Kelner K, Koroshetz W, Krainc D, Lazic SE, Levine MS, Macleod MR, McCall JM, Moxley RT, Narasimhan K, Noble LJ, Perrin S, Porter JD, Steward O, Unger E, Utz U, Silberberg SD: A call for transparent reporting to optimize the predictive value of preclinical research. Nature 2012,490(7419):187–191. [http://dx.doi.org/10.1038/nature11556] CrossRef
- Hooijmans CR, Leenaars M, Ritskes-Hoitinga M: A gold standard publication checklist to improve the quality of animal studies, to fully integrate the Three Rs, and to make systematic reviews more feasible. Altern Lab Anim 2010,38(2):167–182.
- Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG: Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLoS Biol 2010,8(6):e1000412. [http://dx.doi.org/10.1371/journal.pbio.1000412] CrossRef
- Making methods clearer Nat Neurosci 2013, 16:1. [http://dx.doi.org/10.1038/nn0113-1]
- Wicherts JM, Borsboom D, Kats J, Molenaar D: The poor availability of psychological research data for reanalysis. Am Psychol 2006,61(7):726–728. [http://dx.doi.org/10.1037/0003-066X.61.7.726] CrossRef
- Bakker M, Wicherts JM: The (mis)reporting of statistical results in psychology journals. Behav Res Methods 2011,43(3):666–678. [http://dx.doi.org/10.3758/s13428-011-0089-5] CrossRef
- Wicherts JM, Bakker M, Molenaar D: Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results. PLoS One 2011,6(11):e26828. [http://dx.doi.org/10.1371/journal.pone.0026828] CrossRef
- Improving basic and translational science by accounting for litter-to-litter variation in animal models
- Open Access
- Available under Open Access This content is freely available online to anyone, anywhere at any time.
- Online Date
- March 2013
- Online ISSN
- BioMed Central
- Additional Links
- Experimental design
- Mixed-effects model
- Nested model
- Valproic acid