Animal Cognition

, Volume 20, Issue 6, pp 1019–1033 | Cite as

Is a local sample internationally representative? Reproducibility of four cognitive tests in family dogs across testing sites and breeds

  • Dóra SzabóEmail author
  • Daniel S. Mills
  • Friederike Range
  • Zsófia Virányi
  • Ádám Miklósi
Methods Paper


A fundamental precept of the scientific method is reproducibility of methods and results, and there is growing concern over the failure to reproduce significant results. Family dogs have become a favoured species in comparative cognition research, but they may be subject to cognitive differences arising from genetic (breeding lines) or cultural differences (e.g. preferred training methods). Such variation is of concern as it affects the validity and generalisability of experimental results. Despite its importance, this problem has not been specifically addressed to date. Therefore, we aimed to test the influence of three factors on reproducibility: testing site (proximal environment), breed and sex (phenotype). The same experimenter tested cognitive performance by more than 200 dogs in four experiments. Additionally, dogs’ performance was tested in an obedience task administered by the owner. Breed of dog and testing site were found to influence the level of performance only mildly, and only in a means-end experiment and the obedience task. Our findings demonstrate that by applying the same test protocols on sufficiently large samples, the reported phenomena in these cognitive tests can be reproduced, but slight differences in performance levels can occur between different samples. Accordingly, we recommend the utilisation of well-described protocols supported by video examples of the whole experimental procedure. Findings should focus on the main outcome variables of the experiments, rather than speculating about the general importance of small or secondary performance outcomes which are more susceptible to random or local noise.


Behaviour tests Reproducibility Replicability Dog Cross-country comparison Breed comparison 



This research was supported by the European Union and the State of Hungary, co-financed by ESF Research Networking Programme “CompCog”: The Evolution of Social Cognition ( (06-RNP-020), and the European Social Fund in the framework of TÁMOP 4.2.4. A/1-11-1-2012-0001 ‘National Excellence Program’. Dóra Szabó also received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (Grant Agreement No. 680040), Ádám Miklósi also received support from the Hungarian Academy of Sciences (MTA 01 031), and Friederike Range received financial support from the FWF (Project Number: P24840-B16). We would like to thank Judit Abdai, Camille Hansart, Beáta Korcsok, Krisztina Kovács and Flóra Szánthó for their help. We are also grateful to all owners and their dogs for their participation in the study.

Supplementary material

10071_2017_1133_MOESM1_ESM.csv (10 kb)
Supplementary material 1 (CSV 9 kb)


  1. Asendorpf JB, Conner M, De Fruyt F, De Houwer J, Denissen JJA, Fiedler K, Fiedler S, Funder DC, Kliegl R, Nosek BA, Perugini M, Roberts BW, Schmitt M, Van Aken MAG, Weber H, Wicherts JM (2013) Recommendations for increasing replicability in psychology. Eur J Pers 27:108–119. doi: 10.1002/per.1919 CrossRefGoogle Scholar
  2. Baldini S, Restani L, Baroncelli L, Coltelli M, Franco R, Cenni MC, Maffei L, Berardi N (2013) Enriched early life experiences reduce adult anxiety-like behavior in rats: a role for insulin-like growth factor 1. J Neurosci 33:11715–11723. doi: 10.1523/JNEUROSCI.3541-12.2013 CrossRefPubMedGoogle Scholar
  3. Brúder I (2010) Development of a test battery for evaluating dog personality and investigating the genetic background of personality traits. Eötvös Loránd University, BudapestGoogle Scholar
  4. Casadevall A, Fang FC (2010) Reproducible science. Infect Immun 78:4972–4975. doi: 10.1128/IAI.00908-10 CrossRefPubMedPubMedCentralGoogle Scholar
  5. Crabbe JC, Wahlsten D, Dudek BC (1999) Genetics of mouse behavior: interactions with laboratory environment. Science 284:1670–1672. doi: 10.1126/science.284.5420.1670 CrossRefPubMedGoogle Scholar
  6. Duranton C, Rödel HG, Bedossa T, Belkhir S (2015) Inverse sex effects on performance of domestic dogs (Canis familiaris) in a repeated problem-solving task. J Comp Psychol 129:84–87. doi: 10.1037/a0037825 CrossRefPubMedGoogle Scholar
  7. Fadel FR, Driscoll P, Pilot M, Wright H, Zulch H, Mills D (2016) Differences in trait impulsivity indicate diversification of dog breeds into working and show lines. Sci Rep 6:22162. doi: 10.1038/srep22162 CrossRefPubMedPubMedCentralGoogle Scholar
  8. Fujita K, Morisaki A, Takaoka A (2012) Incidental memory in dogs (Canis familiaris): adaptive behavioral solution at an unexpected memory test. Anim Cogn 15:1055–1063. doi: 10.1007/s10071-012-0529-3 CrossRefPubMedGoogle Scholar
  9. Fukuzawa M, Mills DS, Cooper JJ (2005) The effect of human command phonetic characteristics on auditory cognition in dogs (Canis familiaris). J Comp Psychol 119:117–121CrossRefPubMedGoogle Scholar
  10. Horn L, Marshall-Pescini S, Virányi Z, Range F (2013a) Cross-cultural differences in domestic dogs’ interactions with humans—preliminary results from Ainsworth’s strange situation test. J Vet Behav Clin Appl Res 8:e39. doi: 10.1016/j.jveb.2013.04.043 CrossRefGoogle Scholar
  11. Horn L, Range F, Huber L (2013b) Dogs’ attention towards humans depends on their relationship, not only on social familiarity. Anim Cogn 16:435–443. doi: 10.1007/s10071-012-0584-9 CrossRefPubMedGoogle Scholar
  12. Huber L, Racca A, Scaf B, Virányi Z, Range F (2013) Discrimination of familiar human faces in dogs (Canis familiaris). Learn Motiv 44:258–269. doi: 10.1016/j.lmot.2013.04.005 CrossRefPubMedPubMedCentralGoogle Scholar
  13. Huber L, Range F, Virányi Z (2012) Dogs imitate selectively, not necessarily rationally: reply to Kaminski et al. (2011). Anim Behav 83:3–5. doi: 10.1016/j.anbehav.2012.03.020 CrossRefGoogle Scholar
  14. Kafkafi N, Golani I, Jaljuli I, Morgan H, Sarig T, Würbel H, Yaacoby S, Benjamini Y (2017) Addressing reproducibility in single-laboratory phenotyping experiments. Nat Methods 14:462–464. doi: 10.1038/nmeth.4259 CrossRefPubMedGoogle Scholar
  15. Kaminski J, Nitzschner M, Wobber V, Tennie C, Brauer J, Call J, Tomasello M (2011) Do dogs distinguish rational from irrational acts? Anim Behav 81:195–203. doi: 10.1016/j.anbehav.2010.10.001 CrossRefGoogle Scholar
  16. Kampis G, Miklosi A, Viranyi Z, Gulyas L (2010) Video deep tagging and data archiving in the comparative mind database. In: Spink AJ, Grieco F, Krips OE, Loijens L, Noldus L, Zimmerman P (eds) Proceedings of the 7th international conference on methods and techniques in behavioral research, Eindhoven. Noldus, Wageningen, NL, pp 185–188Google Scholar
  17. Klein RA, Ratliff KA, Vianello M, Adams RB, Bahník Š, Bernstein MJ, Bocian K, Brandt MJ, Brooks B, Brumbaugh CC, Cemalcilar Z, Chandler J, Cheong W, Davis WE, Devos T, Eisner M, Frankowska N, Furrow D, Galliani EM, Hasselman F, Hicks JA, Hovermale JF, Hunt SJ, Huntsinger JR, Ijzerman H, John MS, Joy-Gaba JA, Kappes HB, Krueger LE, Kurtz J, Levitan CA, Mallett RK, Morris WL, Nelson AJ, Nier JA, Packard G, Pilati R, Rutchick AM, Schmidt K, Skorinko JL, Smith R, Steiner TG, Storbeck J, Van Swol LM, Thompson D, Van’T Veer AE, Vaughn LA, Vranka M, Wichman AL, Woodzicka JA, Nosek BA (2014) Investigating variation in replicability: a “many labs” replication project. Soc Psychol 45:142–152. doi: 10.1027/1864-9335/a000178 CrossRefGoogle Scholar
  18. Marshall-Pescini S, Frazzi C, Valsecchi P (2016) The effect of training and breed group on problem-solving behaviours in dogs. Anim Cogn 19:571–579. doi: 10.1007/s10071-016-0960-y CrossRefPubMedGoogle Scholar
  19. Marshall-Pescini S, Valsecchi P, Petak I, Accorsi PA, Previde EP (2008) Does training make you smarter? The effects of training on dogs’ performance (Canis familiaris) in a problem solving task. Behav Process 78:449–454. doi: 10.1016/j.beproc.2008.02.022 CrossRefGoogle Scholar
  20. Miklósi Á (2014) Dog behaviour, evolution, and cognition, 2nd edn. Oxford University Press, OxfordCrossRefGoogle Scholar
  21. Miklósi Á, Topál J (2013) What does it take to become “best friends”? Evolutionary changes in canine social competence. Trends Cogn Sci 17:287–294. doi: 10.1016/j.tics.2013.04.005 CrossRefPubMedGoogle Scholar
  22. Moonesinghe R, Khoury MJ, Janssens ACJW (2007) Most published research findings are false—but a little replication goes a long way. PLoS Med 4:218–221. doi: 10.1371/journal.pmed.0040028 CrossRefGoogle Scholar
  23. Müller CA, Mayer C, Dorrenberg S, Huber L, Range F (2011) Female but not male dogs respond to a size constancy violation. Biol Lett 7:689–691. doi: 10.1098/rsbl.2011.0287 CrossRefPubMedPubMedCentralGoogle Scholar
  24. Müller CA, Riemer S, Virányi Z, Huber L, Range F (2014) Dogs learn to solve the support problem based on perceptual cues. Anim Cogn 17:1071–1080. doi: 10.1007/s10071-014-0739-y CrossRefPubMedPubMedCentralGoogle Scholar
  25. Open Science Collaboration (2015) Estimating the reproducibility of psychological science. Science 349:aac4716. doi: 10.1126/science.aac4716 CrossRefGoogle Scholar
  26. Pongrácz P, Bánhegyi P, Miklósi Á (2012) When rank counts—dominant dogs learn better from a human demonstrator in a two-action test. Behaviour 149:111–132. doi: 10.1163/156853912X629148 CrossRefGoogle Scholar
  27. Pongrácz P, Gácsi M, Hegedüs D, Péter A, Miklósi A (2013) Test sensitivity is important for detecting variability in pointing comprehension in canines. Anim Cogn 16:721–735. doi: 10.1007/s10071-013-0607-1 CrossRefPubMedGoogle Scholar
  28. Prato-Previde E, Marshall-Pescini S, Valsecchi P (2008) Is your choice my choice? The owners’ effect on pet dogs’ (Canis lupus familiaris) performance in a food choice task. Anim Cogn 11:167–174. doi: 10.1007/s10071-007-0102-7 CrossRefPubMedGoogle Scholar
  29. Range F, Hentrup M, Virányi Z (2011) Dogs are able to solve a means-end task. Anim Cogn 14:575–583. doi: 10.1007/s10071-011-0394-5 CrossRefPubMedPubMedCentralGoogle Scholar
  30. Richter SH, Garner JP, Würbel H (2009) Environmental standardization: cure or cause of poor reproducibility in animal experiments? Nat Methods 6:257–261. doi: 10.1038/NmETH.1312 CrossRefPubMedGoogle Scholar
  31. Sidman M (1960) Tactics of scientific research: Evaluating experimental data in psychology, vol 5. Basic Books, New YorkGoogle Scholar
  32. Topál J, Miklósi Á, Csányi V (1997) Dog–human relationship affects problem solving behavior in the dog. Anthrozoös 10:214–224. doi: 10.2752/089279397787000987 CrossRefGoogle Scholar
  33. Tuyttens FAM, de Graaf S, Heerkens JLT, Jacobs L, Nalon E, Ott S, Stadig L, Van Laer E, Ampe B (2014) Observer bias in animal behaviour research: can we believe what we score, if we score what we believe? Anim Behav 90:273–280. doi: 10.1016/j.anbehav.2014.02.007 CrossRefGoogle Scholar
  34. van der Staay JF, Arndt SS, Nordquist RE (2010) The standardization–generalization dilemma: a way out. Genes Brain Behav 9:849–855. doi: 10.1111/j.1601-183X.2010.00628.x CrossRefPubMedGoogle Scholar
  35. Wahlsten D, Bachmanov A, Finn DA, Crabbe JC (2006) Stability of inbred mouse strain differences in behavior and brain size between laboratories and across decades. Proc Natl Acad Sci USA 103:16364–16369. doi: 10.1073/pnas.0605342103 CrossRefPubMedPubMedCentralGoogle Scholar
  36. Yang H, Harrington CA, Vartanian K, Coldren CD, Hall R, Churchill GA, Richter SH, Garner JP, Zipser B, Lewejohann L, Sachser N, Schindler B, Chourbaji S, Brandwein C, Gass P, Van Stipdonk N, Wolfer DP, Wu H (2008) Randomization in laboratory procedure is key to obtaining reproducible microarray results. PLoS ONE 3:e3724. doi: 10.1371/journal.pone.0003724 CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.Department of EthologyEötvös Loránd UniversityBudapestHungary
  2. 2.School of Life SciencesUniversity of LincolnLincolnUK
  3. 3.Messerli Research Institute, University of Veterinary Medicine ViennaMedical University of Vienna and University of ViennaViennaAustria
  4. 4.MTA-ELTE Comparative Ethology Research GroupBudapestHungary

Personalised recommendations