Advertisement

Aeroecology pp 239-257 | Cite as

Multi-Camera Videography Methods for Aeroecology

  • Margrit Betke
  • Tyson Hedrick
  • Diane Theriault
Chapter

Abstract

The ability to study the behavior of flying animals in their natural environment has been dramatically improved by recent advances in imaging technologies and video analysis tools. Stereo videography, in particular, has taken aeroecology to a new level. We here describe methods for estimating the three-dimensional flight paths of animals from image data recorded simultaneously by multiple visible-light or infrared cameras. We explain how the accuracy of these estimates is influenced by camera properties and placements, as well as calibration procedures. We include a description of potential pitfalls drawn from field experience with various bat and bird species and show some example stereoscopic reconstructions of bat and bird flight.

Notes

Acknowledgements

We wish to thank N. Fuller, B. Jackson, E. Bluhm, D. Evangelista, Z. Wu, B. Borucki, A. Banks, A. Froschauer, and K. Swift for their research assistance. We thank D. Davis and D. Bamberger for property access and the Department of Texas Parks and Wildlife for permitting assistance. This work was partially funded by the Office of Naval Research [N000141010952 to M.B. and T.H.], the National Science Foundation [0910908 and 0855065 to M. B. and 1253276 to T. H.], and the Air Force Office of Scientific Research [FA9550-07-1-0540 to M. B.].

References

  1. Abdel-Aziz YI, Karara HM (1971) Direct linear transformation from comparator coordinates into object space coordinates in close-range photogrammetry. In: Proceedings of the symposium on close range photogrammetry, Direct linear transformation into object space coordinates in close-range photogrammetry. American Society of Photogrammetry, Falls Church, VA, pp 1–18Google Scholar
  2. Attanasi A, Cavagna A, Castello LD, Giardina I, Jelic A, Melillo S, Parisi L, Pellacini F, Shen E, Silvestri E, Viale M (2013) Tracking in three dimensions via recursive multi-path branching. http://arxiv.org/abs/1305.1495
  3. Attanasi A, Cavagna A, Castello LD, Giardina I, Jelic A, Melillo S, Parisi L, Pohl O, Shen E, Viale M (2014) Emergence of collective changes in travel direction of starling flocks from individual birds fluctuations. http://arxiv.org/abs/1410.3330
  4. Balch T, Khan Z, Veloso M (2001) Automatically tracking and analyzing the behavior of live insect colonies. In: Proceedings of the fifth international conference on autonomous agents, Montreal, Canada, pp 521–528Google Scholar
  5. Betke M, Hirsh DE, Bagchi A, Hristov NI, Makris NC, Kunz TH (2007) Tracking large variable numbers of objects in clutter. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition (CVPR), Minneapolis, MN, 8 p.  https://doi.org/10.1109/CVPR.2007.382994
  6. Betke M, Hirsh DE, Makris NC, McCracken GF, Procopio M, Hristov NI, Tang S, Bagchi A, Reichard JD, Horn JW, Crampton S, Cleveland CJ, Kunz TH (2008) Thermal imaging reveals significantly smaller Brazilian free-tailed bat colonies than previously estimated. J Mammal 89(1):18–24. https://academic.oup.com/jmammal/article/89/1/18/1021254 [Also discussed in Nature 452, Research highlights, p 507]
  7. Boardman BL, Hedrick TL, Theriault DH, Fuller NW, Betke M, Morgansen KA (2013) Collision avoidance in biological systems using collision cones. In: Proceedings of the 2013 American control conference, 8 p.  https://doi.org/10.1109/ACC.2013.6580285
  8. Bouguet JY (1999) Visual methods for three-dimensional modeling. PhD thesis, California Institute of TechnologyGoogle Scholar
  9. Bouguet JY (2004) Camera calibration toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/
  10. Bradski G (2000) The OpenCV library. Doctor. Dobbs J 25(11):120–126Google Scholar
  11. Breslav M, Fuller NW, Betke M (2012) Vision system for wingbeat analysis of bats in the wild. In: Proceedings of the workshop on visual observation and analysis of animal and insect behavior (VAIB 2012), held in conjunction with the 21st international conference on pattern recognition (ICPR 2012) Tsukuba, Japan, p 4. http://homepages.inf.ed.ac.uk/rbf/VAIB12PAPERS/breslav.pdf
  12. Breslav M, Fuller N, Sclaroff S, Betke M (2014) 3d pose estimation of bats in the wild. In: Proceedings of the IEEE winter conference on applications of computer vision, Steamboat Springs, CO, 8 p.  https://doi.org/10.1109/WACV.2014.6836114, http://ieeexplore.ieee.org/document/6836114/
  13. Brown RG, Hwang PYC (1997) Introduction to random signals and applied Kalman filtering. Wiley, New YorkGoogle Scholar
  14. Cavagna A, Giardina I, Orlandi A, Parisi G, Procaccini A, Viale M, Zdravkovic V (2008) The STARFLAG handbook on collective animal behaviour: 1. Empirical methods. Anim Behav 76(1):217–236CrossRefGoogle Scholar
  15. Clark CJ (2009) Courtship dives of Anna’s Hummingbird offer insights into flight performance limits. Proc R Soc B Biol Sci 276(1670):3047–3052CrossRefGoogle Scholar
  16. Cleveland CJ, Betke M, Federico P, Frank JD, Hallam TG, Horn J, Kunz TH, López JD, McCracken GF, Medellín RA, Moreno-Valdez A, Sansone CG, Westbrook JK (2006) Economic value of pest control services provided by Brazilian free-tailed bats in south-central Texas. Front Ecol Environ 4:238–243.  https://doi.org/10.1890/1540-9295(2006)004[0238:EVOTPC]2.0.CO;2 CrossRefGoogle Scholar
  17. Erwin HR, Wilson WW, Moss CF (2001) A computational sensorimotor model of bat echolocation. J Acoust Soc Am 110(2):1176–1187CrossRefPubMedGoogle Scholar
  18. Evangelista JD, Ray DD, Raja SK, Hedrick TL (2017) Three-dimensional trajectories and network analyses of group behaviour within chimney swift flocks during approaches to the roost. Proc R Soc B 284(1849):20162602.  https://doi.org/10.1098/rspb.2016.2602, http://rspb.royalsocietypublishing.org/content/royprsb/284/1849/20162602.full.pdf
  19. Federico P, Hallam TG, McCracken GF, Purucker S, Grant W, Sandoval AN, Westbrook J, Medellín R, Cleveland C, Sansone CG, López JD Jr, Betke M, Moreno-Valdez A, Kunz TH (2008) Brazilian free-tailed bats (Tadarida brasiliensis) as insect pest regulators in transgenic and conventional cotton crops. Ecol Appl 18(4):826–837. http://onlinelibrary.wiley.com/doi/10.1890/07-0556.1/epdf
  20. Hallam TG, Raghavan A, Kolli H, Dimitrov D, Federico P, Qi H, McCracken GF, Betke M, Westbrook JK, Kennard K, Kunz TH (2010) Dense and sparse aggregations in complex motion: video coupled with simulation modeling. Ecol Complex 7(1):69–75.  https://doi.org/10.1016/j.ecocom.2009.05.012
  21. Hartley R, Zisserman A (2004) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  22. Hedrick TL (2008) Software techniques for two- and three-dimensional kinematic measurements of biological and biomimetic systems. Bioinspir Biomim 3:034001. http://iopscience.iop.org/article/10.1088/1748-3182/3/3/034001/pdf
  23. Hedrick T, Usherwood J, Biewener A (2004) Wing inertia and whole-body acceleration: an analysis of instantaneous aerodynamic force production in cockatiels Nymphicus hollandicus flying across a range of speeds. J Exp Biol 207:1689–1702CrossRefPubMedGoogle Scholar
  24. Heikkila J, Silvén O (1997) A four-step camera calibration procedure with implicit image correction. In: Computer vision and pattern recognition, 1997. Proceedings, 1997 I.E. computer society conference on, IEEE, pp 1106–1112Google Scholar
  25. Hristov NI, Betke M, Kunz TH (2008) Applications of thermal infrared imaging for research in aeroecology. Integr Comp Biol 48(1):50–59. https://academic.oup.com/icb/article/48/1/50/628016
  26. Hristov NI, Betke M, Hirsh D, Bagchi A, Kunz TH (2010) Seasonal variation in colony size of Brazilian free-tailed bats at Carlsbad Caverns using thermal imaging. J Mammal 91(1):183–192.  https://doi.org/10.1644/08-MAMM-A-391R.1 CrossRefGoogle Scholar
  27. Khan Z, Balch T, Dellaert F (2004) A Rao-Blackwellized particle filter for eigentracking. In: IEEE conference on computer vision and pattern recognition, Washington, DC, pp 980–986Google Scholar
  28. Kong Z, Özcimder K, Fuller N, Greco A, Theriault D, Wu Z, Kunz T, Betke M, Baillieul J (2013) Optical flow sensing and the inverse perception problem for flying bats. In: The 2013 I.E. conference on decision and control (CDC), Florence, Italy, 8 p.  https://doi.org/10.1109/CDC.2013.6760112, http://ieeexplore.ieee.org/document/6760112/
  29. Kong Z, Özcimder K, Fuller N, Theriault D, Betke M, Baillieul J (2014) Perception and steering control in paired bat flight. In: The 19th world congress of the international federation of automatic control (IFAC), South Africa, 7 pGoogle Scholar
  30. Kunz T, Gauthreaux SJ, Hristov N, Horn J, Jones G, Kalko E, Larkin R, McCracken G, Swartz S, Srygley R, Dudley R, Westbrook J, Wikelski M (2008) Aeroecology: probing and modeling the aerosphere. Integr Comp Biol 48(1):1–11CrossRefPubMedGoogle Scholar
  31. Kunz TH, Betke M, Hristov NI, Vonhof M (2009) Methods for assessing colony size, population size, and relative abundance of bats. In: Kunz TH, Parsons S (eds) Ecological and behavioral methods for the study of bats, 2nd edn. Johns Hopkins University Press, Baltimore, MD, pp 133–157Google Scholar
  32. Lee EY, Betke M, Kunz TH (2004) Bats in motion: stereo object recognition and trajectory analysis of flying bats. In: Proceedings of the 34th Annual North American symposium on bat research (NASBR), Salt Lake City, UT, p 61Google Scholar
  33. Lourakis MA, Argyros A (2009) SBA: a software package for generic sparse bundle adjustment. ACM Trans Math Softw 36(1):1–30.  https://doi.org/10.1145/1486525.1486527 CrossRefGoogle Scholar
  34. Munk JD (2011) The descent of ant. PhD thesis, University of California, Berkeley. http://rspb.royalsocietypublishing.org/content/royprsb/284/1849/20162602.full.pdf
  35. Premerlani LB (2007) Stereoscopic reconstruction and analysis of infrared video of bats. Master’s thesis, Boston UniversityGoogle Scholar
  36. Reid DB (1979) An algorithm for tracking multiple targets. IEEE Trans Autom Control 24:843–854CrossRefGoogle Scholar
  37. Riskin D, Willis D, Iriarte-Díaz J, Hedrick T, Kostandov M, Chen J, Laidlaw D, Breuer K, Swartz S (2008) Quantifying the complexity of bat wing kinematics. J Theor Biol 254(3):604–615CrossRefPubMedGoogle Scholar
  38. Sheltom RM, Jackson BE, Hedrick TL (2014) The mechanics and behavior of cliff swallows during tandem flights. J Exp Biol 217: 2717–2725.  https://doi.org/10.1242/jeb.101329, http://jeb.biologists.org/content/jexbio/217/15/2717.full.pdf?with-ds=yes
  39. Sikes R, Gannon W (2011) Animal Care and Use Committee of the American Society of Mammalogists guidelines of the American Society of Mammalogists for the use of wild mammals in research. J Mammal 92:235–253CrossRefGoogle Scholar
  40. Theriault DH, Wu Z, Hristov NI, Swartz SM, Breuer KS, Kunz TH, Betke M (2010) Reconstruction and analysis of 3D trajectories of Brazilian free-tailed bats in flight. In: Workshop on visual observation and analysis of animal and insect behavior, held in conjunction with the 20th international conference on pattern recognition, August, 2010, Istanbul, Turkey, 4 p. http://homepages.inf.ed.ac.uk/rbf/VAIB10PAPERS/theriault-icpr-20pdf
  41. Theriault DH, Fuller N, Betke M (2013) Understanding collective behavior in Tadarida brasiliensis using computer vision and multi-target tracking. In: Proceedings of the 16th international bat research conference, San José, Costa RicaGoogle Scholar
  42. Theriault DH, Fuller NW, Jackson BE, Bluhm E, Evangelista D, Wu Z, Betke M, Hedrick TL (2014) A protocol and calibration method for accurate multi-camera field videography. J Exp Biol 217:1843–1848.  https://doi.org/10.1242/jeb.100529, http://jeb.biologists.org/content/217/11/1843.full.pdf
  43. Towne G, Theriault DH, Wu Z, Fuller NW, Kunz TH, Betke M (2012) Error analysis and design considerations for stereo vision systems used to analyze animal behavior. In: Proceedings of the workshop on visual observation and analysis of animal and insect behavior (VAIB 2012), held in conjunction with the 21st international conference on pattern recognition (ICPR 2012) Tsukuba, Japan, 4 p. http://www.cs.bu.edu/fac/betke/papers/VAIB-Towne-etal-2012.pdf
  44. Tweed D, Calway A (2002) Tracking many objects using subordinated CONDENSATION. In: Proceedings of the British machine vision conference, Cardiff, UK, pp 283–292Google Scholar
  45. Veeraraghavan A, Chellappa R, Srinivasan M (2008) Shape-and-behavior-encoded tracking of bee dances. IEEE Trans Pattern Anal Mach Intell 30(3):463–476CrossRefPubMedGoogle Scholar
  46. Walker SM, Thomas AL, Taylor GK (2009) Photogrammetric reconstruction of high-resolution surface topographies and deformable wing kinematics of tethered locusts and free-flying hoverflies. J R Soc Interface 6(33):351–366CrossRefPubMedGoogle Scholar
  47. Wu Z, Hristov NI, Hedrick TL, Kunz TH, Betke M (2009a) Tracking a large number of objects from multiple views. In: Proceedings of the international conference on computer vision (ICCV), Kyoto, Japan, 8 p.  https://doi.org/10.1109/ICCV.2009.5459274, http://ieeexplore.ieee.org/document/5459274/
  48. Wu Z, Hristov NI, Kunz TH, Betke M (2009b) Tracking-reconstruction or reconstruction-tracking? Comparison of two multiple hypothesis tracking approaches to interpret 3D object motion from several camera views. In: Proceedings of the IEEE workshop on motion and video computing (WMVC), Snowbird, Utah, 8 p.  https://doi.org/10.1109/WMVC.2009.5399245, http://ieeexplore.ieee.org/document/5399245/
  49. Wu Z, Kunz TH, Betke M (2011) Efficient track linking methods for track graphs using network-flow and set-cover techniques. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Colorado Springs, pp 1185–1192.  https://doi.org/10.1109/CVPR.2011.5995515, http://ieeexplore.ieee.org/document/5995515/
  50. Wu Z, Thangali A, Sclaroff S, Betke M (2012) Coupling detection and data association for multiple object tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Providence, RI, 8 p.  https://doi.org/10.1109/CVPR.2012.6247896, http://ieeexplore.ieee.org/document/6247896/
  51. Wu Z, Fuller N, Theriault D, Betke M (2014) A thermal infrared video benchmark for visual analysis. In: Proceedings of the 10th IEEE workshop on perception beyond the visible spectrum (PBVS), Columbus, OH, 8 p.  https://doi.org/10.1109/CVPRW.2014.39, http://ieeexplore.ieee.org/document/6909984/
  52. Yang X, Schaaf C, Strahler A, Kunz T, Fuller N, Betke M, Zu WZ, Theriault D, Culvenor D, Jupp D, Newnham G, Lovell J (2013) Study of bat flight behavior by combining thermal image analysis with a LiDAR forest reconstruction. Can J Remote Sens 39(S1):S112–S125.  https://doi.org/10.5589/m13-034 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2017

Authors and Affiliations

  1. 1.Boston UniversityBostonUSA
  2. 2.University of North Carolina at Chapel HillChapel HillUSA

Personalised recommendations