Skip to main content
SpringerLink
Log in

A Multi-Camera Active-Vision System for Deformable-Object-Motion Capture

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

A novel methodology is proposed to select the on-line, near-optimal positions and orientations of a set of dynamic cameras, for a reconfigurable multi-camera active-vision system to capture the motion of a deformable object. The active-vision system accounts for the deformation of the object-of-interest by fusing tracked vertices on its surface with those triangulated from features detected in each camera’s view, in order to predict the shape of the object at subsequent demand instants. It then selects a system configuration that minimizes error in the recovered position of each of these features. The tangible benefits of using a reconfigurable system, particularly with translational cameras, versus systems with static cameras in a fixed configuration, are demonstrated through simulations and experiments in both obstacle-free and obstacle-laden environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. White, R., Crane, K., Forsyth D.: Capturing and animating occluded cloth. ACM Trans. Graph. 26, 34 (2007)

    Article  Google Scholar 

  2. Bai, X., Liu, W., Wang, X., Latecki, L.J., Tu, Z.: Active skeleton for non-rigid object detection. In: IEEE Int. Conf. Computer Vision, Kyoto, Japan, pp. 575–582 (2009)

  3. Park, S.I., Hodgins, J.K.: Capturing and animating skin deformation in human motion. ACM Trans. Graph. 25(3), 881–889 (2006)

    Article  Google Scholar 

  4. Vlasic, D., et al.: Dynamic shape capture using multi-view photometric stereo. ACM Trans. Graph. 28(5), 174:1–147:11 (2009)

    Article  Google Scholar 

  5. Drouin, S., Hebert, P., Parizeau, M.: Incremental discovery of object parts in video sequences. Comput. Vision Image Underst. 110, 60–74 (2008)

    Article  Google Scholar 

  6. Scholz, V., Stich, T., Magnor, M., Keckeisen, M., Wacker, M.: Garment motion capture using color-coded patterns. CGF 24(3), 439–448 (2005)

    Google Scholar 

  7. Bradley, D., Heidrich, W., Popa, T., Sheffer, A.: High resolution passive facial performance capture. Proceedings of ACM SIGGRAPH (Los Angeles, USA, July 25–29, 2010). ACM Trans. Graph. 29(4), 41:1–41:10 (2010)

    Article  Google Scholar 

  8. de Aguiar, E., et al.: Performance capture from sparse multi-view video. ACM Trans. Graph. 27(3), 98–108 (2008)

    Article  MathSciNet  Google Scholar 

  9. Cerezo, E., et al.: Real-Time facial expression recognition for natural interaction. In: Iberian Conf. Pattern Recognition and Image Analysis, pp. 40–47. Girona, Spain (2007)

  10. Petit, B., et al.: Multicamera real-tme 3D modeling for telepresence and remote collaboration. Int. J. Digit. Multimed. Broadcast. 2010, 24108–12 (2009)

    Google Scholar 

  11. Lo, B., Chung, A., Stoyanov, D., Mylonas, G., Yan, G.Z.: Real-time intra-operative 3D tissue deformation recovery. In: IEEE Int. Symp. Biomedical Imaging, pp. 1387–1390. Paris, France (2008)

  12. Richa, R., Bo, A.P., Poignet, P.: Towards robust 3D visual tracking for motion compensation in beating heart surgery. Med. Image Anal. 15(3), 301–315 (2011)

    Article  Google Scholar 

  13. Sharf, A., et al.: Space-time surface reconstruction using incompressible flow. ACM Trans. Graph. 27(5), 110–120 (2008)

    Article  Google Scholar 

  14. Chen, S., Li, Y., Ming Kwok, N.: Active vision in robotic systems: a survey of recent developments. Int. J. Robot. Res. 20(11), 1343–1377 (2011)

    Article  Google Scholar 

  15. Chellappa, R., Roy-Chowdhury, A.K., Zhou, S.K.: Human Activity Recognition. Morgan & Claypool Publishing, San Rafael, CA (2005)

    Google Scholar 

  16. Abrams, S., Allen, P.K., Tarabanis, K.A.: Dynamic sensor planning. In: IEEE Int. Conf. Robotics and Automation, pp. 605–610. Atlanta, GA (1993)

  17. Miao, Y.-Q., Khamis, A., Kamel, M.: Coordinated motion control of mobile sensors in surveillance systems. In: Int. Conf. Signals, Circuits and Systems, pp. 1–6. Djerba, Tunisia (2009)

  18. Caglioti, V.: A unified criterion for minimum uncertainty sensing in object recognition and localization. In: IEEE Int. Conf. on Intelligent Robots and Systems, pp. 1585–1590. Raleigh, NC (1992)

  19. Beb, R., Paulus, D., Niemann, H.: 3D recovery using calibrated active cameras. In: Int. Conf. Image Processing, pp. 855–858. Lausanne, Switzerland (1996)

  20. Farid, H., Lee, S.W., and Bajcsy, R.: View selection strategies for multi-view, wide-baseline stereo. Univ. of Pennsylvania, Dept. of Comput. Inform. Sci., Philadelphia, PA, Tech. Rep. MS-CIS-94-18 (1994)

  21. Olague, G., Mohr, R.: Optimal camera placement for accurate reconstruction. Pattern Recog. 35(4), 927–944 (2002)

    Article  MATH  Google Scholar 

  22. Klarquist, W.N., Bovik, A.C.: OVEA: a foveated vergent active stereo vision system for dynamic three-dimensional scene recovery. IEEE Trans. Robot. Autom. 14(5), 755–770 (1998)

    Article  Google Scholar 

  23. Wu, X., Matsuyama, T.: Real-time active 3D shape reconstruction for 3D video. In: 3rd Int. Symp. Image Signal Processing Analysis, pp. 186–191. Rome, Italy (2003)

  24. Tan, J.K., Ishikawa, S., Yamaguchi, I., Naito, T., Yokota, M.: 3-D recovery of human motion by mobile stereo cameras. Artif. Life Robot. 10, 64–68 (2006)

    Article  Google Scholar 

  25. Mackay, M., Fenton, R., Benhabib, B.: Time-varying-geometry object surveillance using a multi-camera active-vision system. Int. J. Smart Sensing Intell. Syst. 1(3), 679–704 (2008)

    Google Scholar 

  26. Yous, S., Ukita, N., Masatsugu, K.: An assignment scheme to control multiple pan/tilt cameras for 3D video. J. Multimedia 2(1), 10–19 (2007)

    Article  Google Scholar 

  27. Ilie, A., Welch, G., Macenko, M.: A stochastic quality metric for optimal control of active camera network configurations for 3D computer vision tasks. In: Workshop Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications. Marseille, France (2008)

  28. Collins, R.T., Amidi, O., Kanade, T.: An active camera system for acquiring multi-view video. In: Conf. Image Processing, pp. 17–520. Rochester, NY (2002)

  29. Davis, J.E.: Mixed scale motion recovery. Ph.D. Dissertation, Dept. Computer Science, Stanford Univ, Stanford, CA (2002)

  30. Doshi, A., Starck, J., Hilton, A.: An empirical study of non-rigid surface feature matching of human from 3D Video. J. Virtual Real. Broadcast. 7(3), 1–20 (2010)

    Google Scholar 

  31. Canton-Ferrer, C., Casas, J.R., Tekalp, M., Pardas, M.: Projective Kalman filter: multiocular tracking of 3D locations towards scene understanding. Lecture Notes Comput. Sci. 3869, 250–261 (2006)

    Article  Google Scholar 

  32. Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press (2004)

  33. Bolme, D.S., Beveridge, J.R., Draper, B.A.: Visual object tracking using adaptive correlation filters. In: Computer Vision and Pattern Recognition, pp. 2544–2550. San Francisco, CA (2010)

  34. Kalman, R.E.: A new approach to linear filtering and prediction problems. J. Basic Eng. 82, 35–45 (1960)

    Article  Google Scholar 

  35. Kokkinos, I., Yuille, A.: Unsupervised learning of object deformation models. In: IEEE Int. Conf. Computer Vision, pp. 1–8. Rio de Janeiro, Brazil (2007)

  36. Gallup, D., Frahm, J.-M., Mordohai, P., Pollefeys, M.: Variable baseline/resolution stereo. In: Proc. Computer Vision and Pattern Recognition, pp. 1–8. Anchorage, AK (2008)

  37. de Aguiar, E., Theobalt, C., Sto, C.: Marker-less deformable mesh tracking for human shape and motion capture. In: Proc. Computer Vision and Pattern Recognition, pp. 1–8. Minneapolis, MN (2007)

  38. Vo, M., Wang, Z., Luu, L., Ma, J.: Advanced geometric camera calibration for machine vision. Opt. Eng. 50(11), 110503 (2011)

    Article  Google Scholar 

  39. Veres, G.V., Gordon, L., Carter, J.N., Nixon, M.S.: What image information is important in silhouette-based gait recognition? In: Proc. Computer Vision and Pattern Recognition, pp. 776–782. Washington, D.C. (2004)

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON, Canada

    David S. Schacter, Evgeny Nuger, Matthew Mackay & Beno Benhabib

  2. Department of Mechanical and Industrial Engineering, University of Calabria, Rende, Italy

    Mario Donnici

Authors
  1. David S. Schacter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mario Donnici
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Evgeny Nuger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Matthew Mackay
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Beno Benhabib
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Beno Benhabib.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Schacter, D.S., Donnici, M., Nuger, E. et al. A Multi-Camera Active-Vision System for Deformable-Object-Motion Capture. J Intell Robot Syst 75, 413–441 (2014). https://doi.org/10.1007/s10846-013-9961-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-013-9961-0

Keywords

Search

Navigation