Skip to main content
SpringerLink
Log in

Problem Statement and Preliminaries

  • Chapter
Animation and Performance Capture Using Digitized Models

Part of the book series: Cognitive Systems Monographs ((COSMOS,volume 5))

  • 664 Accesses

Abstract

This part of the book first reviews the most relevant work on motion capture, scene reconstruction, and 3D Video. Thereafter, it describes three algorithms to passively capture the performance of human actors, and presents a system to generate high-quality 3D Videos.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ariel dynamics: Apas performance analysis system, http://www.arielnet.com

  2. Motion analysis corporation: Eagle and hawk, http://www.motionanalysis.com

  3. Qualisys: Oqus and proreflex, http://www.qualisys.com

  4. Simi reality motion systems: Simi motion, http://www.simi.com

  5. Vicon - vicon mx and vicon motus, http://www.vicon.com

  6. Agarwal, A., Triggs, B.: Recovering 3d human pose from monocular images. IEEE PAMI 28(1), 44–58 (2006)

    Google Scholar 

  7. de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.P., Thrun, S.: Performance capture from sparse multi-view video. In: ACM TOG, Proc. SIGGRAPH (2008)

    Google Scholar 

  8. de Aguiar, E., Theobalt, C., Magnor, M., Seidel, H.P.: Reconstructing human shape and motion from multi-view video. In: CVMP 2005, pp. 42–49 (2005)

    Google Scholar 

  9. de Aguiar, E., Theobalt, C., Magnor, M., Theisel, H., Seidel, H.P.: M3: Marker-free model reconstruction and motion tracking from 3d voxel data. In: Proc. of PG, pp. 101–110 (2004)

    Google Scholar 

  10. de Aguiar, E., Theobalt, C., Stoll, C., Seidel, H.: Marker-less 3d feature tracking for mesh-based human motion capture. In: Proc. ICCV HUM 2007, pp. 1–15 (2007)

    Google Scholar 

  11. de Aguiar, E., Theobalt, C., Stoll, C., Seidel, H.P.: Marker-less deformable mesh tracking for human shape and motion capture. In: Proc. CVPR (2007)

    Google Scholar 

  12. Allen, B., Curless, B., Popović, Z.: Articulated body deformation from range scan data. ACM Trans. Graph. 21(3), 612–619 (2002)

    Article  Google Scholar 

  13. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: Scape: Shape completion and animation of people. ACM Trans. Graph. 24(3), 408–416 (2005)

    Article  Google Scholar 

  14. Balan, A., Black, M.: An adaptive appearance model approach for model-based articulated object tracking. In: Proc. of CVPR, pp. 758–765 (2006)

    Google Scholar 

  15. Balan, A., Sigal, L., Black, M., Davis, J.E., Haussecker, H.W.: Detailed human shape and pose from images. In: Proc. CVPR (2007)

    Google Scholar 

  16. Balan, A., Sigal, L., Black, M., Haussecker, H.: Shining a light on human pose: On shadows, shading and the estimation of pose and shape. In: Proc. ICCV (2007)

    Google Scholar 

  17. Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., Gross, M.: Multi-scale capture of facial geometry and motion. In: Proc. of SIGGRAPH (2007)

    Google Scholar 

  18. Bodenheimer, B., Rose, C., Rosenthal, S., Pella, J.: The process of motion capture: Dealing with the data. In: Computer Animation and Simulation 1997, pp. 3–18 (1997)

    Google Scholar 

  19. Botsch, M., Sorkine, O.: On linear variational surface deformation methods. IEEE TVCG 14(1), 213–230 (2008)

    Google Scholar 

  20. Bray, M., Koller-Meier, E., Gool, L.V.: Smart particle filtering for high-dimensional tracking. Comput. Vis. Image Underst. 106(1), 116–129 (2007)

    Article  Google Scholar 

  21. Bregler, C.: Learning and recognizing human dynamics in video sequences. In: Proc. of CVPR 1997, p. 568 (1997)

    Google Scholar 

  22. Bregler, C., Malik, J.: Tracking people with twists and exponential maps. In: Proc. of CVPR 1998, pp. 8–15 (1998)

    Google Scholar 

  23. Brox, T., Rosenhahn, B., Cremers, D., Seidel, H.-P.: High accuracy optical flow serves 3-D pose tracking: Exploiting contour and flow based constraints. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 98–111. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  24. Brubaker, M.A., Fleet, D.J., Hertzmann, A.: Physics-based person tracking using simplified lower-body dynamics. In: Proc. CVPR (2007)

    Google Scholar 

  25. Carranza, J., Theobalt, C., Magnor, M., Seidel, H.P.: Free-viewpoint video of human actors. ACM TOG (Proc. of SIGGRAPH 2003) 22(3), 569–577 (2003)

    Article  Google Scholar 

  26. Cham, T.J., Rehg, J.M.: A multiple hypothesis approach to figure tracking. In: Proc. of CVPR, pp. 2239–2245 (1999)

    Google Scholar 

  27. Chen, Z., Lee, H.: Knowledge-guided visual perception of 3d human gait from a single image sequence. IEEE Transactions on Systems, Man and Cybernetics 22(2), 336–342 (1992)

    Article  Google Scholar 

  28. Cheung, G., Baker, S., Kanada, T.: Shape-from-silhouette of articulated objects and its use for human body kinematics estimation and motion capture. In: Proc. of CVPR, pp. 77–84 (2003)

    Google Scholar 

  29. Cheung, G., Baker, S., Kanade, T.: Shape-from-silhouette across time part ii: Applications to human modeling and markerless motion tracking. Int. J. Comput. Vision 63(3), 225–245 (2005)

    Article  Google Scholar 

  30. Cheung, G., Kanade, T., Bouguet, J.Y., Holler, M.: A real time system for robust 3D voxel reconstruction of human motions. In: Proc. of CVPR, vol. 2, pp. 714–720 (2000)

    Google Scholar 

  31. Corazza, S., Mundermann, L., Chaudhari, A., Demattio, T., Cobelli, C., Andriacchi, T.: A markerless motion capture system to study musculoskeletal biomechanics: Visual hull and simulated annealing approach. Annals of Biomedical Engineering 34(11), 1019–1029 (2006)

    Article  Google Scholar 

  32. Covelle, M.M., Rahimi, A., Harville, M., Darrell, T.J.: Articulated pose estimation using brighness and depth constancy constraints. In: Proc. of CVPR, vol. 2, pp. 438–445 (2000)

    Google Scholar 

  33. Decarlo, D., Metaxas, D.: Optical flow constraints on deformable models with applications to face tracking. IJCV 38(2), 99–127 (2000)

    Article  MATH  Google Scholar 

  34. Delamarre, Q., Faugeras, O.: 3D articulated models and multi-view tracking with silhouettes. In: ICCV 1999, pp. 716–721 (1999)

    Google Scholar 

  35. Deutscher, B., Blake, A., Reid, I.: Articulated body motion capture by annealed particle filtering. In: Proc. of CVPR 2000, vol. 2, pp. 126–133 (2000)

    Google Scholar 

  36. Deutscher, J., Reid, I.: Articulated body motion capture by stochastic search. Int. J. Comput. Vision 61(2), 185–205 (2005)

    Article  Google Scholar 

  37. Drummond, T., Cipolla, R.: Real-time tracking of highly articulated structures in the presence of noisy measurements. In: Proc. of ICCV, vol. 2, pp. 315–320 (2001)

    Google Scholar 

  38. Einarsson, P., Chabert, C.F., Jones, A., Ma, W.C., Lamond, B., Hawkins, T., Bolas, M., Sylwan, S., Debevec, P.: Relighting human locomotion with flowed reflectance fields. In: Proc. EGSR, pp. 183–194 (2006)

    Google Scholar 

  39. Esteban, C.H., Schmitt, F.: Silhouette and stereo fusion for 3d object modeling. CVIU 96(3), 367–392 (2004)

    Google Scholar 

  40. Furukawa, Y., Ponce, J.: Carved visual hulls for image-based modeling. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 564–577. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  41. Gall, J., Rosenhahn, B., Seidel, H.-P.: Robust pose estimation with 3D textured models. In: Chang, L.-W., Lie, W.-N. (eds.) PSIVT 2006. LNCS, vol. 4319, pp. 84–95. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  42. Gall, J., Rosenhahn, B., Seidel, H.P.: Drift-free tracking of rigid and articulated objects. In: Proc. of CVPR (2008)

    Google Scholar 

  43. Gavrila, D.M., Davis, L.S.: 3D model-based tracking of humans in action: A multi-view approach. In: Proc. of CVPR, pp. 73–80 (1996)

    Google Scholar 

  44. Goldluecke, B., Magnor, M.: Space-time isosurface evolution for temporally coherent 3d reconstruction. In: Proc. CVPR, vol. I, pp. 350–355 (2004)

    Google Scholar 

  45. Goncalves, L., Di Bernardo, E., Ursella, E., Perona, P.: Monocular tracking of the human arm in 3D. In: Proc. of CVPR, pp. 764–770 (1995)

    Google Scholar 

  46. Grammalidis, N., Goussis, G., Troufakos, G., Strintzis, M.G.: Estimating body animation parameters from depth images using analysis by synthesis. In: Proc. of DCV 2001, p. 93 (2001)

    Google Scholar 

  47. Grauman, K., Shakhnarovich, G., Darrell, T.: Inferring 3d structure with a statistical image-based shape model. In: Proc. of ICCV 2003, p. 641 (2003)

    Google Scholar 

  48. Gross, M., Würmlin, S., Näf, M., Lamboray, E., Spagno, C., Kunz, A., Koller-Meier, E., Svoboda, T., Gool, L.V., Lang, S., Strehlke, K., Moere, A.V., Staadt, O.: Blue-c: a spatially immersive display and 3d video portal for telepresence. ACM TOG 22(3), 819–827 (2003)

    Google Scholar 

  49. Heap, T., Hogg, D.: Towards 3d hand tracking using a deformable model. In: FG 1996 (1996)

    Google Scholar 

  50. Herda, L., Fua, P., Plänkers, R., Boulic, R., Thalmann, D.: Skeleton-based motion capture for robust reconstruction of human motion. In: CA 2000: Proc. of the Computer Animation, p. 77 (2000)

    Google Scholar 

  51. Kakadiaris, I.A., Metaxas, D.: 3D human body model acquisition from multiple views. In: Proc. of ICCV 1995, pp. 618–623 (1995)

    Google Scholar 

  52. Kakadiaris, I.A., Metaxas, D.: Model-based estimation of 3D human motion with occlusion based on active multi-viewpoint selection. In: Proc. CVPR, pp. 81–87 (1996)

    Google Scholar 

  53. Kameda, Y., Minoh, M., Ikeda, K.: Three dimensional pose estimation of an articulated object from its silhouette image. In: Proc. of ACCV 1993, pp. 612–615 (1993)

    Google Scholar 

  54. Koch, R.: Dynamic 3D scene analysis through synthesis feedback control. PAMI 15(6), 556–568 (1993)

    Google Scholar 

  55. Lee, C.S., Elgammal, A.: Modeling view and posture manifolds for tracking. In: Proc. of ICCV 2007 (2007)

    Google Scholar 

  56. Lee, H.J., Chen, Z.: Determination of 3d human body postures from a single view 30(2), 148–168 (1985)

    Google Scholar 

  57. Loy, G., Eriksson, M., Sullivan, J., Carlsson, S.: Monocular 3d reconstruction of human motion in long action sequences. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 442–455. Springer, Heidelberg (2004)

    Google Scholar 

  58. MacCormick, J., Isard, M.: Partitioned sampling, articulated objects, and interface-quality hand tracking. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 3–19. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  59. Martinez, G.: 3D motion estimation of articulated objects for object-based analysis-synthesis coding (OBASC). In: VLBV 1995, pp. 175–199 (1995)

    Google Scholar 

  60. Matsuyama, T., Takai, T.: Generation, visualization, and editing of 3D video. In: Proc. of 3DPVT 2002, p. 234 (2002)

    Google Scholar 

  61. Matusik, W., Buehler, C., Raskar, R., Gortler, S.J., McMillan, L.: Image-based visual hulls. In: Proc. SIGGRAPH, pp. 369–374 (2000)

    Google Scholar 

  62. Metaxas, D., Terzopoulos, D.: Shape and nonrigid motion estimation through physics-based synthesis. IEEE PAMI 15(6), 580–591 (1993)

    Google Scholar 

  63. Meyer, D., Denzler, J.: Model based extraction of articulated objects in image sequences for gait analysis. In: Proc. of ICIP 1997, p. 78 (1997)

    Google Scholar 

  64. Mikić, I., Trivedi, M., Hunter, E., Cosman, P.: Human body model acquisition and tracking using voxel data. Int. J. Comput. Vision 53(3), 199–223 (2003)

    Article  Google Scholar 

  65. Mikić, I., Triverdi, M., Hunter, E., Cosman, P.: Articulated body posture estimation from multicamera voxel data. In: Proc. of CVPR, vol. 1, p. 455 (2001)

    Google Scholar 

  66. Mittal, A., Zhao, L., Davism, L.S.: Human body pose estimation using silhouette shape analysis. In: Proc. of AVSS, p. 263 (2003)

    Google Scholar 

  67. Moeslund, T.B., Granum, E.: A survey of computer vision-based human motion capture. CVIU 81(3), 231–268 (2001)

    MATH  Google Scholar 

  68. Moeslund, T.B., Hilton, A., Krüger, V.: A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 104(2), 90–126 (2006)

    Article  Google Scholar 

  69. Mori, G., Malik, J.: Recovering 3d human body configurations using shape contexts. IEEE PAMI 28(7), 1052–1062 (2006)

    Google Scholar 

  70. Mündermann, L., Corazza, S., Andriacchi, T.P.: Accurately measuring human movement using articulated icp with soft-joint constraints and a repository of articulated models. In: Proc. of CVPR (2007)

    Google Scholar 

  71. O’Rourke, J., Badler, N.I.: Model-based image analysis of human motion using constraint propagation. PAMI 2(6), 522–536 (1980)

    Google Scholar 

  72. Park, S.I., Hodgins, J.K.: Capturing and animating skin deformation in human motion. ACM Transactions on Graphics (SIGGRAPH 2006) 25(3), 881–889 (2006)

    Article  Google Scholar 

  73. Pentland, A., Horowitz, B.: Recovery of nonrigid motion and structure. IEEE PAMI 13(7), 730–742 (1991)

    Google Scholar 

  74. PhaseSpace, http://www.phasespace.com

  75. Plänkers, R., Fua, P.: Articulated soft objects for multiview shape and motion capture. IEEE PAMI 25(9), 1182–1187 (2003)

    Google Scholar 

  76. Poppe, R.: Vision-based human motion analysis: An overview. CVIU 108, 4–18 (2007)

    Google Scholar 

  77. Pritchard, D., Heidrich, W.: Cloth motion capture. In: Eurographics, pp. 263–271 (2003)

    Google Scholar 

  78. Rosenhahn, B., Brox, T., Kersting, U., Smith, D., Gurney, J., Klette, R.: A system for marker-less human motion estimation. Kuenstliche Intelligenz (KI) 1, 45–51 (2006)

    Google Scholar 

  79. Rosenhahn, B., Brox, T., Seidel, H.P.: Scaled motion dynamics for markerless motion capture. In: Proc. CVPR (2007)

    Google Scholar 

  80. Rosenhahn, B., Brox, T., Weickert, J.: Three-dimensional shape knowledge for joint image segmentation and pose tracking. In: IJCV, pp. 243–262 (2006)

    Google Scholar 

  81. Rosenhahn, B., Kersting, U., Powel, K., Seidel, H.P.: Cloth x-ray: Mocap of people wearing textiles. In: Franke, K., Müller, K.-R., Nickolay, B., Schäfer, R. (eds.) DAGM 2006. LNCS, vol. 4174, pp. 495–504. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  82. Salzmann, M., Ilic, S., Fua, P.: Physically valid shape parameterization for monocular 3-d deformable surface tracking. In: BMVC (2005)

    Google Scholar 

  83. Sand, P., McMillan, L., Popović, J.: Continuous capture of skin deformation. ACM Trans. Graph. 22(3), 578–586 (2003)

    Article  Google Scholar 

  84. Shakhnarovich, G., Viola, P., Darrell, T.: Fast pose estimation with parameter-sensitive hashing. In: Proc. of ECCV 2003, pp. 750–757 (2003)

    Google Scholar 

  85. Shinya, M.: Unifying measured point sequences of deforming objects. In: Proc. of 3DPVT, pp. 904–911 (2004)

    Google Scholar 

  86. Shum, H.Y., Chan, S.C., Kang, S.B.: Image-based rendering. Springer, Heidelberg (2007)

    Google Scholar 

  87. Sidenbladh, H., Black, M.J., Sigal, L.: Implicit probabilistic models of human motion for synthesis and tracking. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 784–800. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  88. Sidenbladh, H., Black, M.J., Fleet, D.J.: Stochastic tracking of 3D human figures using 2D image motion. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 702–718. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  89. Sminchisescu, C., Kanaujia, A., Metaxas, D.N.: Bm3e: Discriminative density propagation for visual tracking. IEEE PAMI 29(11), 2030–2044 (2007)

    Google Scholar 

  90. Sminchisescu, C., Triggs, B.: Kinematic jump processes for monocular 3d human tracking. In: Proc.of IEEE CVPR, pp. 69–76 (2003)

    Google Scholar 

  91. Starck, J., Hilton, A.: Model-based multiple view reconstruction of people. In: Proc. of ICCV, p. 915 (2003)

    Google Scholar 

  92. Starck, J., Hilton, A.: Spherical matching for temporal correspondence of non-rigid surfaces. In: IEEE ICCV, pp. 1387–1394 (2005)

    Google Scholar 

  93. Starck, J., Hilton, A.: Surface capture for performance based animation. IEEE CGAA 27(3), 21–31 (2007)

    Google Scholar 

  94. Stoll, C., Karni, Z., Rössl, C., Yamauchi, H., Seidel, H.P.: Template deformation for point cloud fitting. In: Symposium on Point-Based Graphics, pp. 27–35 (2006)

    Google Scholar 

  95. Theobalt, C., Magnor, M., Schüler, P., Seidel, H.P.: Combining 2d feature tracking and volume reconstruction for online video-based human motion capture. In: Proc. PG, pp. 96–103 (2002)

    Google Scholar 

  96. Torresani, L., Hertzmann, A.: Automatic non-rigid 3D modeling from video. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3022, pp. 299–312. Springer, Heidelberg (2004)

    Google Scholar 

  97. Urtasun, R., Fua, P.: 3D human body tracking using deterministic temporal motion models. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3023, pp. 92–106. Springer, Heidelberg (2004)

    Google Scholar 

  98. Vacchetti, L., Lepetit, V., Fua, P.: Stable real-time 3d tracking using online and offline information. IEEE Trans. Pattern Anal. Mach. Intell. 26(10), 1385–1391 (2004)

    Article  Google Scholar 

  99. Wand, M., Jenke, P., Huang, Q., Bokeloh, M., Guibas, L., Schilling, A.: Reconstruction of deforming geometry from time-varying point clouds. In: Proc. SGP, pp. 49–58 (2007)

    Google Scholar 

  100. Wang, P., Rehg, J.M.: A modular approach to the analysis and evaluation of particle filters for figure tracking. In: Proc. CVPR 2006, pp. 790–797 (2006)

    Google Scholar 

  101. Waschbüsch, M., Würmlin, S., Cotting, D., Sadlo, F., Gross, M.: Scalable 3D video of dynamic scenes. In: Proc. PG, pp. 629–638 (2005)

    Google Scholar 

  102. Wilburn, B., Joshi, N., Vaish, V., Talvala, E., Antunez, E., Barth, A., Adams, A., Horowitz, M., Levoy, M.: High performance imaging using large camera arrays. ACM TOG 24(3), 765–776 (2005)

    Google Scholar 

  103. Wuermlin, S., Lamboray, E., Staadt, O.G., Gross, M.H.: 3d video recorder. In: Proc. of Pacific Graphics 2002, pp. 325–334 (2002)

    Google Scholar 

  104. Xu, W., Zhou, K., Yu, Y., Tan, Q., Peng, Q., Guo, B.: Gradient domain editing of deforming mesh sequences. ACM TOG 26(3), 84 (2007)

    Google Scholar 

  105. Xu, X., Li, B.: Learning motion correlation for tracking articulated human body with a rao-blackwellised particle filter. In: Proc. ICCV (2007)

    Google Scholar 

  106. Yonemoto, S., Arita, D., Taniguchi, R.: Real-time human motion analysis and ik-based human figure control. In: Proc. of IEEE Workshop on Human Motion, pp. 149–154 (2000)

    Google Scholar 

  107. Zitnick, C.L., Kang, S.B., Uyttendaele, M., Winder, S., Szeliski, R.: High-quality video view interpolation using a layered representation. ACM TOG 23(3), 600–608 (2004)

    Google Scholar 

Download references

Authors
  1. Edilson de Aguiar
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

de Aguiar, E. (2010). Problem Statement and Preliminaries. In: Animation and Performance Capture Using Digitized Models. Cognitive Systems Monographs, vol 5. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-10316-2_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-10316-2_8

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-10315-5

  • Online ISBN: 978-3-642-10316-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation