Accuracy and Performance Analysis of Time Coherent 3D Animation Reconstruction from RGB-D Video

  • Naveed Ahmed
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 746)


We present an accuracy and performance analysis of Time Coherent 3D Animation Reconstruction methods from RGB-D video. We analyze the existing methods that can reconstruct a time coherent 3D animation using RGB-D video. We also present a modified algorithm using only the RGB data that extends the analysis of existing methods. We show that using all the methods it is possible to reconstruct a time-coherent 3D animation using either only the color data, color and depth data, or only the depth data. We compare all the methods using a number of error measures and analyze the strength and weaknesses of each method in terms of their accuracy and runtime performance. Our analysis demonstrates that given RGB-D video data, it is possible to select the best algorithm for time coherent 3D animation reconstruction under a number of constraints in terms of the required accuracy and runtime performance.


3D animation Kinect Multi-view video Time coherence 3D reconstruction 


  1. 1.
    Carranza, J., Theobalt, C., Magnor, M.A., Seidel, H.P.: Free-viewpoint video of human actors. ACM Trans. Graph. 22(3), 569–577 (2003)CrossRefGoogle Scholar
  2. 2.
    Theobalt, C., Ahmed, N., Ziegler, G., Seidel, H.P.: High-quality reconstruction of virtual actors from multi-view video streams. IEEE Sig. Process. Mag. 24(6), 45–57 (2007)CrossRefGoogle Scholar
  3. 3.
    Starck, J., Hilton, A.: Surface capture for performance-based animation. IEEE Comput. Graph. App. 27(3), 21–31 (2007)CrossRefGoogle Scholar
  4. 4.
    de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.P., Thrun, S.: Performance capture from sparse multi-view video. ACM Trans. Graph. 27(3), 98 (2008)CrossRefGoogle Scholar
  5. 5.
    Vlasic, D., Baran, I., Matusik, W., Popovic, J.: Articulated mesh animation from multi-view silhouettes. ACM Trans. Graph. 27(3), 97 (2008)CrossRefGoogle Scholar
  6. 6.
    Ahmed, N., Theobalt, C., Rössl, C., Thrun, S., Seidel, H.P.: Dense correspondence finding for parametrization-free animation reconstruction from video. In: CVPR (2008)Google Scholar
  7. 7.
    Kim, Y.M., Chan, D., Theobalt, C., Thrun, S.: Design and calibration of a multi-view TOF sensor fusion system. In: CVPR Workshop (2008)Google Scholar
  8. 8.
    Castaneda, V., Mateus, D., Navab, N.: Stereo time-of-flight. In: ICCV (2011)Google Scholar
  9. 9.
    Kim, Y.M., Theobalt, C., Diebel, J., Kosecka, J., Micusik, B., Thrun, S.: Multi-view image and TOF sensor fusion for dense 3D reconstruction. In: 3DIM, Kyoto, Japan, pp. 1542–1549. IEEE (2009)Google Scholar
  10. 10.
    Microsoft: Kinect for microsoft windows and xbox 360, November 2010.
  11. 11.
    Berger, K., Ruhl, K., Schroeder, Y., Bruemmer, C., Scholz, A., Magnor, M.A.: Markerless motion capture using multiple color-depth sensors. In: VMV (2011)Google Scholar
  12. 12.
    Weiss, A., Hirshberg, D., Black, M.J.: Home 3D body scans from noisy image and range data. In: ICCV (2011)Google Scholar
  13. 13.
    Baak, A., Muller, M., Bharaj, G., Seidel, H.P., Theobalt, C.: A data-driven approach for real-time full body pose reconstruction from a depth camera. In: ICCV (2011)Google Scholar
  14. 14.
    Girshick, R., Shotton, J., Kohli, P., Criminisi, A., Fitzgibbon, A.: Efficient regression of general-activity human poses from depth images. In: ICCV (2011)Google Scholar
  15. 15.
    Ye, M., Wang, X., Yang, R., Ren, L., Pollefeys, M.: Accurate 3D pose estimation from a single depth image. In: Proceedings of the 2011 International Conference on Computer Vision. ICCV 2011, Washington, DC, USA, pp. 731–738. IEEE Computer Society (2011)Google Scholar
  16. 16.
    Ahmed, N.: A system for 360 degree acquisition and 3D animation reconstruction using multiple RGB-D cameras. In: Proceedings of the 25th International Conference on Computer Animation and Social Agents (CASA). Casa 2012 (2012)Google Scholar
  17. 17.
    Ahmed, N., Junejo, I.: A system for 3D video acquisition and spatio-temporally coherent 3D animation reconstruction using multiple RGB-D cameras. Int. J. Sig. Process. Image Process. Pattern Recogn. 6(2), 113–128 (2013)Google Scholar
  18. 18.
    Ahmed, N., Khalifa, S.: Time-coherent 3D animation reconstruction from RGB-D video. Sig. Image Video Process. 10(4), 783–790 (2016)CrossRefGoogle Scholar
  19. 19.
    Ahmed, N., Junejo, I.: Using multiple RGB-D cameras for 3D video acquisition and spatio-temporally coherent 3D animation reconstruction. Int. J. Comput. Theory Eng. 6(6), 447–450 (2014)CrossRefGoogle Scholar
  20. 20.
    Ahmed, N.: Multi-view RGB-D synchronized video acquisition and temporally coherent 3D animation reconstruction using multiple kinects. In: Feature Detectors and Motion Detection in Video Processing, pp. 142–163. IGI Global (2016)Google Scholar
  21. 21.
    Lowe, D.G.: Object recognition from local scale-invariant features. In: ICCV, pp. 1150–1157 (1999)Google Scholar
  22. 22.
    Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput. Vis. Image Underst. 110(3), 346–359 (2008)CrossRefGoogle Scholar
  23. 23.
    Scovanner, P., Ali, S., Shah, M.: A 3-dimensional sift descriptor and its application to action recognition. In: Proceedings of the 15th ACM International Conference on Multimedia. MM 2007, pp. 357–360. ACM, New York (2007)Google Scholar
  24. 24.
    Aldoma, A., Tombari, F., Rusu, R.B., Vincze, M.: In: OUR-CVFH - Oriented, Unique and Repeatable Clustered Viewpoint Feature Histogram for Object Recognition and 6DOF Pose Estimation. Springer, Heidelberg (2012)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of SharjahSharjahUAE

Personalised recommendations