Advertisement

Parametric Reshaping of Humans in Videos Incorporating Motion Retargeting

Conference paper
  • 1.2k Downloads
Part of the Communications in Computer and Information Science book series (CCIS, volume 841)

Abstract

We propose a system capable of changing the shape of humans in monocular video sequences. Initially, a 3D model is fit over each frame of the video sequence in a spatio-temporally coherent manner, using the feature points provided by the user in a semi-automatic interface and the silhouette correspondences obtained from background subtraction. The 3D morphable model learned from laser scans of different human subjects is used to generate a model having the shape parameters like height, weight, leg length, etc. specified by the user. The deformed model is then retargeted to transfer the semantics of the motion, like step size of the person. This retargeted model is used to perform a body-aware warping of the foreground of each frame. Finally, the warped foreground is composited over the inpainted background. Spatio-temporal consistency is achieved through the combination of automatic pose fitting and body-aware frame warping. Motion retargeting makes the system produce visually pleasing and natural results like the motion of a taller human is higher than that of the human before warping. We have demonstrated the results of shape changes on different subjects with a variety of actions.

Keywords

Video retouching Video editing Reshaping humans Application of motion retargeting 

Notes

Acknowledgements

We would like to thank University of Surrey (Gkalelis et al. [4]) for permitting us to use their video footages. We thank Hasler et al. [5] and Starck and Hilton [12] for making their datasets available. We also thank Newson et al. [10] for making their source code publicly available. We extend our gratitude to Ganesh and Aswinn for being the test subjects.

Supplementary material

466950_1_En_11_MOESM1_ESM.mp4 (12.5 mb)
Supplementary material 1 (mp4 12843 KB)

References

  1. 1.
    Bai, X., Wang, J., Simons, D., Sapiro, G.: Video SnapCut: robust video cutout using localized classifiers. ACM Trans. Graph. (TOG) 28(3) (2009)CrossRefGoogle Scholar
  2. 2.
    Baran, I., Popovic, J.: Automatic rigging and animation of 3D characters. ACM Trans. Graph. (TOG) 29(3) (2007)Google Scholar
  3. 3.
    Criminisi, A., Perez, P., Toyama, K.: A region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)CrossRefGoogle Scholar
  4. 4.
    Gkalelis, N., Kim, H., Hilton, A., Nikolaidis, N., Pitas, I.: The i3DPost multi-view and 3D human action/interaction. In: CVMP (2009)Google Scholar
  5. 5.
    Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., Seidel, H.-P.: A statistical model of human pose and body shape. In: Computer Graphics Forum (Proceedings of the Eurographics 2008), vol. 2 (2009)CrossRefGoogle Scholar
  6. 6.
    Jain, A., Thormahlen, T., Seidel, H.-P., Theobalt, C.: MovieReshape: tracking and reshaping of humans in videos. Trans. Graph. 29(6), 148:1–148:10 (2010)CrossRefGoogle Scholar
  7. 7.
    Kraevoy, V., Sheffer, A., Van De Panne, M.: Modelling from contour drawings. In: SBIM (2009)Google Scholar
  8. 8.
    Kumar, H., Arora, N., Dhaliwal, J.S., Kalra, P., Chaudhuri, P.: Improved interactive reshaping of humans in images. In: 21st International Conference on Computer Graphics, Visualization and Computer Vision (2013)Google Scholar
  9. 9.
    Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 228–242 (2008)CrossRefGoogle Scholar
  10. 10.
    Newson, A., Almansa, A., Fradet, M., Gousseau, Y., Perez, P.: Video inpainting of complex scenes. SIAM J. Imaging Sci. 17(4), 1993–2019 (2014)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Richter, M., Varanasi, K., Hasler, N., Theobalt, C.: Real-time reshaping of humans. In: International Conference on 3D Imaging, Data Processing, Visualization and Transmission (3DIMPVT) (2012)Google Scholar
  12. 12.
    Starck, J., Hilton, A.: Surface capture for performance based animation. IEEE Comput. Graph. Appl. 27, 21–31 (2007)CrossRefGoogle Scholar
  13. 13.
    Yang, Y., Yu, Y., Zhou, Y., Du, S., Davis, J., Yang, R.: Semantic parametric reshaping of human body models. In: Second International Conference on 3D Vision (2014)Google Scholar
  14. 14.
    Zhou, S., Fu, H., Liu, L., Cohen-Or, D., Han, X.: Parametric reshaping of human bodies in images. ACM Trans. Graph. (Proc. ACM SIGGRAPH) 29(3), 1–10 (2010). Article No. 126Google Scholar
  15. 15.
    Schaefer, S., McPhail, T., Warren, J.: Image deformation using moving least squares. ACM Trans. Graph. 25(3), 533–540 (2006)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.IIT DelhiDelhiIndia

Personalised recommendations