Parametric Reshaping of Humans in Videos Incorporating Motion Retargeting
- 1.2k Downloads
We propose a system capable of changing the shape of humans in monocular video sequences. Initially, a 3D model is fit over each frame of the video sequence in a spatio-temporally coherent manner, using the feature points provided by the user in a semi-automatic interface and the silhouette correspondences obtained from background subtraction. The 3D morphable model learned from laser scans of different human subjects is used to generate a model having the shape parameters like height, weight, leg length, etc. specified by the user. The deformed model is then retargeted to transfer the semantics of the motion, like step size of the person. This retargeted model is used to perform a body-aware warping of the foreground of each frame. Finally, the warped foreground is composited over the inpainted background. Spatio-temporal consistency is achieved through the combination of automatic pose fitting and body-aware frame warping. Motion retargeting makes the system produce visually pleasing and natural results like the motion of a taller human is higher than that of the human before warping. We have demonstrated the results of shape changes on different subjects with a variety of actions.
KeywordsVideo retouching Video editing Reshaping humans Application of motion retargeting
We would like to thank University of Surrey (Gkalelis et al. ) for permitting us to use their video footages. We thank Hasler et al.  and Starck and Hilton  for making their datasets available. We also thank Newson et al.  for making their source code publicly available. We extend our gratitude to Ganesh and Aswinn for being the test subjects.
- 2.Baran, I., Popovic, J.: Automatic rigging and animation of 3D characters. ACM Trans. Graph. (TOG) 29(3) (2007)Google Scholar
- 4.Gkalelis, N., Kim, H., Hilton, A., Nikolaidis, N., Pitas, I.: The i3DPost multi-view and 3D human action/interaction. In: CVMP (2009)Google Scholar
- 7.Kraevoy, V., Sheffer, A., Van De Panne, M.: Modelling from contour drawings. In: SBIM (2009)Google Scholar
- 8.Kumar, H., Arora, N., Dhaliwal, J.S., Kalra, P., Chaudhuri, P.: Improved interactive reshaping of humans in images. In: 21st International Conference on Computer Graphics, Visualization and Computer Vision (2013)Google Scholar
- 11.Richter, M., Varanasi, K., Hasler, N., Theobalt, C.: Real-time reshaping of humans. In: International Conference on 3D Imaging, Data Processing, Visualization and Transmission (3DIMPVT) (2012)Google Scholar
- 13.Yang, Y., Yu, Y., Zhou, Y., Du, S., Davis, J., Yang, R.: Semantic parametric reshaping of human body models. In: Second International Conference on 3D Vision (2014)Google Scholar
- 14.Zhou, S., Fu, H., Liu, L., Cohen-Or, D., Han, X.: Parametric reshaping of human bodies in images. ACM Trans. Graph. (Proc. ACM SIGGRAPH) 29(3), 1–10 (2010). Article No. 126Google Scholar