Abstract
We present a novel algorithm for the markerless tracking of deforming surfaces such as faces. We acquire a sequence of 3D scans along with color images at 40Hz. The data is then represented by implicit surface and color functions, using a novel partition-of-unity type method of efficiently combining local regressors using nearest neighbor searches. Both these functions act on the 4D space of 3D plus time, and use temporal information to handle the noise in individual scans. After interactive registration of a template mesh to the first frame, it is then automatically deformed to track the scanned surface, using the variation of both shape and color as features in a dynamic energy minimization problem. Our prototype system yields high-quality animated 3D models in correspondence, at a rate of approximately twenty seconds per timestep. Tracking results for faces and other objects are presented.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Zhang, L., Snavely, N., Curless, B., Seitz, S.M.: Spacetime faces: High-resolution capture for modeling and animation. In: ACM SIGGRAPH (August 2004)
Borshukov, G., Lewis, J.P.: Realistic human face rendering for the matrix reloaded. In: SIGGRAPH 2003 Sketches. ACM Press, New York (2003)
Wand, M., Jenke, P., Huang, Q., Bokeloh, M., Guibas, L., Schilling, A.: Reconstruction of deforming geometry from time-varying point clouds. In: SGP 2007: Proc. fifth Eurographics symp. on Geometry processing, Aire-la-Ville, Switzerland, ACM, Eurographics Association, pp. 49–58 (2007)
Huang, X., Zhang, S., Wang, Y., Metaxas, D., Samaras, D.: A hierarchical framework for high resolution facial expression tracking. In: Articulated and non-rigid motion, Washington, DC, USA, vol. 1. IEEE Computer Society, Los Alamitos (2004)
Ohtake, Y., Belyaev, A., Alexa, M., Turk, G., Seidel, H.P.: Multi-level partition of unity implicits. ACM Trans. on Graphics 22(3), 463–470 (2003)
Desbrun, M., Meyer, M., Schröder, P., Barr, A.H.: Discrete differential-geometry operators for triangulated 2-manifolds. VisMath 2, 35–57 (2002)
Kazhdan, M., Bolitho, M., Hoppe, H.: Poisson surface reconstruction. In: SGP 2006: Proceedings of the fourth Eurographics symposium on Geometry processing, Aire-la-Ville, Switzerland, Switzerland, ACM, Eurographics Association, pp. 61–70 (2006)
Botsch, M., Kobbelt, L.: An intuitive framework for real-time freeform modeling. In: SIGGRAPH, pp. 630–634. ACM, New York (2004)
Botsch, M., Sorkine, O.: On linear variational surface deformation methods. IEEE Trans. Visualization and Computer Graphics 14(1), 213–230 (2008)
Merkwirth, C., Parlitz, U., Lauterborn, W.: Fast nearest neighbor searching for nonlinear signal processing. Phys. Rev. E 62(2), 2089–2097 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Walder, C., Breidt, M., Bülthoff, H., Schölkopf, B., Curio, C. (2009). Markerless 3D Face Tracking. In: Denzler, J., Notni, G., Süße, H. (eds) Pattern Recognition. DAGM 2009. Lecture Notes in Computer Science, vol 5748. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03798-6_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-03798-6_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03797-9
Online ISBN: 978-3-642-03798-6
eBook Packages: Computer ScienceComputer Science (R0)