Abstract
One main challenge in Augmented Reality (AR) applications is to keep track of video objects with their movement, orientation, size, and position accurately. This poses a challenging task to recover non-rigid shape and global pose in real-time AR applications. This paper proposes a novel two-stage scheme for online non-rigid shape recovery toward AR applications using Active Appearance Models (AAMs). First, we construct 3D shape models from AAMs offline, which do not involve processing of the 3D scan data. Based on the computed 3D shape models, we propose an efficient online algorithm to estimate both 3D pose and non-rigid shape parameters via local bundle adjustment for building up point correspondences. Our approach, without manual intervention, can recover the 3D non-rigid shape effectively from either real-time video sequences or single image. The recovered 3D pose parameters can be used for AR registrations. Furthermore, the facial feature can be tracked simultaneously, which is critical for many face related applications. We evaluate our algorithms on several video sequences. Promising experimental results demonstrate our proposed scheme is effective and significant for real-time AR applications.
Chapter PDF
Similar content being viewed by others
Keywords
- Augmented Reality
- Active Appearance Model
- Augmented Reality Application
- Promising Experimental Result
- IEEE CVPR
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Kato, H., Billinghurst, M.: Marker tracking and hmd calibration for a video-based augmented reality conferencing system. In: Proceedings of the 2nd IEEE and ACM International Workshop on Augmented Reality (2004)
Vacchetti, L., Lepetit, V., Fua, P.: Stable real-time 3d tracking using online and offline information. IEEE Trans. PAMI 26 (2004)
Cootes, T., Edwards, G., Taylo, C.: Active appearance models. IEEE Trans. PAMI 23 (2001)
Cootes, T., Kittipanyangam, P.: Comparing variations on the active appearance model algorithm. In: British Machine Vision Conference
Stegmann, M., Ersboll, B., Larsen, R.: Fame-a flexible appearance modeling environment. IEEE Trans. Medical Imaging 22 (2003)
Mittrapiyanumic, P., DeSouza, G., Kak, A.: Calculating the 3d-pose of rigid-objects using active appearance models. In: Proceedings of the International Conference in Robotics and Automation, vol. 5, pp. 5147–5152 (2004)
Ahlberg, J.: Using the active appearance algorithm for face and facial feature tracking. In: Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, pp. 68–72 (2001)
Xiao, J., Baker, S., Matthews, I., Kanade, T.: Real-time combined 2d+3d active appearance models. In: IEEE CVPR 2004, vol. 2, pp. 535–542 (2004)
Blanz, V., Vetter, T.: Face recognition based on fitting a 3d morphable model. IEEE Trans. PAMI 25 (2003)
Hou, X., Zhang, S.Z.L., Cheng, H., Direct, Q.: appearance models. In: IEEE CVPR 2001, vol. 1, pp. 828–833 (2001)
Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3d shape from image streams. In: IEEE CVPR 2000, vol. 2, pp. 690–696 (2000)
Torresani, L., Yang, D., Alexander, E., Bregler, C.: Tracking and modeling non-rigid objects with rank constraints. In: IEEE CVPR 2001, vol. 1, pp. 493–500 (2001)
Medioni, G., Kang, S.B.: Emerging topics in computer vision. Prentice-Hall, Englewood Cliffs (2004)
Nielson, G.: Scattered data modeling. IEEE Computer Graphics and Applications 13 (1993)
Gross, R., Matthews, I., Baker, S.: Generic vs. person specific active appearance models. In: British Machine Vision Conference (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhu, J., Hoi, S.C.H., Lyu, M.R. (2006). Real-Time Non-rigid Shape Recovery Via Active Appearance Models for Augmented Reality. In: Leonardis, A., Bischof, H., Pinz, A. (eds) Computer Vision – ECCV 2006. ECCV 2006. Lecture Notes in Computer Science, vol 3951. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11744023_15
Download citation
DOI: https://doi.org/10.1007/11744023_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33832-1
Online ISBN: 978-3-540-33833-8
eBook Packages: Computer ScienceComputer Science (R0)