ECCV 2006: Computer Vision – ECCV 2006 pp 186-197 | Cite as
Real-Time Non-rigid Shape Recovery Via Active Appearance Models for Augmented Reality
Abstract
One main challenge in Augmented Reality (AR) applications is to keep track of video objects with their movement, orientation, size, and position accurately. This poses a challenging task to recover non-rigid shape and global pose in real-time AR applications. This paper proposes a novel two-stage scheme for online non-rigid shape recovery toward AR applications using Active Appearance Models (AAMs). First, we construct 3D shape models from AAMs offline, which do not involve processing of the 3D scan data. Based on the computed 3D shape models, we propose an efficient online algorithm to estimate both 3D pose and non-rigid shape parameters via local bundle adjustment for building up point correspondences. Our approach, without manual intervention, can recover the 3D non-rigid shape effectively from either real-time video sequences or single image. The recovered 3D pose parameters can be used for AR registrations. Furthermore, the facial feature can be tracked simultaneously, which is critical for many face related applications. We evaluate our algorithms on several video sequences. Promising experimental results demonstrate our proposed scheme is effective and significant for real-time AR applications.
Keywords
Augmented Reality Active Appearance Model Augmented Reality Application Promising Experimental Result IEEE CVPRPreview
Unable to display preview. Download preview PDF.
References
- 1.Kato, H., Billinghurst, M.: Marker tracking and hmd calibration for a video-based augmented reality conferencing system. In: Proceedings of the 2nd IEEE and ACM International Workshop on Augmented Reality (2004)Google Scholar
- 2.Vacchetti, L., Lepetit, V., Fua, P.: Stable real-time 3d tracking using online and offline information. IEEE Trans. PAMI 26 (2004)Google Scholar
- 3.Cootes, T., Edwards, G., Taylo, C.: Active appearance models. IEEE Trans. PAMI 23 (2001)Google Scholar
- 4.Cootes, T., Kittipanyangam, P.: Comparing variations on the active appearance model algorithm. In: British Machine Vision ConferenceGoogle Scholar
- 5.Stegmann, M., Ersboll, B., Larsen, R.: Fame-a flexible appearance modeling environment. IEEE Trans. Medical Imaging 22 (2003)Google Scholar
- 6.Mittrapiyanumic, P., DeSouza, G., Kak, A.: Calculating the 3d-pose of rigid-objects using active appearance models. In: Proceedings of the International Conference in Robotics and Automation, vol. 5, pp. 5147–5152 (2004)Google Scholar
- 7.Ahlberg, J.: Using the active appearance algorithm for face and facial feature tracking. In: Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, pp. 68–72 (2001)Google Scholar
- 8.Xiao, J., Baker, S., Matthews, I., Kanade, T.: Real-time combined 2d+3d active appearance models. In: IEEE CVPR 2004, vol. 2, pp. 535–542 (2004)Google Scholar
- 9.Blanz, V., Vetter, T.: Face recognition based on fitting a 3d morphable model. IEEE Trans. PAMI 25 (2003)Google Scholar
- 10.Hou, X., Zhang, S.Z.L., Cheng, H., Direct, Q.: appearance models. In: IEEE CVPR 2001, vol. 1, pp. 828–833 (2001)Google Scholar
- 11.Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3d shape from image streams. In: IEEE CVPR 2000, vol. 2, pp. 690–696 (2000)Google Scholar
- 12.Torresani, L., Yang, D., Alexander, E., Bregler, C.: Tracking and modeling non-rigid objects with rank constraints. In: IEEE CVPR 2001, vol. 1, pp. 493–500 (2001)Google Scholar
- 13.Medioni, G., Kang, S.B.: Emerging topics in computer vision. Prentice-Hall, Englewood Cliffs (2004)Google Scholar
- 14.Nielson, G.: Scattered data modeling. IEEE Computer Graphics and Applications 13 (1993)Google Scholar
- 15.Gross, R., Matthews, I., Baker, S.: Generic vs. person specific active appearance models. In: British Machine Vision Conference (2004)Google Scholar