Face Tracking for Spatially Aware Mobile User Interfaces

  • Jari Hannuksela
  • Pekka Sangi
  • Markus Turtinen
  • Janne Heikkilä
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5099)


This paper introduces a new face tracking approach for controlling user interfaces in hand-held mobile devices. The proposed method detects the face and the eyes of the user by employing a method based on local texture features and boosting. An extended Kalman filter combines local motion features extracted from the face region and the detected eye positions to estimate the 3-D position and orientation of the camera with respect to the face. The camera position is used as an input for the spatially aware user interface. Experimental results on real image sequences captured with a camera-equipped mobile phone validate the feasibility of the method.


facial feature extraction motion analysis pose estimation 


  1. 1.
    Fitzmaurice, G.W.: Situated information spaces and spatially aware palmtop computers. Communications of the ACM 36(7), 38–49 (1993)CrossRefGoogle Scholar
  2. 2.
    Yee, K.P.: Peephole displays: pen interaction on spatially aware handheld computers. In: Proc. of the SIGCHI conference on human factors in computing systems, pp. 1–8 (2003)Google Scholar
  3. 3.
    Hinckley, K., Pierce, J., Sinclair, M., Horvitz, E.: Sensing techniques for mobile interaction. In: Proc. 13th ACM symposium on User Interface Software and Technology, pp. 91–100 (2000)Google Scholar
  4. 4.
    Haro, A., Mori, K., Capin, T., Wilkinson, S.: Mobile camera-based user interaction. In: Proc. of IEEE Workshop on Human-Computer Interaction, pp. 79–89 (2005)Google Scholar
  5. 5.
    Azarbayejani, A., Starner, T., Horowitz, B., Pentland, A.: Visually controlled graphics. IEEE Transactions on Pattern Analysis and Machine Intelligence 15(6), 602–605 (1993)CrossRefGoogle Scholar
  6. 6.
    Black, M.J., Yacoob, Y.: Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion. In: Proc. IEEE International Conference on Computer Vision, pp. 374–381 (1995)Google Scholar
  7. 7.
    Basu, S., Essa, I., Pentland, A.: Motion regularization for model-based head tracking. In: Proceedings of the 13th IEEE International Conf. on Pattern Recognition, pp. 611–616 (1996)Google Scholar
  8. 8.
    La Cascia, M., Sclaroff, S., Athitsos, V.: Fast, reliable head tracking under varying illumination: An approach based on registration of texture-mapped 3d models. IEEE Trans. on Pattern Analysis and Machine Intelligence 22(4), 322–336 (2000)CrossRefGoogle Scholar
  9. 9.
    Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7), 971–987 (2002)CrossRefGoogle Scholar
  10. 10.
    Freund, Y., Schapire, R.: A decision-theoretic generalization of on-line learning and an application to boosting. In: Europ. Conf. on Computational Learning Theory, pp. 23–37 (1995)Google Scholar
  11. 11.
    Hadid, A., Zhao, G., Ahonen, T., Pietikäinen, M.: Face analysis using local binary patterns. In: Mirmehdi, M. (ed.) Handbook of Texture Analysis. Imperial College Press (2007)Google Scholar
  12. 12.
    Hadid, A., Pietikäinen, M., Ahonen, T.: A discriminative feature space for detecting and recognizing faces. In: IEEE Conference on Computer Vision and Pattern Recognition (2004)Google Scholar
  13. 13.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE International Conf. on Computer Vision and Pattern Recognition, pp. 511–518 (2001)Google Scholar
  14. 14.
    Sangi, P., Hannuksela, J., Heikkilä, J.: Global motion estimation using block matching with uncertainty analysis. In: 15th European Signal Processing Conference (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Jari Hannuksela
    • 1
  • Pekka Sangi
    • 1
  • Markus Turtinen
    • 1
  • Janne Heikkilä
    • 1
  1. 1.Machine Vision Group, Infotech Oulu, Department of Electrical and Information EngineeringUniversity of OuluFinland

Personalised recommendations