Skip to main content

Advertisement

SpringerLink
Log in

A color-based face tracking algorithm for enhancing interaction with mobile devices

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

A color-based face tracking algorithm is proposed to be used as a human-computer interaction tool on mobile devices. The solution provides a natural means of interaction enabling a motion parallax effect in applications. The algorithm considers the characteristics of mobile use-constrained computational resources and varying environmental conditions. The solution is based on color comparisons and works on images gathered from the front camera of a device. In addition to color comparisons, the coherency of the facial pixels is considered in the algorithm. Several applications are also demonstrated in this work, which use the face position to determine the viewpoint in a virtual scene, or for browsing large images. The accuracy of the system is tested under different environmental conditions such as lighting and background, and the performance of the system is measured in different types of mobile devices. According to these measurements the system allows for accurate (7% RMS error) face tracking in real time (20–100 fps).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Barnard, M., Hannuksela, J., Sangi, P., Heikkilä, J.: A vision based motion interface for mobile phones. In: Proc. of 5th International Conference on Computer Vision Systems (ICVS), Bielefeld, Germany (2007)

  2. Bradski, G.R.: Computer vision face tracking for use in a perceptual user interface. Intel Technol. J. 95–103 (1998)

  3. Brand, J., Mason, J.: A comparative assessment of three approaches to pixel-level human skin detection. In: Proc. of IEEE International Conf. Pattern Recognition, vol. 1, pp. 1056–1059 (2000)

  4. Bulbul, A., Cipiloglu, Z., Capin, T.: A face tracking algorithm for user interaction in mobile devices. In: Proc. of Cyberworlds, International Conference, pp. 385–390 (2009)

  5. Capin, T., Haro, A., Setlur, V., Wilkinson, S.: Camera-Based Virtual Environment Interaction on Mobile Devices, Lecture Notes in Computer Science, vol. 4263, pp. 765–773. Springer, Berlin (2006). ISBN: 9783540472421

    Book  Google Scholar 

  6. Capin, T., Pulli, K., Akenine-Möller, T.: The state of the art in mobile graphics research. IEEE Comput. Graph. Appl. 28(4), 74–84 (2008)

    Article  Google Scholar 

  7. Chai, D., Ngan, K.N.: Face segmentation using skin color map in videophone applications. IEEE Trans. Circuits Syst. Video Technol. 9(4), 551–564 (1999)

    Article  Google Scholar 

  8. Hannuksela, J., Huttunen, S., Sangi, P., Heikkilä, J.: Motion-based finger tracking for user interaction with mobile phones. In: Proc. of 4th European Conference on Visual Media Production (CVMP). London, UK (2007)

  9. Hannuksela, J., Sangi, P., Heikkilä, J.: A Vision-based approach for controlling user interfaces of mobile devices. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Workshop on Vision for Human-Computer Interaction (V4HCI), vol. 6, p. 71, San Diego, CA (2005)

  10. Haro, A., Mori, K., Capin, T., Wilkinson, S.: Mobile camera-based user interaction. In: Proc. of ICCV-HCI 2005, pp. 79–89 (2005)

  11. Hjelmas, B.K.L.E.: Face detection: a survey. Comput. Vis. Image Underst. 3(3), 236–274 (2001)

    Article  Google Scholar 

  12. Home of the Xith3D Project. Xith3D.org. http://xith.org/. Accessed 27 October 2009

  13. Hunke, M., Waibel, A.: Face locating and tracking for human-computer interaction. In: Proc. of the 28th Asilomar Conf. on Signals, Systems and Computers, vol. 2, pp. 1277–1281 (1994)

  14. Jaimes, A., Sebe, N.: Multimodal human computer interaction: a survey. In: Proc. of 11th IEEE International Workshop Human Computer Interaction (HCI) (2005)

  15. Kakumanu, P., Makrogiannis, S., Bourbakis, N.: A survey of skin colormodeling and detectionmethods. Pattern Recognit. 40, 1106–1122 (2007)

    Article  MATH  Google Scholar 

  16. Shirley, P.: Fundamentals of Computer Graphics. AK Peters, Natick (2002)

    MATH  Google Scholar 

  17. Siriluck, W., Kamolphiwong, S., Kamolphiwong, T., Sae-Whong, S.: Blink and click. In: Proc. of the 1st International Convention on Rehabilitation Engineering & Assistive Technology: in Conjunction with 1st Tan Tock Seng Hospital Neurorehabilitation Meeting, pp. 43–46 (2007)

  18. Vezhnevets, V., Sazonov, V., Andreeva, A.: A survey on pixel-based skin color detection techniques. In: Proc. of GRAPHICON03, pp. 85–92 (2003)

  19. Ware, C.: Space Perception and the Display of Data in Space. Information Visualization: Perception for Design. Morgan Kauffman, San Mateo (2004)

    Google Scholar 

  20. Yee, K.P.: Peephole displays: Pen interaction on spatially aware hand-held computers. In: Proc. of CHI 2003, pp. 1–8 (2003)

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Bilkent University, Ankara, Turkey

    Abdullah Bulbul, Zeynep Cipiloglu & Tolga Capin

Authors
  1. Abdullah Bulbul
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zeynep Cipiloglu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tolga Capin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abdullah Bulbul.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bulbul, A., Cipiloglu, Z. & Capin, T. A color-based face tracking algorithm for enhancing interaction with mobile devices. Vis Comput 26, 311–323 (2010). https://doi.org/10.1007/s00371-010-0419-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-010-0419-0

Keywords

Search

Navigation