Advertisement

Eye Gaze Correction with a Single Webcam Based on Eye-Replacement

  • Yalun Qin
  • Kuo-Chin Lien
  • Matthew Turk
  • Tobias Höllerer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9474)

Abstract

In traditional video conferencing systems, it is impossible for users to have eye contact when looking at the conversation partner’s face displayed on the screen, due to the disparity between the locations of the camera and the screen. In this work, we implemented a gaze correction system that can automatically maintain eye contact by replacing the eyes of the user with the direct looking eyes (looking directly into the camera) captured in the initialization stage. Our real-time system has good robustness against different lighting conditions and head poses, and it provides visually convincing and natural results while relying only on a single webcam that can be positioned almost anywhere around the screen.

Keywords

Face Tracker Laplacian Pyramid Destination Image Videoconferencing System Correction Zone 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

This work was supported in part by Citrix Online and by NSF Grants IIS-1423676 and IIS-1219261.

References

  1. 1.
    Jones, A., Lang, M., Fyffe, G., Yu, X., Busch, J., McDowall, I., Bolas, M., Debevec, P.: Achieving eye contact in a one-to-many 3d video teleconferencing system. ACM Trans. Graph. (TOG) 28, 64:1–64:8 (2009)CrossRefGoogle Scholar
  2. 2.
    Okada, K.I., Maeda, F., Ichikawaa, Y., Matsushita, Y.: Multiparty videoconferencing at virtual social distance: Majic design. In: Proceedings of the 1994 ACM Conference on Computer Supported Cooperative Work, pp. 385–393. ACM (1994)Google Scholar
  3. 3.
    Kuster, C., Popa, T., Bazin, J.C., Gotsman, C., Gross, M.: Gaze correction for home video conferencing. ACM Trans. Graph. (TOG) 31, 174 (2012)CrossRefGoogle Scholar
  4. 4.
    Zhu, J., Yang, R., Xiang, X.: Eye contact in video conference via fusion of time-of-flight depth sensor and stereo. 3D Res. 2, 1–10 (2011)CrossRefGoogle Scholar
  5. 5.
    Criminisi, A., Shotton, J., Blake, A., Torr, P.H.: Gaze manipulation for one-to-one teleconferencing. In: Ninth IEEE International Conference on Computer Vision, 2003, Proceedings, pp. 191–198. IEEE (2003)Google Scholar
  6. 6.
    Yang, R., Zhang, Z.: Eye gaze correction with stereovision for video-teleconferencing. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part II. LNCS, vol. 2351, pp. 479–494. Springer, Heidelberg (2002) CrossRefGoogle Scholar
  7. 7.
    Cham, T.J., Krishnamoorthy, S., Jones, M.: Analogous view transfer for gaze correction in video sequences. In: 7th International Conference on Control, Automation, Robotics and Vision, 2002. ICARCV 2002, vol. 3, pp. 1415–1420. IEEE (2002)Google Scholar
  8. 8.
    Yip, B., Jin, J.S.: An effective eye gaze correction operation for video conference using antirotation formulas. In: Proceedings of the 2003 Joint Conference of the Fourth International Conference on Information, Communications and Signal Processing and Fourth Pacific Rim Conference on Multimedia, vol. 2, pp. 699–703. IEEE (2003)Google Scholar
  9. 9.
    Gemmell, J., Toyama, K., Zitnick, C.L., Kang, T., Seitz, S.: Gaze awareness for video-conferencing: a software approach. IEEE Multimedia 7, 26–35 (2000)Google Scholar
  10. 10.
    Gemmell, J., Zhu, D.: Implementing gaze-corrected videoconferencing. In: Communications, Internet, and Information Technology, pp. 382–387 (2002)Google Scholar
  11. 11.
    Giger, D., Bazin, J.C., Kuster, C., Popa, T., Gross, M.: Gaze correction with a single webcam. In: 2014 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2014)Google Scholar
  12. 12.
    Hundt, E., Riegel, T., Schwartzel, H., Ziegler, M.: Correction of the gaze direction for a videophone, US Patent 5,499,303 (1996)Google Scholar
  13. 13.
    Jerald, J., Daily, M.: Eye gaze correction for videoconferencing. In: Proceedings of the 2002 Symposium on Eye Tracking Research & Applications, pp. 77–81. ACM (2002)Google Scholar
  14. 14.
    Weiner, D., Kiryati, N.: Virtual gaze redirection in face images. In: 12th International Conference on Image Analysis and Processing, 2003, Proceedings, pp. 76–81. IEEE (2003)Google Scholar
  15. 15.
    Wolf, L., Freund, Z., Avidan, S.: An eye for an eye: a single camera gaze-replacement method. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 817–824. IEEE (2010)Google Scholar
  16. 16.
    Kononenko, D., Lempitsky, V.: Learning to look up: realtime monocular gaze correction using machine learning. In: Computer Vision and Pattern Recognition (CVPR). IEEE (2015)Google Scholar
  17. 17.
    Smith, B.A., Yin, Q., Feiner, S.K., Nayar, S.K.: Gaze locking: passive eye contact detection for human-object interaction. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, pp. 271–280. ACM (2013)Google Scholar
  18. 18.
    Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 532–539. IEEE (2013)Google Scholar
  19. 19.
    Basri, R., Jacobs, D.W.: Lambertian reflectance and linear subspaces. IEEE Trans. Pattern Anal. Mach. Intell. 25, 218–233 (2003)CrossRefGoogle Scholar
  20. 20.
    Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., Nayar, S.K.: Face swapping: automatically replacing faces in photographs. ACM Trans. Graph. (TOG) 27, 39 (2008)CrossRefGoogle Scholar
  21. 21.
    Burt, P.J., Adelson, E.H.: A multiresolution spline with application to image mosaics. ACM Trans. Graph. (TOG) 2, 217–236 (1983)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Yalun Qin
    • 1
  • Kuo-Chin Lien
    • 1
  • Matthew Turk
    • 1
  • Tobias Höllerer
    • 1
  1. 1.University of CaliforniaSanta BarbaraUSA

Personalised recommendations