Advertisement

Appearance-Based Gaze Estimation Using Dilated-Convolutions

  • Zhaokang ChenEmail author
  • Bertram E. Shi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11366)

Abstract

Appearance-based gaze estimation has attracted more and more attention because of its wide range of applications. The use of deep convolutional neural networks has improved the accuracy significantly. In order to improve the estimation accuracy further, we focus on extracting better features from eye images. Relatively large changes in gaze angles may result in relatively small changes in eye appearance. We argue that current architectures for gaze estimation may not be able to capture such small changes, as they apply multiple pooling layers or other downsampling layers so that the spatial resolution of the high-level layers is reduced significantly. To evaluate whether the use of features extracted at high resolution can benefit gaze estimation, we adopt dilated-convolutions to extract high-level features without reducing spatial resolution. In cross-subject experiments on the Columbia Gaze dataset for eye contact detection and the MPIIGaze dataset for 3D gaze vector regression, the resulting Dilated-Nets achieve significant (up to 20.8%) gains when compared to similar networks without dilated-convolutions. Our proposed Dilated-Net achieves state-of-the-art results on both the Columbia Gaze and the MPIIGaze datasets.

Keywords

Appearance-based gaze estimation Dilated-convolutions 

References

  1. 1.
    Baltrusaitis, T., Zadeh, A., Lim, Y.C., Morency, L.P.: OpenFace 2.0: facial behavior analysis toolkit. In: IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, pp. 59–66. IEEE (2018)Google Scholar
  2. 2.
    Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 27 (2011)CrossRefGoogle Scholar
  3. 3.
    Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXiv preprint arXiv:1412.7062 (2014)
  4. 4.
    Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018)CrossRefGoogle Scholar
  5. 5.
    Chen, Z., Shi, B.E.: Using variable dwell time to accelerate gaze-based web browsing with two-step selection. Int. J. Hum.-Comput. Interact. 35, 240–255 (2018)CrossRefGoogle Scholar
  6. 6.
    Deng, H., Zhu, W.: Monocular free-head 3D gaze tracking with deep learning and geometry constraints. In: IEEE International Conference on Computer Vision, pp. 3162–3171. IEEE (2017)Google Scholar
  7. 7.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)Google Scholar
  8. 8.
    Funes Mora, K.A., Monay, F., Odobez, J.M.: EYEDIAP: a database for the development and evaluation of gaze estimation algorithms from RGB and RGB-D cameras. In: ACM Symposium on Eye Tracking Research & Applications, pp. 255–258. ACM (2014)Google Scholar
  9. 9.
    George, A., Routray, A.: Real-time eye gaze direction classification using convolutional neural network. In: International Conference on Signal Processing and Communications, pp. 1–5. IEEE (2016)Google Scholar
  10. 10.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385 (2015)
  11. 11.
    Hoppe, S., Loetscher, T., Morey, S.A., Bulling, A.: Eye movements during everyday behavior predict personality traits. Front. Hum. Neurosci. 12, 105 (2018)CrossRefGoogle Scholar
  12. 12.
    Huang, C.M., Mutlu, B.: Anticipatory robot control for efficient human-robot collaboration. In: ACM/IEEE International Conference on Human Robot Interaction, pp. 83–90. IEEE (2016)Google Scholar
  13. 13.
    Ioffe, S.: Batch renormalization: towards reducing minibatch dependence in batch-normalized models. In: Advances in Neural Information Processing Systems, pp. 1942–1950. MIT Press (2017)Google Scholar
  14. 14.
    King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10(Jul), 1755–1758 (2009)Google Scholar
  15. 15.
    Krafka, K., et al.: Eye tracking for everyone. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2176–2184. IEEE (2016)Google Scholar
  16. 16.
    Lin, M., Chen, Q., Yan, S.: Network in network. arXiv preprint arXiv:1312.4400 (2013)
  17. 17.
    van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(Nov), 2579–2605 (2008)Google Scholar
  18. 18.
    Parekh, V., Subramanian, R., Jawahar, C.V.: Eye contact detection via deep neural networks. In: Stephanidis, C. (ed.) HCI 2017. CCIS, vol. 713, pp. 366–374. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-58750-9_51CrossRefGoogle Scholar
  19. 19.
    Patney, A., et al.: Towards foveated rendering for gaze-tracked virtual reality. ACM Trans. Graph. 35(6), 179 (2016)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Pi, J., Shi, B.E.: Probabilistic adjustment of dwell time for eye typing. In: International Conference on Human System Interactions, pp. 251–257. IEEE (2017)Google Scholar
  21. 21.
    Ranjan, R., De Mello, S., Kautz, J.: Light-weight head pose invariant gaze tracking. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2156–2164. IEEE (2018)Google Scholar
  22. 22.
    Schneider, T., Schauerte, B., Stiefelhagen, R.: Manifold alignment for person independent appearance-based gaze estimation. In: International Conference on Pattern Recognition, pp. 1167–1172. IEEE (2014)Google Scholar
  23. 23.
    Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., Webb, R.: Learning from simulated and unsupervised images through adversarial training. In: IEEE International Conference on Computer Vision, pp. 2242–2251. IEEE (2017)Google Scholar
  24. 24.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  25. 25.
    Smith, B.A., Yin, Q., Feiner, S.K., Nayar, S.K.: Gaze locking: passive eye contact detection for human-object interaction. In: ACM Symposium on User Interface Software and Technology, pp. 271–280. ACM (2013)Google Scholar
  26. 26.
    Su, H., Qi, C.R., Li, Y., Guibas, L.J.: Render for CNN: viewpoint estimation in images using CNNs trained with rendered 3D model views. In: IEEE International Conference on Computer Vision, pp. 2686–2694 (2015)Google Scholar
  27. 27.
    Sugano, Y., Matsushita, Y., Sato, Y.: Learning-by-synthesis for appearance-based 3D gaze estimation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1821–1828. IEEE (2014)Google Scholar
  28. 28.
    Wang, H., Pi, J., Qin, T., Shen, S., Shi, B.E.: SLAM-based localization of 3D gaze using a mobile eye tracker. In: ACM Symposium on Eye Tracking Research & Applications, p. 65. ACM (2018)Google Scholar
  29. 29.
    Wang, P., et al.: Understanding convolution for semantic segmentation. arXiv preprint arXiv:1702.08502 (2017)
  30. 30.
    Wood, E., Baltrusaitis, T., Zhang, X., Sugano, Y., Robinson, P., Bulling, A.: Rendering of eyes for eye-shape registration and gaze estimation. In: IEEE International Conference on Computer Vision, pp. 3756–3764. IEEE (2015)Google Scholar
  31. 31.
    Ye, Z., Li, Y., Liu, Y., Bridges, C., Rozga, A., Rehg, J.M.: Detecting bids for eye contact using a wearable camera. In: IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, pp. 1–8. IEEE (2015)Google Scholar
  32. 32.
    Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122 (2015)
  33. 33.
    Yu, F., Koltun, V., Funkhouser, T.: Dilated residual networks. In: IEEE International Conference on Computer Vision, pp. 636–644. IEEE (2017)Google Scholar
  34. 34.
    Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: Appearance-based gaze estimation in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4511–4520. IEEE (2015)Google Scholar
  35. 35.
    Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: It’s written all over your face: full-face appearance-based gaze estimation. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2299–2308. IEEE (2017)Google Scholar
  36. 36.
    Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: MPIIGaze: real-world dataset and deep appearance-based gaze estimation. IEEE Trans. Pattern Anal. Mach. Intell. 41, 162–175 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.The Hong Kong University of Science and TechnologyKowloonHong Kong SAR

Personalised recommendations