Abstract
Considering the main disadvantage of the existing gaze point estimation methods which restrict user’s head movement and have potential injury on eyes, we propose a gaze point estimation method based on facial normal and binocular vision. Firstly, we calibrate stereo cameras to determine the extrinsic and intrinsic parameters of the cameras; Secondly, face is quickly detected by Viola–Jones framework and the center position of the two irises can be located based on integro-differential operators; The two nostrils and mouth are detected based on the saturation difference and their 2D coordinates can be calculated; Thirdly, the 3D coordinates of these five points are obtained by stereo matching and 3D reconstruction; After that, a plane fitting algorithm based on least squares is adopted to get the approximate facial plane, then, the normal via the midpoint of the two pupils can be figured out; Finally, the point-of-gaze can be obtained by getting the intersection point of the facial normal and the computer screen. Experimental results confirm the accuracy and robustness of the proposed method.
Similar content being viewed by others
References
Arantxa Villanueva and Rafael Cabeza (2008) A novel gaze estimation system with one calibration point. IEEE Transactions on Systems, Man, and Cybernetics 38(6):1123–1138
Shao G, Che M, Zhang B, Cen K, Gao W (2010) A novel simple 2D model of eye gaze estimation. In: IEEE 2nd International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), vol 1. Nanjing, Jiangsu, pp 300–304
Zhu ZW, Ji Q (2007) Novel eye gaze tracking techniques under natural head movement. IEEE Trans Biomed Eng 54(12):2246–2260
Ning Zhang, Lei Chang, Xiping Xu (2012) Research on the technology of three-dimensional reconstruction based on machine vision. Laser Optoelectron Prog. doi:10.3788/LOP49.051001
JONESM (2009) Face recognition: where we are and where to go from here. IEEE TEIS 129:770–777
Wen G, Bo C, Shiguang Shan et al (2008) The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Trans Syst Man Cybern 38(1):149–161
Yamazoe H, Utsumi A, Yonezawa T, Abe S (2008) Remote gaze estimation with a single camera based on facial-feature tracking without special calibration actions. In: Proceedings of the 2008 symposium on eye tracking research & applications. ACM, pp 245–250
Yanhui S, Wenyong W, Xiaochun C (2005) The amelioration to the face recognition algorithm based on the Viola-Jones frame. J Northeast Norm Univ 37(3):24–27
Freund Yoav, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139
Luan B, Sörös P, Sejdić E (2013) A study of brain networks associated with swallowing using graph-theoretical approaches. PLoS ONE 8(8):e73577
Daugman J (2004) How iris recognition works. Circuit Syst Video Technol IEEE Trans 14(1):21–30
Sun W, Guo BL, Li DJ, Jia W (2013) Fast single-image dehazing method for visible-light systems. Opt Eng 52(9):093103
Wei S (2013) A new single image fog removal algorithm based on physical model. Int J Light Electron Opt 124(21):4770–4775
Sun W, Han L, Guo B, Jia W, Sun M (2014) A fast color image enhancement algorithm based on max intensity channel. J Mod Opt 61(6):466–477
Viola Paul, Michael J Jones (2004) “Robust real-time face detection”, Int J Comp Vis, 57.2: 137–154
Tan TN, He ZF, Sun ZN (2010) Efficient and robust segmentation of noisy iris images for non-cooperative iris recognition. Image Vis Comput. doi:10.1016/j.imavis.2009.05.008
Papageorgiou Constantine, Poggio Tomaso (2000) A trainable system for object detection. Int J Comput Vis 38(1):15–33
Castillo Carlos D, Jacobs DW (2009) Using stereo matching with general epipolar geometry for 2D face recognition across pose. IEEE Trans Pattern Anal Mach Intell 31(12):1198–2304
Jiangbo Lu, Cai Hua, Lou Jian-Guang, Li Jiang (2007) An epipolar geometry-based fast disparity estimation algorithm for multiview image and video coding. IEEE Trans Circuit Syst Video Technol 17(6):737–750
Zhao Y, Liu HX, Wang ZY et al (2009) An improved nearest neighbor searching method for classification problems. J Nanjing Univ (Nat Sci) 45:455–462
Heiko Hirschmuller and Daniel Scharstein (2009) Evaluation of stereo matching costs on images with radiometric differences. IEEE Trans Pattern Anal Mach Intell 31(9):1582–1599
Andreas Geiger, Julius Ziegler and Christoph Stiller (2011) StereoScan: dense 3D reconstruction in real-time. In: IEEE Intelligent vehicles symposium (IV). Baden-Baden, pp 963–968
Acknowledgments
This work was supported by Fundamental Research Funds for the Central Universities (Grant JB141307); National Nature Science Foundation of China (NSFC) (Grants 61201290), and NSFC Grants 61105066, 61305041, 61305040; the China Scholarship Council (CSC) and the National Institutes of Health (Grant R01CA165255) of the United States.
Author information
Authors and Affiliations
Corresponding author
Additional information
Wei Sun and Nan Sun are co-first authors.
Rights and permissions
About this article
Cite this article
Sun, W., Sun, N., Guo, B. et al. An auxiliary gaze point estimation method based on facial normal. Pattern Anal Applic 19, 611–620 (2016). https://doi.org/10.1007/s10044-014-0407-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10044-014-0407-5