An Efficient Eye Tracking Using POMDP for Robust Human Computer Interaction

  • Ji Hye Rhee
  • Won Jun Sung
  • Mi Young Nam
  • Hyeran Byun
  • Phill Kyu Rhee
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9163)


We propose an adaptive eye tracking system for robust human-computer interaction under dynamically changing environments based on the partially observable Markov Decision Process (POMDP). In our system, real-time eye tracking optimization is tackled using a flexible world-context model based POMDP approach that requires less data and time in adaptation than those of hard world-context model approaches. The challenge is to divide the huge belief space into world-context models, and to search for optimal control parameters in the current world-context model with real-time constraints. The offline learning determines multiple world-context models based on image-quality analysis over the joint space of transition, observation, reward distributions, and an approximate world-context model is balanced with the online learning over a localized horizon. The online learning is formulated as a dynamic parameter control with incomplete information under real-time constraints, and is solved by the real-time Q-learning approach. Extensive experiments conducted using realistic videos have provided us with very encouraging results.


Eye tracking POMDP Real-time Q-learning World-context model Image-quality analysis 


  1. 1.
    Vazquez, L.J.G., Minor, M.A.: Low Cost Human Computer Interface Voluntary Eye Movement as Communication System for Disabled People with Limited Movements. In: PAHCE, pp. 165–170. IEEE, Rio de Janeiro (2011)Google Scholar
  2. 2.
    Corcoran, P.M., Nanu, F., Petrescu, S.: Real-time eye gaze tracking for gaming design and consumer electronics systems. IEEE Trans. Consum. Electron. 58(2), 347–355 (2012)CrossRefGoogle Scholar
  3. 3.
    Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  4. 4.
    Shani, G.: A survey of point-based POMDP solvers. Auton Agent Multi-Agent Syst 27(1), 1–51 (2012)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Broz, F., Nourbakhsh, I.: R., Simmons: Planning for Human-Robot Interaction in Socially Situated Tasks. International Journal of Social Robotics 5(2), 193–214 (2013)CrossRefGoogle Scholar
  6. 6.
    Gianni, M., Kruijff, G.J.M., Pirri, F.: A stimulus-response framework for robot control. ACM Trans. Interact. Intell. Syst. 4(4), 1–41 (2015). Article 21CrossRefGoogle Scholar
  7. 7.
    Bennett, C.C., Hauser, K.: Artificial intelligence framework for simulating clinical decision-making: a markov decision process approach. Artif. Intell. Med. 57(1), 9–19 (2013)CrossRefGoogle Scholar
  8. 8.
    Hoey, J., Poupart, P., et al.: Automated hand washing assistance for persons with dementia using video and a partially observable Markov decision process. Comput. Vis. Image Underst. 114(5), 503–519 (2010)CrossRefGoogle Scholar
  9. 9.
    Yoshino, K., Kawahara, T.: Conversational System for Information Navigation based on POMDP with User Focus Tracking. Computer Speech & Language 26(5), 349–370 (2015)Google Scholar
  10. 10.
    Pineau, J., Gordon, G., Thrun, S.: Anytime point-based approximations for large POMDPs. J. Artif. Intell. Res. 27, 335–380 (2006)zbMATHGoogle Scholar
  11. 11.
    S., Paquet: Distributed decision-making and task coordination in dynamic, uncertain and real-time multiagent environments. Ph.D. thesis, Laval University (2006)Google Scholar
  12. 12.
    Ross, S., Pineau, J., Paquet, S., Chaib-Draa, B.: Online planning algorithms for POMDPs. J. Artif. Intell. Res. 32, 663–704 (2008)zbMATHMathSciNetGoogle Scholar
  13. 13.
    Doshi, F., Pineau, J., Roy, N.: Reinforcement learning with limited reinforcement: using Bayes risk for active learning in POMDPs. In: ICML 2008 Proceedings of the 25th international conference on Machine learning, pp.256–263. New York (2008)Google Scholar
  14. 14.
    Doshi-Velez, F.: The infinite partially observable Markov decision process. Adv. Neural Inf. Process. Syst. 22, 477–485 (2009)Google Scholar
  15. 15.
    Krause, A., Ihmig, M., et. al.: Trading off prediction accuracy and power consumption for context-aware wearable computing. In: Proceedings of the 9th IEEE International Symposium on Wearable Computers, pp. 20–26. IEEE (2005)Google Scholar
  16. 16.
    Au, L., Batalin, M.A., et. al.: Episodic sampling: Towards energy-efficient patient monitoring with wearable sensors. In: Proceedings of the IEEE Annu. International Conference Engineering in Medicine and Biology Society, pp. 6901–6905. IEEE, Minneapolis (2009)Google Scholar
  17. 17.
    Bhanu, B., Peng, J.: Adaptive integrated image segmentation and object recognition. IEEE Trans on Systems, Man, and Cybernetics-PART C: Appl. Rev. 30(4), 427–441 (2000)CrossRefGoogle Scholar
  18. 18.
    Sellahewa, H., Jassim, S.A.: Image-quality-based adaptive face recognition. IEEE Trans Instrum. Meas. 59(4), 805–813 (2010)CrossRefGoogle Scholar
  19. 19.
    Dearden, R., Friedman, N., Andre, D.: Model based Bayesian exploration. In: UAI 1999 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence, pp. 150–159. Morgan Kaufmann, San Francisco (1999)Google Scholar
  20. 20.
    Rhee, P.K., Nam, M.Y., Wang, L.: Pupil location and movement measurement for efficient emotional sensibility Analysis. In: 2010 IEEE International Symposium on ISSPIT, pp. 1–6 (2010)Google Scholar
  21. 21.
    Kadal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2012)CrossRefGoogle Scholar
  22. 22.
    Shen, Y., Shin, H.C., Sung, W.J.: Evolutionary adaptive eye tracking for low-cost human computer interaction applications. J. Electron. Imaging 22(1), 013031 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Ji Hye Rhee
    • 1
  • Won Jun Sung
    • 2
  • Mi Young Nam
    • 3
  • Hyeran Byun
    • 1
  • Phill Kyu Rhee
    • 2
  1. 1.Department of Computer ScienceYonsei UniversitySeoulSouth Korea
  2. 2.Department of Computer Science & EngineeringInha UniversityIncheonSouth Korea
  3. 3.YM-NaeultechIncheonSouth Korea

Personalised recommendations