Skip to main content
SpringerLink
Log in

Vision-Based Human Attention Modelling

  • Chapter
  • First Online:
Vision-Based Human Activity Recognition

Part of the book series: SpringerBriefs in Intelligent Systems ((BRIEFSINSY))

  • 360 Accesses

Abstract

This chapter is primarily concerned with human attention modelling, specifically the human driver. Previous related studies have typically focused on a single side without providing a comprehensive understanding. As a result, this chapter introduces a context-aware human driver attention estimation framework that combines scene visual saliency information and appearance-based driver state to provide a more accurate estimation. It has been validated in a VR-based experimental platform. Currently, the implicit and explicit of the human cognitive state and the contextual scenario still ill-defined, and we hope the related research can investigate it further.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Shadrin SS, Ivanova AA (2019) Analytical review of standard sae j3016 taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles with latest updates. Avtomobil Doroga Infrastruktura 3(21):10

    Google Scholar 

  2. Sikander G, Anwar S (2019) Driver fatigue detection systems: a review. IEEE Trans Intell Transp Syst 20(6):2339–2352

    Article  Google Scholar 

  3. Hu Z, Zhang Y, Xing Y, Zhao Y, Cao D, Lv C (2022) Toward human-centered automated driving: a novel spatiotemporal vision transformer-enabled head tracker. IEEE Vehi Technol Mag 2–9. https://doi.org/10.1109/MVT.2021.3140047

  4. Hu Z, Lv C, Hang P, Huang C, Xing Y (2022) Data-driven estimation of driver attention using calibration-free eye gaze and scene features. IEEE Trans Ind Electron 69(2):1800–1808. https://doi.org/10.1109/TIE.2021.3057033

    Article  Google Scholar 

  5. Kashevnik A, Lashkov I, Gurtov A (2020) Methodology and mobile application for driver behavior analysis and accident prevention. IEEE Trans Intell Transp Syst 21(6):2427–2436. https://doi.org/10.1109/TITS.2019.2918328

    Article  Google Scholar 

  6. Chiou C-Y, Wang W-C, Lu S-C, Huang C-R, Chung P-C, Lai Y-Y (2020) Driver monitoring using sparse representation with part-based temporal face descriptors. IEEE Trans Intell Transp Syst 21(1):346–361. https://doi.org/10.1109/TITS.2019.2892155

    Article  Google Scholar 

  7. Takahashi H, Ukishima D, Kawamoto K, Hirota K (2007) A study on predicting hazard factors for safe driving. IEEE Trans Ind Electron 54(2):781–789

    Article  Google Scholar 

  8. Deng T, Yang K, Li Y, Yan H (2016) Where does the driver look? top-down-based saliency detection in a traffic driving environment. IEEE Trans Intell Transp Syst 17(7):2051–2062

    Article  Google Scholar 

  9. Palazzi A, Abati D, Calderara S, Solera F, Cucchiara R (2019) Predicting the driver’s focus of attention: The dr(eye)ve project. IEEE Trans Pattern Anal Mach Intell 41(7):1720–1733

    Google Scholar 

  10. Vora S, Rangesh A, Trivedi MM (2018) Driver gaze zone estimation using convolutional neural networks: A general framework and ablative analysis. IEEE Trans Intell Veh 3(3):254–265

    Article  Google Scholar 

  11. Tawari A, Chen KH, Trivedi MM (2014) Where is the driver looking: Analysis of head, eye and iris for robust gaze zone estimation. In: 17th International IEEE conference on intelligent transportation systems (ITSC), pp 988–994

    Google Scholar 

  12. Lundgren M, Hammarstrand L, McKelvey T (2016) Driver-gaze zone estimation using bayesian filtering and gaussian processes. IEEE Trans Intell Transp Syst 17(10):2739–2750

    Article  Google Scholar 

  13. Martin S, Vora S, Yuen K, Trivedi MM (2018) Dynamics of driver’s gaze: explorations in behavior modeling and maneuver prediction. IEEE Transactions on Intelligent Vehicles 3(2):141–150

    Article  Google Scholar 

  14. Borji A (2021) Saliency prediction in the deep learning era: successes and limitations. IEEE Trans Pattern Anal Mach Intell 43(2):679–700. https://doi.org/10.1109/TPAMI.2019.2935715

    Article  Google Scholar 

  15. Borji A, Itti L (2012) State-of-the-art in visual attention modeling. IEEE Trans Pattern Anal Mach Intell 35(1):185–207

    Article  Google Scholar 

  16. Zhang L, Tong MH, Marks TK, Shan H, Cottrell GW (2008) Sun: a Bayesian framework for saliency using natural statistics. J Vis 8(7):32–32

    Article  Google Scholar 

  17. Bruce N, Tsotsos J (2005) Saliency based on information maximization. In: Advances in neural information processing systems, pp 155–162

    Google Scholar 

  18. Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X, Shum H-Y (2010) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353–367

    Google Scholar 

  19. Kruthiventi, S.S, Gudisa, V, Dholakiya, J.H, Babu, R.V.: Saliency unified: A deep architecture for simultaneous eye fixation prediction and salient object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5781–5790 (2016)

    Google Scholar 

  20. Jetley S, Murray N, Vig E (2016) End-to-end saliency mapping via probability distribution prediction. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5753–5761

    Google Scholar 

  21. Kümmerer M, Wallis TS, Bethge M (2016) Deepgaze ii: reading fixations from deep features trained on object recognition. arXiv:1610.01563

  22. Cornia M, Baraldi L, Serra G, Cucchiara R (2016) A deep multi-level network for saliency prediction. In: 2016 23rd International conference on pattern recognition (ICPR). IEEE, pp 3488–3493

    Google Scholar 

  23. Xia Y, Zhang D, Kim J, Nakayama K, Zipser K, Whitney D (2018) Predicting driver attention in critical situations. In: Asian conference on computer vision. Springer, pp 658–674

    Google Scholar 

  24. Deng T, Yan H, Qin L, Ngo T, Manjunath B (2019) How do drivers allocate their potential attention? driving fixation prediction via convolutional neural networks. IEEE Trans Intell Transp Syst 21(5):2146–2154

    Article  Google Scholar 

  25. Tawari A, Kang B (2017) A computational framework for driver’s visual attention using a fully convolutional architecture. In: 2017 IEEE intelligent vehicles symposium (IV). IEEE, pp 887–894

    Google Scholar 

  26. Palazzi A, Solera F, Calderara S, Alletto S, Cucchiara R (2017) Learning where to attend like a human driver. In: 2017 IEEE intelligent vehicles symposium (IV). IEEE, pp 920–925

    Google Scholar 

  27. Lateef F, Kas M, Ruichek Y (2021) Saliency heat-map as visual attention for autonomous driving using generative adversarial network (GAN). IEEE Trans Intell Transp Syst

    Google Scholar 

  28. Fang J, Yan D, Qiao J, Xue J, Yu H (2021) Dada: driver attention prediction in driving accident scenarios. IEEE Trans Intell Transp Syst

    Google Scholar 

  29. Baee S, Pakdamanian E, Kim I, Feng L, Ordonez V, Barnes L (2021) MEDIRL: predicting the visual attention of drivers via maximum entropy deep inverse reinforcement learning. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 13178–13188

    Google Scholar 

  30. Yang L, Dong K, Dmitruk AJ, Brighton J, Zhao Y (2020) A dual-cameras-based driver gaze mapping system with an application on non-driving activities monitoring. IEEE Trans Intell Transp Syst 21(10):4318–4327

    Article  Google Scholar 

  31. Pan J, Ferrer CC, McGuinness K, O’Connor NE, Torres J, Sayrol E, Giro-i-Nieto X (2017) SalGAN: visual saliency prediction with generative adversarial networks. arXiv:1701.01081

  32. Cornia M, Baraldi L, Serra G, Cucchiara R (2018) Predicting human eye fixations via an LSTM-based saliency attentive model. IEEE Trans Image Process 27(10):5142–5154

    Article  MathSciNet  Google Scholar 

  33. Bylinskii Z, Judd T, Oliva A, Torralba A, Durand F (2018) What do different evaluation metrics tell us about saliency models? IEEE Trans Pattern Anal Mach Intell 41(3):740–757

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mechanical Aerospace Engineering, Nanyang Technological University, Singapore, Singapore

    Zhongxu Hu & Chen Lv

Authors
  1. Zhongxu Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhongxu Hu .

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Hu, Z., Lv, C. (2022). Vision-Based Human Attention Modelling. In: Vision-Based Human Activity Recognition. SpringerBriefs in Intelligent Systems. Springer, Singapore. https://doi.org/10.1007/978-981-19-2290-9_5

Download citation

  • DOI: https://doi.org/10.1007/978-981-19-2290-9_5

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-19-2289-3

  • Online ISBN: 978-981-19-2290-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation