Skip to main content

Gaze2Segment: A Pilot Study for Integrating Eye-Tracking Technology into Medical Image Segmentation

Part of the Lecture Notes in Computer Science book series (LNIP,volume 10081)


In this study, we developed a novel system, called Gaze2Segment, integrating biological and computer vision techniques to support radiologists’ reading experience with an automatic image segmentation task. During diagnostic assessment of lung CT scans, the radiologists’ gaze information were used to create a visual attention map. Next, this map was combined with a computer-derived saliency map, extracted from the gray-scale CT images. The visual attention map was used as an input for indicating roughly the location of a region of interest. With computer-derived saliency information, on the other hand, we aimed at finding foreground and background cues for the object of interest found in the previous step. These cues are used to initiate a seed-based delineation process. The proposed Gaze2Segment achieved a dice similarity coefficient of 86% and Hausdorff distance of 1.45 mm as a segmentation accuracy. To the best of our knowledge, Gaze2Segment is the first true integration of eye-tracking technology into a medical image segmentation task without the need for any further user-interaction.


  • Eye tracking
  • Local saliency
  • Human computer interface
  • Medical image segmentation
  • Visual attention

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. Atkins, M.S., Moise, A., Rohling, R.: An application of eyegaze tracking for designing radiologists’ workstations: insights for comparative visual search tasks. ACM Trans. Appl. Percept. (TAP) 3(2), 136–151 (2006)

    CrossRef  Google Scholar 

  2. Bagci, U., Chen, X., Udupa, J.K.: Hierarchical scale-based multiobject recognition of 3-d anatomical structures. IEEE Trans. Med. Imaging 31(3), 777–789 (2012)

    CrossRef  Google Scholar 

  3. Drew, T., Vo, M.L.H., Olwal, A., Jacobson, F., Seltzer, S.E., Wolfe, J.M.: Scanners and drillers: characterizing expert visual search through volumetric images. J. Vis. 13(10), 3 (2013)

    CrossRef  Google Scholar 

  4. Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. IEEE Trans. PAMI 34(10), 1915–1926 (2012)

    CrossRef  Google Scholar 

  5. Grady, L.: Random walks for image segmentation. IEEE Trans. PAMI 28(11), 1768–1783 (2006)

    CrossRef  Google Scholar 

  6. von Helmholtz, H., Southall, J.P.C.: Treatise on physiological optics, vol. 3. Courier Corporation (2005)

    Google Scholar 

  7. Just, M.A., Carpenter, P.A.: A theory of reading: from eye fixations to comprehension. Psychol. Rev. 87(4), 329 (1980)

    CrossRef  Google Scholar 

  8. Mallett, S., Phillips, P., Fanshawe, T.R., Helbren, E., Boone, D., Gale, A., Taylor, S.A., Manning, D., Altman, D.G., Halligan, S.: Tracking eye gaze during interpretation of endoluminal three-dimensional CT colonography: visual perception of experienced and inexperienced readers. Radiology 273(3), 783–792 (2014)

    CrossRef  Google Scholar 

  9. McAuliffe, M.J., Lalonde, F.M., McGarry, D., Gandler, W., Csaky, K., Trus, B.L.: Medical image processing, analysis and visualization in clinical research. In: 14th IEEE Symposium on Computer-Based Medical Systems (CBMS 2001), Proceedings, pp. 381–386. IEEE (2001)

    Google Scholar 

  10. Sadeghi, M., Tien, G., Hamarneh, G., Atkins, M.S.: Hands-free interactive image segmentation using eyegaze. In: SPIE Medical Imaging, p. 72601H. International Society for Optics and Photonics (2009)

    Google Scholar 

  11. Ware, C., Mikaelian, H.H.: An evaluation of an eye tracker as a device for computer input2. ACM SIGCHI Bull. 17, 183–188 (1987)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Ulas Bagci .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Khosravan, N. et al. (2017). Gaze2Segment: A Pilot Study for Integrating Eye-Tracking Technology into Medical Image Segmentation. In: Müller, H., et al. Medical Computer Vision and Bayesian and Graphical Models for Biomedical Imaging. BAMBI MCV 2016 2016. Lecture Notes in Computer Science(), vol 10081. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-61187-7

  • Online ISBN: 978-3-319-61188-4

  • eBook Packages: Computer ScienceComputer Science (R0)