Efficient Ultrasound Image Analysis Models with Sonographer Gaze Assisted Distillation

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11767)


Recent automated medical image analysis methods have attained state-of-the-art performance but have relied on memory and compute-intensive deep learning models. Reducing model size without significant loss in performance metrics is crucial for time and memory-efficient automated image-based decision-making. Traditional deep learning based image analysis only uses expert knowledge in the form of manual annotations. Recently, there has been interest in introducing other forms of expert knowledge into deep learning architecture design. This is the approach considered in the paper where we propose to combine ultrasound video with point-of-gaze tracked for expert sonographers as they scan to train memory-efficient ultrasound image analysis models. Specifically we develop teacher-student knowledge transfer models for the exemplar task of frame classification for the fetal abdomen, head, and femur. The best performing memory-efficient models attain performance within 5% of conventional models that are \(1000{\times }\) larger in size.


Model compression Gaze tracking Expert knowledge 



We acknowledge the ERC (ERC-ADG-2015 694581, project PULSE), the EPSRC (EP/GO36861/1, EP/MO13774/1, EP/R013853/1), the Rhodes Trust, and the NIHR Biomedical Research Centre funding scheme.

Supplementary material

490278_1_En_43_MOESM1_ESM.pdf (1.1 mb)
Supplementary material 1 (pdf 1118 KB)


  1. 1.
    Becker, D.M., et al.: The use of portable ultrasound devices in low-and middle-income countries: a systematic review of the literature. Trop. Med. Int. Health 21(3), 294–311 (2016)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Liu, B., et al.: Sparse convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 806–814 (2015)Google Scholar
  3. 3.
    He, Y., et al.: Channel pruning for accelerating very deep neural networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1389–1397 (2017)Google Scholar
  4. 4.
    Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. In: NIPS 2014 Deep Learning Workshop (2014)Google Scholar
  5. 5.
    Romero, A., et al.: FitNets: Hints for thin deep nets. arXiv:1412.6550 (2014)
  6. 6.
    Patra, A., Huang, W., Noble, J.A.: Learning spatio-temporal aggregation for fetal heart analysis in ultrasound video. In: Cardoso, M.J., et al. (eds.) DLMIA/ML-CDS -2017. LNCS, vol. 10553, pp. 276–284. Springer, Cham (2017). Scholar
  7. 7.
    Patra, A., et al.: Sequential anatomy localization in fetal echocardiography videos. arXiv preprint arXiv:1810.11868 (2018)
  8. 8.
    Vaseli, H., et al.: Designing lightweight deep learning models for echocardiography view classification. In: SPIE Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 10951 (2019)Google Scholar
  9. 9.
    Cai, Y., et al.: SonoEyeNet: standardized fetal ultrasound plane detection informed by eye tracking. In: 15th IEEE ISBI, pp. 1475–1478. IEEE (2018)Google Scholar
  10. 10.
    Buciluǎ, C., et al.: Model compression. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 535–541 (2006)Google Scholar
  11. 11.
    PULSE: Perception ultrasound by learning sonographic experience (2018).
  12. 12.
    Iandola, F.N., et al.: SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and \(<\)0.5 MB model size. arXiv preprint arXiv:1602.07360 (2016)
  13. 13.
    Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)
  14. 14.
    Sandler, M., et al.: MobileNetV2: inverted residuals and linear bottlenecks. In: CVPR (2018)Google Scholar
  15. 15.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 (2014)
  16. 16.
    Wong, A.: NetScore: towards universal metrics for large-scale performance analysis of deep neural networks for practical usage. arXiv:1806.05512 (2018)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.University of OxfordOxfordUK

Personalised recommendations