Abstract
In this paper, we present SonoSAM - a promptable foundational model for segmenting objects of interest on ultrasound images. Fine-tuned exclusively on a rich, diverse set of objects from \(\approx 200\)k ultrasound image-mask pairs, SonoSAM demonstrates state-of-the-art performance on 8 unseen ultrasound data-sets, outperforming competing methods by a significant margin on all metrics of interest. SonoSAM achieves average dice similarity score of >90% on almost all test data-sets within 2–6 clicks on an average. Further, to increase practical utility of SonoSAM, we propose a two-step process of fine-tuning followed by knowledge distillation to a smaller footprint model without comprising the performance. We present detailed qualitative and quantitative comparisons of SonoSAM with state-of-the-art methods showcasing efficacy of SonoSAM as one of the first reliable, generic foundational model for ultrasound.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Falk, T., et al.: U-net: deep learning for cell counting, detection, and morphometry. Nat. Methods 16(1), 67–70 (2019)
Chen, X., Zhao, Z., Zhang, Y., Duan, M., Qi, D., Zhao, H.: FocalClick: towards practical interactive image segmentation. In: CVPR (2022)
Kirillov, A., et al.: Segment anything. arXiv preprint arXiv:2304.02643 (2023)
Liu, Q., Xu, Z., Bertasius, G., Niethammer, M.: SimpleClick: interactive image segmentation with simple vision transformers. arXiv preprint arXiv:2210.11006 (2022)
Author, A.-B.: Contribution title. In: 9th International Proceedings on Proceedings, pp. 1–2. Publisher, Location (2010)
LNCS Homepage. http://www.springer.com/lncs. Accessed 4 Oct 2017
Bai, J., Wu, X.: Error-tolerant scribbles based interactive image segmentation. In: CVPR (2014)
Kim, T.H., Lee, K.M., Lee, S.U.: Generative image segmentation using random walks with restart. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5304, pp. 264–275. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88690-7_20
Zhang, S., Liew, J.H., Wei, Y., Wei, S., Zhao, Y.: Interactive object segmentation with inside-outside guidance. In: CVPR (2020)
Wu, J., Zhao, Y., Zhu, J.-Y., Luo, S., Tu, Z.: MILCut: a sweeping line multiple instance learning paradigm for interactive image segmentation. In: CVPR (2014)
Maninis, K.-K., Caelles, S., Pont-Tuset, J., Gool, L.V.: Deep extreme cut: from extreme points to object segmentation. In: CVPR (2018)
Rother, C., Kolmogorov, V., Blake, A.: “GrabCut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23 (2004)
Boykov, Y., Jolly, M.-P.: Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images. In: ICCV (2001)
Grady, L.: Random walks for image segmentation. IEEE Trans. PAMI 28 (2006)
Gulshan, V., Rother, C., Criminisi, A., Blake, A., Zisserman, A.: Geodesic star convexity for interactive image segmentation. In: CVPR (2010)
Xu, N., Price, B., Cohen, S., Yang, J., Huang, T.: Deep interactive object selection. In: CVPR (2016)
Radford, A., et al.: Learning transferable visual models from natural language supervision. In: ICML (2021)
Sofiiuk, K., Petrov, I.A., Konushin, A.: Reviving iterative training with mask guidance for interactive segmentation. arXiv:2102.06583 (2021)
Mahadevan, S., Voigtlaender, P., Leibe, B.: Iteratively trained interactive segmentation. In: BMVC (2018)
Forte, M., Price, B., Cohen, S., Xu, N., Pitié, F.: Getting to 99% accuracy in interactive segmentation. arXiv:2003.07932 (2020)
Jang, W.-D., Kim, C.-S.: Interactive image segmentation via backpropagating refinement scheme. In: CVPR (2019)
Sofiiuk, K., Petrov, I., Barinova, O., Konushin, A.: f-BRS: rethinking backpropagating refinement for interactive segmentation. In: CVPR (2020)
Liu, Q., Xu, Z., Bertasius, G., Niethammer, M.: SimpleClick: interactive image segmentation with simple vision transformers. arXiv:2210.11006 (2022)
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 213–229. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_13
Hu, E.J., et al.: Lora: low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)
Mazurowski, M.A., et al.: Segment anything model for medical image analysis: an experimental study. arXiv preprint arXiv:2304.10517 (2023)
Ma, J., Wang, B.: Segment anything in medical images. arXiv preprint arXiv:2304.12306 (2023)
Zhang, S., Metaxas, D.: On the challenges and perspectives of foundation models for medical image analysis. arXiv preprint arXiv:2306.05705 (2023)
Zhang, K., Liu, D.: Customized segment anything model for medical image segmentation. arXiv preprint arXiv:2304.13785 (2023)
Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)
Al-Dhabyani, W., Gomaa, M., Khaled, H., Fahmy, A.: Dataset of breast ultrasound images. Data Brief 28, 104863 (2020). https://doi.org/10.1016/j.dib.2019.104863
Leclerc, S., Smistad, E., Pedrosa, J., Ostvik, A., et al.: Deep learning for segmentation using an open large-scale dataset in 2D echocardiography. IEEE Trans. Med. Imaging 38(9), 2198–2210 (2019). https://doi.org/10.1109/TMI.2019.2900516
Montoya, H.: kaggle446, shirzad, Will Cukierski, yffud, ultrasound-nerve-segmentation, Kaggle (2016). https://kaggle.com/competitions/ultrasound-nerve-segmentation
Marzola, F., van Alfen, N., Doorduin, J., Meiburger, K.M.: Deep learning segmentation of transverse musculoskeletal ultrasound images for neuromuscular disease assessment. Comput. Biol. Med. 104623 (2021). ISSN 0010-4825, https://doi.org/10.1016/j.compbiomed.2021.104623
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ravishankar, H., Patil, R., Melapudi, V., Annangi, P. (2023). SonoSAM - Segment Anything on Ultrasound Images. In: Kainz, B., Noble, A., Schnabel, J., Khanal, B., Müller, J.P., Day, T. (eds) Simplifying Medical Ultrasound. ASMUS 2023. Lecture Notes in Computer Science, vol 14337. Springer, Cham. https://doi.org/10.1007/978-3-031-44521-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-44521-7_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44520-0
Online ISBN: 978-3-031-44521-7
eBook Packages: Computer ScienceComputer Science (R0)
