Abstract
Multiple Instance Learning (MIL) has been widely applied to medical imaging diagnosis, where bag labels are known and instance labels inside bags are unknown. Traditional MIL assumes that instances in each bag are independent samples from a given distribution. However, instances are often spatially or sequentially ordered, and one would expect similar diagnostic importance for neighboring instances. To address this, in this study, we propose a smooth attention deep MIL (SA-DMIL) model. Smoothness is achieved by the introduction of first and second order constraints on the latent function encoding the attention paid to each instance in a bag. The method is applied to the detection of intracranial hemorrhage (ICH) on head CT scans. The results show that this novel SA-DMIL: (a) achieves better performance than the non-smooth attention MIL at both scan (bag) and slice (instance) levels; (b) learns spatial dependencies between slices; and (c) outperforms current state-of-the-art MIL methods on the same ICH test set.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
RSNA intracranial hemorrhage detection. https://kaggle.com/c/rsna-intracranial-hemorrhage-detection
Arbabshirani, M.R., et al.: Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration. NPJ Digit.Med. 1(1), 9 (2018)
Arendts, G., Manovel, A., Chai, A.: Cranial CT interpretation by senior emergency department staff. Australas. Radiol. 47(4), 368–374 (2003)
Belkin, M., Niyogi, P., Sindhwani, V.: Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res. 7(11), 2399–2434 (2006)
Caceres, J.A., Goldstein, J.N.: Intracranial hemorrhage. Emerg. Med. Clin. North Am. 30(3), 771–794 (2012)
Carbonneau, M.A., Cheplygina, V., Granger, E., Gagnon, G.: Multiple instance learning: a survey of problem characteristics and applications. Pattern Recogn. 77, 329–353 (2018)
Chang, P.D., et al.: Hybrid 3d/2d convolutional neural network for hemorrhage evaluation on head CT. Am. J. Neuroradiol. 39(9), 1609–1616 (2018)
Cheplygina, V., de Bruijne, M., Pluim, J.P.: Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Med. Image Anal. 54, 280–296 (2019)
Cordonnier, C., Demchuk, A., Ziai, W., Anderson, C.S.: Intracerebral hemorrhage: current approaches to acute management. Lancet 392(10154), 1257–1268 (2018)
Dietterich, T.G., Lathrop, R.H., Lozano-Pérez, T.: Solving the multiple instance problem with axis-parallel rectangles. Artif. Intell. 89(1–2), 31–71 (1997)
Elliott, J., Smith, M.: The acute management of intracerebral hemorrhage: a clinical review. Anesth. Analg. 110(5), 1419–1427 (2010)
Erly, W.K., Berger, W.G., Krupinski, E., Seeger, J.F., Guisto, J.A.: Radiology resident evaluation of head CT scan orders in the emergency department. Am. J. Neuroradiol. 23(1), 103–107 (2002)
Gadermayr, M., Tschuchnig, M.: Multiple instance learning for digital pathology: a review on the state-of-the-art, limitations & future potential. arXiv preprint arXiv:2206.04425 (2022)
Grewal, M., Srivastava, M.M., Kumar, P., Varadarajan, S.: RadNet: radiologist level accuracy using deep learning for hemorrhage detection in CT scans. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 281–284. IEEE (2018)
Ilse, M., Tomczak, J., Welling, M.: Attention-based deep multiple instance learning. In: International Conference on Machine Learning, pp. 2127–2136. PMLR (2018)
Ker, J., Singh, S.P., Bai, Y., Rao, J., Lim, T., Wang, L.: Image thresholding improves 3-dimensional convolutional neural network diagnosis of different acute brain hemorrhages on computed tomography scans. Sensors 19(9), 2167 (2019)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)
Li, H., et al.: Multi-modal multi-instance learning using weakly correlated histopathological images and tabular clinical information. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12908, pp. 529–539. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87237-3_51
López-Pérez, M., Schmidt, A., Wu, Y., Molina, R., Katsaggelos, A.K.: Deep gaussian processes for multiple instance learning: application to CT intracranial hemorrhage detection. Comput. Methods Program. Biomed. 219, 106783 (2022)
McDonald, R.J., et al.: The effects of changes in utilization and technological advancements of cross-sectional imaging on radiologist workload. Acad. Radiol. 22(9), 1191–1198 (2015)
Quellec, G., Cazuguel, G., Cochener, B., Lamard, M.: Multiple-instance learning for medical image and video analysis. IEEE Rev. Biomed. Eng. 10, 213–234 (2017)
Qureshi, A.I., Tuhrim, S., Broderick, J.P., Batjer, H.H., Hondo, H., Hanley, D.F.: Spontaneous intracerebral hemorrhage. New England J. Med. 344(19), 1450–1460 (2001)
Ripley, B.: Spatial Statistics. Wiley, New York (1981)
Shao, Z., Bian, H., Chen, Y., Wang, Y., Zhang, J., Ji, X., et al.: Transmil: transformer based correlated multiple instance learning for whole slide image classification. Adv. Neural Inf. Process. Syst. 34, 2136–2147 (2021)
Strub, W., Leach, J., Tomsick, T., Vagal, A.: Overnight preliminary head CT interpretations provided by residents: locations of misidentified intracranial hemorrhage. Am. J. Neuroradiol. 28(9), 1679–1682 (2007)
Teneggi, J., Yi, P.H., Sulam, J.: Weakly supervised learning significantly reduces the number of labels required for intracranial hemorrhage detection on head ct. arXiv preprint arXiv:2211.15924 (2022)
Titano, J.J., et al.: Automated deep-neural-network surveillance of cranial images for acute neurologic events. Nat. Med. 24(9), 1337–1341 (2018)
Wang, Y., Li, J., Metze, F.: A comparison of five multiple instance learning pooling functions for sound event detection with weak labeling. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 31–35. IEEE (2019)
Wu, Y., Schmidt, A., Hernández-Sánchez, E., Molina, R., Katsaggelos, A.K.: Combining attention-based multiple instance learning and gaussian processes for CT hemorrhage detection. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12902, pp. 582–591. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87196-3_54
Ye, H., et al.: Precise diagnosis of intracranial hemorrhage and subtypes using a three-dimensional joint convolutional and recurrent neural network. Eur. Radiol. 29(11), 6191–6201 (2019). https://doi.org/10.1007/s00330-019-06163-2
Yeo, M., et al.: Review of deep learning algorithms for the automatic detection of intracranial hemorrhages on computed tomography head imaging. J. Neurointerventional Surg. 13(4), 369–378 (2021)
Acknowledgement
This work was supported by project PID2019-105142RB-C22 funded by Ministerio de Ciencia e Innovación and by project B-TIC-324-UGR20 funded by FEDER/Junta de Andalucía and Universidad de Granada. The work by Francisco M. Castro-Macías was supported by Ministerio de Universidades under FPU contract FPU21/01874.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wu, Y., Castro-Macías, F.M., Morales-Álvarez, P., Molina, R., Katsaggelos, A.K. (2023). Smooth Attention for Deep Multiple Instance Learning: Application to CT Intracranial Hemorrhage Detection. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14224. Springer, Cham. https://doi.org/10.1007/978-3-031-43904-9_32
Download citation
DOI: https://doi.org/10.1007/978-3-031-43904-9_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43903-2
Online ISBN: 978-3-031-43904-9
eBook Packages: Computer ScienceComputer Science (R0)
