Doubly Weak Supervision of Deep Learning Models for Head CT

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11766)


Recent deep learning models for intracranial hemorrhage (ICH) detection on computed tomography of the head have relied upon large datasets hand-labeled at either the full-scan level or at the individual slice-level. Though these models have demonstrated favorable empirical performance, the hand-labeled datasets upon which they rely are time-consuming and expensive to create. Further, given limited time, modelers must currently make an explicit choice between scan-level supervision, which leverages large numbers of patients, and slice-level supervision, which yields clinically insightful output in the axial and in-plane dimensions. In this work, we propose doubly weak supervision, where we (1) weakly label at the scan-level to scalably incorporate data from large populations and (2) model the problem using an attention-based multiple-instance learning approach that can provide useful signal at both axial and in-plane granularities, even with scan-level supervision. Models trained using this doubly weak supervision approach yield an average ROC-AUC score of 0.91, which is competitive with those of models trained using large, hand-labeled datasets, while requiring less than 10 h of clinician labeling time. Further, our models place large attention weights on the same slices used by the clinician to arrive at the ICH classification, and occlusion maps indicate heavy influence from clinically salient in-plane regions.


Weak supervision Multiple instance learning Head CT 

Supplementary material

490277_1_En_90_MOESM1_ESM.pdf (135 kb)
Supplementary material 1 (pdf 134 KB)


  1. 1.
    Chang, P., et al.: Hybrid 3D/2D convolutional neural network for hemorrhage evaluation on head CT. Am. J. Neuroradiol. 39(9), 1609–1616 (2018)CrossRefGoogle Scholar
  2. 2.
    Chilamkurthy, S., et al.: Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study. Lancet 392(10162), 2388–2396 (2018)CrossRefGoogle Scholar
  3. 3.
    Coles, J.: Imaging after brain injury. Br. J. Anaesth. 99(1), 49–60 (2007)CrossRefGoogle Scholar
  4. 4.
    Dunnmon, J.A., Yi, D., Langlotz, C.P., Ré, C., Rubin, D.L., Lungren, M.P.: Assessment of convolutional neural networks for automated classification of chest radiographs. Radiology 290, 181422 (2018)Google Scholar
  5. 5.
    Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639), 115 (2017)CrossRefGoogle Scholar
  6. 6.
    Fries, J.A., et al.: Weakly supervised classification of aortic valve malformations using unlabeled cardiac MRI sequences. Nat. Commun. 10(1), 3111 (2019)CrossRefGoogle Scholar
  7. 7.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  8. 8.
    Ilse, M., Tomczak, J., Welling, M.: Attention-based deep multiple instance learning. In: International Conference on Machine Learning, pp. 2132–2141 (2018)Google Scholar
  9. 9.
    Jnawali, K., Arbabshirani, M.R., Rao, N., Patel, A.A.: Deep 3D convolution neural network for CT brain hemorrhage classification. In: Medical Imaging 2018: Computer-Aided Diagnosis, vol. 10575, p. 105751C. International Society for Optics and Photonics (2018)Google Scholar
  10. 10.
    Lee, H., et al.: An explainable deep-learning algorithm for the detection of acute intracranial haemorrhage from small datasets. Nat. Biomed. Eng. 3(2018).
  11. 11.
    Ratner, A., Bach, S.H., Ehrenberg, H., Fries, J., Wu, S., Re, C.: Snorkel: rapid training data creation with weak supervision. Proc. VLDB Endow. 11(3), 269–282 (2017)CrossRefGoogle Scholar
  12. 12.
    Sun, L., Lu, Y., Yang, K., Li, S.: ECG analysis using multiple instance learning for myocardial infarction detection. IEEE Trans. Biomed. Eng. 59(12), 3348–3356 (2012)CrossRefGoogle Scholar
  13. 13.
    Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3462–3471 (2017)Google Scholar
  14. 14.
    Xu, Y., Zhang, J., Chang, E.I.-C., Lai, M., Tu, Z.: Context-constrained multiple instance learning for histopathology image segmentation. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012. LNCS, vol. 7512, pp. 623–630. Springer, Heidelberg (2012). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Electrical EngineeringStanford UniversityStanfordUSA
  2. 2.Department of Computer ScienceStanford UniversityStanfordUSA
  3. 3.Department of RadiologyStanford UniversityStanfordUSA
  4. 4.Department of Biomedical Data ScienceStanford UniversityStanfordUSA

Personalised recommendations