Skip to main content

Advertisement

SpringerLink
Log in

Improvement of the diagnostic accuracy for intracranial haemorrhage using deep learning–based computer-assisted detection

  • Diagnostic Neuroradiology
  • Published:
Neuroradiology Aims and scope Submit manuscript

Abstract

Purpose

To elucidate the effect of deep learning–based computer-assisted detection (CAD) on the performance of different-level physicians in detecting intracranial haemorrhage using CT.

Methods

A total of 40 head CT datasets (normal, 16; haemorrhagic, 24) were evaluated by 15 physicians (5 board-certificated radiologists, 5 radiology residents, and 5 medical interns). The physicians attended 2 reading sessions without and with CAD. All physicians annotated the haemorrhagic regions with a degree of confidence, and the reading time was recorded in each case. Our CAD system was developed using 433 patients’ head CT images (normal, 203; haemorrhagic, 230), and haemorrhage rates were displayed as corresponding probability heat maps using U-Net and a machine learning–based false-positive removal method. Sensitivity, specificity, accuracy, and figure of merit (FOM) were calculated based on the annotations and confidence levels.

Results

In patient-based evaluation, the mean accuracy of all physicians significantly increased from 83.7 to 89.7% (p < 0.001) after using CAD. Additionally, accuracies of board-certificated radiologists, radiology residents, and interns were 92.5, 82.5, and 76.0% without CAD and 97.5, 90.5, and 81.0% with CAD, respectively. The mean FOM of all physicians increased from 0.78 to 0.82 (p = 0.004) after using CAD. The reading time was significantly lower when CAD (43 s) was used than when it was not (68 s, p < 0.001) for all physicians.

Conclusion

The CAD system developed using deep learning significantly improved the diagnostic performance and reduced the reading time among all physicians in detecting intracranial haemorrhage.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Abbreviations

CT:

Computed tomography

AUC:

Area under the curve

CAD:

Computer-assisted detection

DL:

Deep learning;

FOM:

Figure of merit

ICH:

Intracranial haemorrhage

MRMC ROC:

Multi-reader, multi-case study of receiver operating characteristics

PACS:

Picture archiving and communication systems

References

  1. Hemphill JC 3rd, Greenberg SM, Anderson CS, Becker K, Bendok BR, Cushman M, Fung GL, Goldstein JN, Macdonald RL, Mitchell PH, Scott PA, Selim MH, Woo D (2015) Guidelines for the management of spontaneous intracerebral hemorrhage: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 46(7):2032–2060

    Article  Google Scholar 

  2. Saad AF, Chaudhari R, Fischbein NJ, Wintermark M (2018) Intracranial hemorrhage imaging. Semin Ultrasound CT MR 39(5):441–456

    Article  Google Scholar 

  3. Takagi Y, Hadeishi H, Mineharu Y, Yoshida K, Ogasawara K, Ogawa A, Miyamoto S (2018) Initially missed or delayed diagnosis of subarachnoid hemorrhage: a nationwide survey of contributing factors and outcomes in Japan. J Stroke Cerebrovasc Dis 27(4):871–877

    Article  Google Scholar 

  4. Strub WM, Leach JL, Tomsick T, Vagal A (2007) Overnight preliminary head CT interpretations provided by residents: locations of misidentified intracranial hemorrhage. AJNR Am J Neuroradiol 28(9):1679–1682

    Article  CAS  Google Scholar 

  5. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J, Kim R, Raman R, Nelson PC, Mega JL, Webster DR (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316(22):2402–2410

    Article  Google Scholar 

  6. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639):115–118

    Article  CAS  Google Scholar 

  7. Nakata N (2019) Recent technical development of artificial intelligence for diagnostic medical imaging. Jpn J Radiol 37(2):103–108

    Article  Google Scholar 

  8. Chang PD, Kuoy E, Grinband J, Weinberg BD, Thompson M, Homo R, Chen J, Abcede H, Shafie M, Sugrue L, Filippi CG, Su MY, Yu W, Hess C, Chow D (2018) Hybrid 3D/2D convolutional neural network for hemorrhage evaluation on head CT. AJNR Am J Neuroradiol 39(9):1609–1616

    Article  CAS  Google Scholar 

  9. Chilamkurthy S, Ghosh R, Tanamala S, Biviji M, Campeau NG, Venugopal VK, Mahajan V, Rao P, Warier P (2018) Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study. Lancet 392(10162):2388–2396

    Article  Google Scholar 

  10. Chan T, Huang HK (2008) Effect of a computer-aided diagnosis system on clinicians' performance in detection of small acute intracranial hemorrhage on computed tomography. Acad Radiol 15(3):290–299

    Article  Google Scholar 

  11. Ueda D, Yamamoto A, Nishimori M, Shimono T, Doishita S, Shimazaki A, Katayama Y, Fukumoto S, Choppin A, Shimahara Y, Miki Y (2019) Deep learning for MR angiography: automated detection of cerebral aneurysms. Radiology 290(1):187–194

    Article  Google Scholar 

  12. Park A, Chute C, Rajpurkar P, Lou J, Ball RL, Shpanskaya K, Jabarkheel R, Kim LH, McKenna E, Tseng J, Ni J, Wishah F, Wittber F, Hong DS, Wilson TJ, Halabi S, Basu S, Patel BN, Lungren MP, Ng AY, Yeom KW (2019) Deep learning-assisted diagnosis of cerebral aneurysms using the HeadXNet model. JAMA Netw Open 2(6):e195600

    Article  Google Scholar 

  13. Sohns C, Angic BC, Sossalla S, Konietschke F, Obenauer S (2010) CAD in full-field digital mammography-influence of reader experience and application of CAD on interpretation of time. Clin Imaging 34(6):418–424

    Article  Google Scholar 

  14. Conant EF, Toledano AY, Periaswamy S, Fotin SV, Go J, Boatsman JE, Hoffmeister JW (2019) Improving accuracy and efficiency with concurrent use of artificial intelligence for digital breast tomosynthesis. Radiol Artif Intell 1(4):e180096

    Article  Google Scholar 

  15. Yuh EL, Gean AD, Manley GT, Callen AL, Wintermark M (2008) Computer-aided assessment of head computed tomography (CT) studies in patients with suspected traumatic brain injury. J Neurotrauma 25(10):1163–1172

    Article  Google Scholar 

  16. Chan T (2007) Computer aided detection of small acute intracranial hemorrhage on computer tomography of brain. Comput Med Imaging Graph 31(4–5):285–298

    Article  Google Scholar 

  17. Prevedello LM, Erdal BS, Ryu JL, Little KJ, Demirer M, Qian S, White RD (2017) Automated critical test findings identification and online notification system using artificial intelligence in imaging. Radiology 285(3):923–931

    Article  Google Scholar 

  18. Cho J, Park KS, Karki M, Lee E, Ko S, Kim JK, Lee D, Choe J, Son J, Kim M, Lee S, Lee J, Yoon C, Park S (2019) Improving sensitivity on identification and delineation of intracranial hemorrhage lesion using cascaded deep learning models. J Digit Imaging 32(3):450–461

    Article  Google Scholar 

  19. Dawud AM, Yurtkan K, Oztoprak H (2019) Application of deep learning in neuroradiology: brain haemorrhage classification using transfer learning. Comput Intell Neurosci 2019:4629859

    Article  Google Scholar 

  20. Ginat DT (2020) Analysis of head CT scans flagged by deep learning software for acute intracranial hemorrhage. Neuroradiology 62(3):335–340

    Article  Google Scholar 

Download references

Funding

The authors received no external funding for this study.

Author information

Authors and Affiliations

  1. Department of Future Diagnostic Radiology, Osaka University Graduate School of Medicine, Osaka, Japan

    Yoshiyuki Watanabe

  2. Department of Radiology, Shiga University of Medical Science, Tsukiwacho Seta Otsu, Shiga, 520-2102, Japan

    Yoshiyuki Watanabe

  3. Dai Nippon Printing Co., Ltd., Tokyo, Japan

    Takahiro Tanaka & Atsushi Nishida

  4. Department of Diagnostic and Interventional Radiology, Osaka University Graduate School of Medicine, Osaka, Japan

    Hiroto Takahashi, Masahiro Fujiwara, Takuya Fujiwara, Atsuko Arisawa, Hiroki Yano & Noriyuki Tomiyama

  5. Department of Neurosurgery, Osaka University Graduate School of Medicine, Osaka, Japan

    Hajime Nakamura

  6. Department of Neurology, Osaka University Graduate School of Medicine, Osaka, Japan

    Kenichi Todo

  7. Department of Traumatology and Acute Critical Medicine, Osaka University Graduate School of Medicine, Osaka, Japan

    Kazuhisa Yoshiya

Authors
  1. Yoshiyuki Watanabe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Takahiro Tanaka
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Atsushi Nishida
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hiroto Takahashi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Masahiro Fujiwara
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Takuya Fujiwara
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Atsuko Arisawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Hiroki Yano
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Noriyuki Tomiyama
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Hajime Nakamura
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Kenichi Todo
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Kazuhisa Yoshiya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yoshiyuki Watanabe.

Ethics declarations

Conflicts of interest

Yoshiyuki Watanabe and Osaka University received the research fund from Dai-Nippon Printing Corp. Ltd. Takahiro Tanaka and Atsushi Nishida are employees of Dai-Nippon Printing Corp. Ltd.

Ethical approval and informed consent

The ethics board of our institution comprehensively reviewed and approved the protocol of this study. Because the images had been acquired during daily routine examination, the need for informed consent was waived by the ethics board.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Watanabe, Y., Tanaka, T., Nishida, A. et al. Improvement of the diagnostic accuracy for intracranial haemorrhage using deep learning–based computer-assisted detection. Neuroradiology 63, 713–720 (2021). https://doi.org/10.1007/s00234-020-02566-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00234-020-02566-x

Keywords

Search

Navigation