Combining Convolutional and Recurrent Neural Networks for Classification of Focal Liver Lesions in Multi-phase CT Images
Computer-aided diagnosis (CAD) systems are useful for assisting radiologists with clinical diagnoses by classifying focal liver lesions (FLLs) based on multi-phase computed tomography (CT) images. Although many studies have conducted in the field, there still remain two challenges. First, the temporal enhancement pattern is hard to represent effectively. Second, the local and global information of lesions both are necessary for this task. In this paper, we proposed a framework based on deep learning, called ResGL-BDLSTM, which combines a residual deep neural network (ResNet) with global and local pathways (ResGL Net) with a bi-directional long short-term memory (BD-LSTM) model for the task of focal liver lesions classification in multi-phase CT images. In addition, we proposed a novel loss function to train the proposed framework. The loss function is composed of an inter-loss and intra-loss, which can improve the robustness of the framework. The proposed framework outperforms state-of-the-art approaches by achieving a 90.93% mean accuracy.
KeywordsDeep learning ResGLNet BD-LSTM Liver lesions classification Computer-aid diagnosis (CAD) system
This work was supported in part by the National Key R&D Program of China under the Grant No. 2017YFB0309800, in part by the Key Science and Technology Innovation Support Program of Hangzhou under the Grant No. 20172011A038, and in part by the Grant-in Aid for Scientific Research from the Japanese Ministry for Education, Science, Culture and Sports (MEXT) under the Grant No. 18H03267 and No. 17H00754.
- 2.Chen, J., et al.: Combining fully convolutional and recurrent neural networks for 3D biomedical image segmentation. In: Advances in Neural Information Processing Systems (2016)Google Scholar
- 4.Yu, M., et al.: Extraction of lesion-partitioned features and retrieval of contrast-enhanced liver images. Comput. Math. Meth. Med. 2012, 12 (2012)Google Scholar
- 7.Xu, Y., et al.: Bag of temporal co-occurrence words for retrieval of focal liver lesions using 3D multiphase contrast-enhanced CT images. In: Proceedings of 23rd International Conference on Pattern Recognition (ICPR 2016), pp. 2283–2288 (2016)Google Scholar
- 8.Wang J., et al.: Sparse codebook model of local structures for retrieval of focal liver lesions using multiphase medical images. Int. J. Biomed. Imaging 2017, 13 p. (2017)Google Scholar
- 10.He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
- 11.Frid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J., Greenspan, H.: Modeling the intra-class variability for liver lesion detection using a multi-class patch-based CNN. In: Wu, G., Munsell, Brent C., Zhan, Y., Bai, W., Sanroma, G., Coupé, P. (eds.) Patch-MI 2017. LNCS, vol. 10530, pp. 129–137. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67434-6_15CrossRefGoogle Scholar
- 13.Liang, D., et al.: Residual convolutional neural networks with global and local pathways for classification of focal liver lesions. In: Geng, X., Kang, B.H. (eds.) PRICAI 2018: Trends in Artificial Intelligence. PRICAI 2018. LNCS, vol. 11012, pp. 617–628. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-97304-3_47Google Scholar
- 14.Dong, C., et al.: Simultaneous segmentation of multiple organs using random walks. J. Inf. Process. 24(2), 320–329 (2016)Google Scholar