Abstract
The bag-of-visual-words (BoVW) model has emerged as an effective approach to represent features for focal liver lesions (FLLs). However, most of the previous methods have the limitation of insufficient consideration of the spatiotemporal co-occurrence information, which provokes the low descriptive power of classic visual words. In contrast to previous work, we propose a novel model for multiphase medical image feature generation named the Bi-gram bag-of-spatiotemporal words (Bi-gram BoSTW) to capture the temporal information, as well as, the spatial co-occurrence relationship of the lesion. First, temporal co-occurrence images from multiphase images are constructed. Second, BoVW is employed to extract temporal features from the temporal co-occurrence images and generates the visual words. Finally, we introduce the N-gram schema to add spatial relation to local descriptors. To the best of our knowledge, this is the first work that introduces visual N-grams scheme to contrast-enhanced CT images, which integrates temporal information with spatial co-occurrence relationship and improves the classification performance. The effectiveness of the proposed model is verified on 132 FLLs with confirmed pathology type. The experimental results indicate that (1) the N-gram enriches the semantics and provides more complete representation; (2) the proposed model achieves the best accuracy (83%) with highest training speed (1.5 min) among several well-known methods based on BoVW model.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Yu, M., Feng, Q., Yang, W., Gao, Y., Chen, W.: Extraction of lesion partitioned features and retrieval of contrast-enhanced liver images. Comput. Math. Methods Med. 2012, 12 (2012)
Roy, S., Chi, Y., Liu, J., Venkatesh, S., Brown, M.: Three-dimensional spatio-temporal features for fast content-based retrieval of focal liver lesions. IEEE Trans. Bio-Med. Eng. 92, 1–10 (2014)
Yang, W., Zhen, L., Mei, Y., Mei, H., Qian, F., Wu, C.: Content-based retrieval of focal liver lesions using bag-of-visual-words representations of single-and multiphase contrast-enhanced CT images. J. Digit. Imaging 25, 708–719 (2012)
Chi, Y., Zhou, J., Venkatesh, S.K., Tian, Q., Liu, J.: Content-based image retrieval of multiphase CT images or focal liver lesion characterization. Med. Phys. 40(10), 103502 (2013)
Diamant, I., Goldberger, J., Klang, E., Amitai, M.: Multi-phase liver lesions classification using relevant visual words based on mutual information. In: IEEE International Symposium on Biomedical Imaging, pp. 407–410. IEEE (2015)
Napel, S.A., Beaulieu, C.F., Rodriguez, C., Cui, J., Xu, J., Korenblum, D., Greenspan, H., Ma, Y., Rubin, D.L.: Automated retrieval of CT images of liver lesions on the basis of image similarity: method and preliminary results. Radiology 256(1), 243–252 (2010)
Diamant, I., Hoogi, A., Beaulieu, C., Safdari, M., Klang, E., Amitai, M., Greenspan, H., Rubin, D.: Improved patch based automated liver lesion classification by separate analysis of the interior and boundary regions. IEEE J Biomed Health Inf. 20, 1585–1594 (2016)
Xu, Y., Lin, L., Hu, H., et al.: Texture-specific bag of visual words model and spatial cone matching-based method for the retrieval of focal liver lesions using multiphase contrast-enhanced CT images. Int. J. Comput. Assist. Radiol. Surg. 13(1), 151–164 (2018)
Burner, A., Donner, R., Mayerhoefer, M., et al.: Texture bags: anomaly retrieval in medical images based on local 3D-texture similarity. In: MICCAI International Workshop on Medical Content-Based Retrieval for Clinical Decision Support, pp. 116–127 (2011)
Wang, J., Han, X.-H., Xu, Y., Lin, L., Hu, H., Chen, Y.-W.: Sparse codebook model of local structures for retrieval of focal liver lesions using multi-phase medical images. Int. J. Biomed. Imaging 2017, ID1413297 (2017)
Wang, J., Li, J., Han, X.-H., Lin, L., Hu, H., Xu, Y., Chen, Q., Iwamoto, Y., Chen, Y.-W.: Tensor-based sparse representations of multi-phase medical images for classification of focal liver lesions. Pattern Recognit. Lett. 2019 (in press)
Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR2006), pp. 2169–2178 (2006)
Xu, Y., Lin, L., et al.: Bag of temporal co-occurrence words for retrieval of focal liver lesions using 3D multiphase contrast-enhanced CT images. In: Proceedings of the 23rd International Conference on Pattern Recognition, pp. 2283–2288 (2016)
Dong, C., et al.: Simultaneous segmentation of multiple organs using random walks. J. Inf. Process. Soc. Jpn 24(2), 320–329 (2016)
Banerji, S., Sinha, A., Liu, C.: A new bag of words LBP (BoWL) descriptor for scene image classification. In: International Conference on Computer Analysis of Images and Patterns, pp. 490–497 (2013)
Liang, D., Lin, L., Hu, H., Zhang, Q., Chen, Q., Iwamoto, Y., Han, X., Chen, Y.-W.: Combining convolutional and recurrent neural networks for classification of focal liver lesions in multi-phase CT images. In: 21st International Conference on Medical Image Computing and Computer Assisted Intervention, pp. 666–675 (2018)
Acknowledgements
This work was supported in part by the Major Scientific Research Project of Zhejiang Lab under the Grant No. 2018DG0ZX01, in part by the Key Science and Technology Innovation Support Program of Hangzhou under the Grant No. 20172011A038, and in part by the Grant-in-Aid for Scientific Research from the Japanese Ministry for Education, Science, Culture and Sports (MEXT) under the Grant No. 18H03267 and No. 17H00754.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Huang, H. et al. (2019). Multiphase Focal Liver Lesions Classification with Combined N-gram and BoVW. In: Chen, YW., Zimmermann, A., Howlett, R., Jain, L. (eds) Innovation in Medicine and Healthcare Systems, and Multimedia. Smart Innovation, Systems and Technologies, vol 145. Springer, Singapore. https://doi.org/10.1007/978-981-13-8566-7_8
Download citation
DOI: https://doi.org/10.1007/978-981-13-8566-7_8
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-8565-0
Online ISBN: 978-981-13-8566-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)