Patient-specific and global convolutional neural networks for robust automatic liver tumor delineation in follow-up CT studies
- 500 Downloads
Radiological longitudinal follow-up of tumors in CT scans is essential for disease assessment and liver tumor therapy. Currently, most tumor size measurements follow the RECIST guidelines, which can be off by as much as 50%. True volumetric measurements are more accurate but require manual delineation, which is time-consuming and user-dependent. We present a convolutional neural networks (CNN) based method for robust automatic liver tumor delineation in longitudinal CT studies that uses both global and patient specific CNNs trained on a small database of delineated images. The inputs are the baseline scan and the tumor delineation, a follow-up scan, and a liver tumor global CNN voxel classifier built from radiologist-validated liver tumor delineations. The outputs are the tumor delineations in the follow-up CT scan. The baseline scan tumor delineation serves as a high-quality prior for the tumor characterization in the follow-up scans. It is used to evaluate the global CNN performance on the new case and to reliably predict failures of the global CNN on the follow-up scan. High-scoring cases are segmented with a global CNN; low-scoring cases, which are predicted to be failures of the global CNN, are segmented with a patient-specific CNN built from the baseline scan. Our experimental results on 222 tumors from 31 patients yield an average overlap error of 17% (std = 11.2) and surface distance of 2.1 mm (std = 1.8), far better than stand-alone segmentation. Importantly, the robustness of our method improved from 67% for stand-alone global CNN segmentation to 100%. Unlike other medical imaging deep learning approaches, which require large annotated training datasets, our method exploits the follow-up framework to yield accurate tumor tracking and failure detection and correction with a small training dataset.
KeywordsLiver tumor segmentation Follow-up CT scans Longitudinal studies Deep learning Convolutional neural networks
This work was partially supported by Grant 53681 from the Israel Ministry of Science, Technology and Space entitled: METASEG: a new medical image segmentation paradigm for clinical decision support and big data radiology.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
Protection of human and animal rights statement
No animals or humans were involved in this research. All scans were anonymized before delivery to the researchers.
- 2.Bourquain H, Schenk A, Link F, Preim B, Peitgen OH (2002) Hepavision2a software assistant for preoperative planning in living related liver transplantation and oncologic liver surgery. Proc 16th Conf Comp Assist Radiol Surg, 341–6Google Scholar
- 5.Cohen AB, Diamant I, Klang E, Amitai M, Greenspan H (2014) Automatic detection and segmentation of liver metastatic lesions on serial CT examinations. Proc SPIE Med Imag Conf, 903519-27Google Scholar
- 6.Deng X, Du G (2008) Proc. MICCAI workshop on 3D segmentation in the clinic: a grand challenge II—liver tumor segmentationGoogle Scholar
- 12.Hong JS, Kaneto T, Sekiguchi R, Park KH (2001) Automatic liver tumor detection from CT. IEEE Trans Inf Syst 84(6):741–748Google Scholar
- 13.Jia Y (2013) Caffe: An open source convolutional architecture for fast feature embedding. http://caffe.berkeleyvision.org
- 15.Lewis RL (2007) Liver and biliary tract tumors. Cecil medicine, 23rd edn. Saunders Elsevier, Philadelphia (Ch. 206)Google Scholar
- 16.Li Y, Hara S, Shimura K (2006) A machine learning approach for locating boundaries of liver tumors in ct images. Proc 18th Int Conf Pattern Recogn, 400–3Google Scholar
- 18.Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. Proc IEEE Conf Comput Vis Pattern Recog :3431–40Google Scholar
- 19.Mala K, Sadasivam V, Avlagappan S (2007) Neural network based texture analysis of liver tumors from computed tomography images. Int J Biomed Sci 2:33–40Google Scholar
- 20.Masuda Y, Tateyama T, Xiong W, Zhou J, Wakamiya M, Kanasaki S, Furukawa A, Chen YW (2011) Liver tumor detection in CT images by adaptive contrast enhancement and the EM/MPM algorithm. 18th IEEE Int Conf Image Process :1421–1424Google Scholar
- 21.Militzer A, Tietjen C, Hornegger J (2013) Learning a prior model for automatic liver lesion segmentation in follow-up CT images. Dept Informatik technical reports, CS-2013-03, ISSN 2191-5008Google Scholar
- 23.Moltz J, Bornemann L, Dicken V, Peitgen H (2008) Segmentation of liver metastases in CT scans by adaptive thresholding and morphological processing. Proc. MICCAI workshop on 3D segmentation in the clinic: a grand challenge II—liver tumor segmentationGoogle Scholar
- 24.Moltz JH, Schwier M, Peitgen HO (2009) A general framework for automatic detection of matching lesions in follow-up CT. Proc IEEE Int Symp Biomed Imag, :843–6Google Scholar
- 25.Pescia D, Paragios N, Chemouny S (2008) Automatic detection of liver tumors. Prof. 5th IEEE Int Symp Biomed Imaging :672–67Google Scholar
- 26.Schmidt G, Binnig G, Kietzmann M, Kim J (2008) Cognition network technology for a fully automated 3D segmentation of liver tumors. Proc. MICCAI workshop on 3D segmentation in the clinic: a grand challenge II—liver tumor segmentationGoogle Scholar
- 27.Shimizu A, Narihira T, Furukawa D, Kobatake H, Nawano S, Shinozaki K (2008) Ensemble segmentation using AdaBoost with application to liver lesion extraction from a CT volume. Proc. MICCAI workshop on 3D segmentation in the clinic: a grand challenge II—liver tumor segmentationGoogle Scholar
- 28.Smeets D, Stijnen B, Loeckx D, Dobbelaer B, Suetens P (2008) Segmentation of liver metastases using a level set method with spiral-scanning technique and supervised fuzzy pixel classification. Proc. MICCAI workshop on 3D segmentation in the clinic: a grand challenge II—liver tumor segmentationGoogle Scholar
- 29.Stawiaski J, Decenciere E, Bidault F (2008) Interactive liver tumor segmentation using graph-cuts and watershed. Proc. MICCAI workshop on 3D segmentation in the clinic: a grand challenge II—liver tumor segmentationGoogle Scholar
- 30.Vivanti R, Joskowicz L, Karaaslan OA, Sosna J (2015a) Automatic lung tumor segmentation with leaks removal in follow-up CT studies. Int J Comp Assist Radiol Surg. https://doi.org/10.1007/s11548-015-1150-0 to appear
- 31.Vivanti R, Ephrat A, Joskowicz L, Lev-Cohain N, Sosna J, (2015b) Automatic liver tumor segmentation in follow-up CT studies using convolutional neural networks. 1st International Workshop on Patch-based Techniques in Medical Imaging, MICCAI workshop Springer International Publishing, 49Google Scholar
- 32.Weizman L, Ben-Sira L, Joskowicz L, Precel R, Constantini S, Ben-Bashat D (2010) Automatic segmentation and components classification of optic pathway gliomas in MRI. Med Image Comput Comput Assist Interv 103–110Google Scholar
- 33.Wong D, Liu J, Fengshou Y, Tian Q, Xiong W, Zhou J, Wang SC (2008) A semi-automated method for liver tumor segmentation based on 2D region growing with knowledge-based constraints. Proc MICCAI Work 41(43):159Google Scholar
- 34.Zhou J, Xiong W, Tian Q, Qi Y, Liu J, Leow WK, Han T, Venkatesh SK, Wang SC (2008) Semi-automatic segmentation of 3D liver tumors from CT scans using voxel classification and propagational learning. Proc MICCAI Work 41:43–51Google Scholar