Extending pretrained segmentation networks with additional anatomical structures
- 77 Downloads
For comprehensive surgical planning with sophisticated patient-specific models, all relevant anatomical structures need to be segmented. This could be achieved using deep neural networks given sufficiently many annotated samples; however, datasets of multiple annotated structures are often unavailable in practice and costly to procure. Therefore, being able to build segmentation models with datasets from different studies and centers in an incremental fashion is highly desirable.
We propose a class-incremental framework for extending a deep segmentation network to new anatomical structures using a minimal incremental annotation set. Through distilling knowledge from the current network state, we overcome the need for a full retraining.
We evaluate our methods on 100 MR volumes from SKI10 challenge with varying incremental annotation ratios. For 50% incremental annotations, our proposed method suffers less than 1% Dice score loss in retaining old-class performance, as opposed to 25% loss of conventional finetuning. Our framework inherently exploits transferable knowledge from previously trained structures to incremental tasks, demonstrated by results superior even to non-incremental training: In a single volume one-shot incremental learning setting, our method outperforms vanilla network performance by>11% in Dice.
With the presented method, new anatomical structures can be learned while retaining performance for older structures, without a major increase in complexity and memory footprint, hence suitable for lifelong class-incremental learning. By leveraging information from older examples, a fraction of annotations can be sufficient for incrementally building comprehensive segmentation models. With our meta-method, a deep segmentation network is extended with only a minor addition per structure, thus can be applicable also for future network architectures.
KeywordsPatient-specific modeling Segmentation Deep learning Class-incremental learning Lifelong learning Intervention planning
This work was funded by the Swiss National Science Foundation (Grant No. 179116) and a Highly Specialized Medicine grant (HSM2) of the Canton of Zurich.
Compliance with ethical standards
Conflict of interest
All authors declare no conflict of interest.
Research involving human participants
All procedures performed in studies involving human participants were in accordance with the ethical standards of the provincial ethics committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.
was obtained from all individual participants included in the study.
- 1.Sloan M, Sheth NP (2018) Projected volume of primary and revision total joint arthroplasty in the united states, 2030–2060. Tech. Rep. 16, annual meeting of the American Academy of Orthopaedic Surgeons, New Orleans, Louisiana, 03Google Scholar
- 2.Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: MICCAI. pp 234–241Google Scholar
- 3.Baumgartner CF, Koch LM, Pollefeys M, Konukoglu E (2018) An exploration of 2D and 3D deep learning techniques for cardiac MR image segmentation. In: Statistical atlases and computational models of the heart. pp 111–119Google Scholar
- 4.Yang L, Zhang Y, Chen J, Zhang S, Chen DZ (2017) Suggestive annotation: a deep active learning framework for biomedical image segmentation. In: MICCAI. pp 399–407Google Scholar
- 5.Ozdemir F, Peng Z, Tanner C, Fuernstahl P, Goksel O (2018) Active learning for segmentation by optimizing content information for maximal entropy. In: MICCAI DLMIA. pp 183–191Google Scholar
- 6.McCloskey M, Cohen NJ (1989) Catastrophic interference in connectionist networks: the sequential learning problem. vol 24, pp 109–165Google Scholar
- 7.Rebuffi S, Kolesnikov A, Sperl G, Lampert CH (2017) iCaRL: Incremental classifier and representation learning. In: IEEE conference on computer vision pattern recog (CVPR). pp 5533–5542Google Scholar
- 8.Castro FM, Marín-Jiménez MJ, Mata NG, Schmid C, Alahari K (2018) End-to-end incremental learning. In: ECCVGoogle Scholar
- 9.Ozdemir F, Fuernstahl P, Goksel O (2018) Learn the new, keep the old: Extending pretrained models with new anatomy and images. In: MICCAI. pp 361–369Google Scholar
- 10.Hinton G, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. In: NIPS deep learning and representation learning workshopGoogle Scholar
- 11.Li Z, Hoiem D (2016) Learning without forgetting. In: ECCV. pp 614–629Google Scholar
- 12.Hochbaum DS (1997) Approximation algorithms for NP-hard problems. PWS Publishing Co., Boston, pp 94–143Google Scholar
- 13.Gal Y, Ghahramani Z (2016) Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In: International conference on machine learning (ICML). pp 1050–1059Google Scholar
- 14.Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958Google Scholar
- 15.Heimann T, Morrison BJ, Styner MA, Niethammer M, Warfield S (2010) Segmentation of knee images: a grand challenge. In MICCAI workshop on medical image analysis for the clinic. pp 207–214Google Scholar