Skip to main content

Advertisement

SpringerLink
Log in

Incremental learning with neural networks for computer vision: a survey

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Incremental learning is one of the most important abilities of human beings. In the age of artificial intelligence, it is the key task to make neural network models as powerful as human beings, to achieve the ability to continuously acquire, fine-tune, and accumulate knowledge while simultaneously avoid catastrophic forgetting. In recent years, by virtue of deep neural networks, incremental learning has been attracting a great deal of attention in the field of computer vision. In this paper, we systematically review the current development of incremental learning and give the overall taxonomy of the incremental learning methods. Specifically, three kinds of mainstream methods, i.e., parameter regularization-based approaches, knowledge distillation-based approaches, and dynamic architecture-based approaches, are surveyed, summarized, and discussed in detail. Furthermore, we comprehensively analyze the performance of data-permuted incremental learning, class-incremental learning, and multi-modal incremental learning on widely used datasets, covering a broad of incremental learning scenarios for image classification and semantic segmentation. Lastly, we point out some possible research directions and inspiring suggestions for incremental learning in the field of computer vision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Albesano D, Gemello R, Laface P, Mana F, Scanzio S (2006) Adaptation of artificial neural networks avoiding catastrophic forgetting. In: The 2006 IEEE International Joint Conference on Neural Network Proceedings, pp 1554–1561, https://doi.org/10.1109/IJCNN.2006.246618

  • Aljundi R, Chakravarty P, Tuytelaars T (2017) Expert gate: Lifelong learning with a network of experts. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 7120–7129, https://doi.org/10.1109/CVPR.2017.753

  • Aljundi R, Babiloni F, Elhoseiny M, Rohrbach M, Tuytelaars T (2018) Memory aware synapses: learning what (not) to forget. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer Vision—ECCV 2018. Springer, Cham, pp 144–161

    Chapter  Google Scholar 

  • Awasthi A, Sarawagi S (2019) Continual learning with neural networks: a review. In: the ACM India Joint International Conference, pp 362–365

  • Benna MK, Fusi S (2016) Computational principles of synaptic memory consolidation. Nat Neurosci 19(12):1697–1706

    Article  Google Scholar 

  • Castro FM, Marín-Jiménez M, Guil N, Schmid C, Alahari K (2018) End-to-end incremental learning. In: European conference on computer vision, pp 241–257

  • Cauwenberghs G, Poggio T (2001) Incremental and decremental support vector machine learning. Adv Neural Inf Process Syst 13:409–412

    Google Scholar 

  • Cermelli F, Mancini M, Rota Bulò S, Ricci E, Caputo B (2020) Modeling the background for incremental learning in semantic segmentation. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 9230–9239, https://doi.org/10.1109/CVPR42600.2020.00925

  • Chaudhry A, Dokania PK, Ajanthan T, Torr PHS (2018a) Riemannian walk for incremental learning: understanding forgetting and intransigence. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer Vision—ECCV 2018. Springer, Cham, pp 556–572

    Chapter  Google Scholar 

  • Chaudhry A, Ranzato M, Rohrbach M, Elhoseiny M (2018b) Efficient lifelong learning with a-gem. arXiv preprint arXiv:1812.00420

  • Chen X, Gupta A (2015) Webly supervised learning of convolutional networks. In: 2015 IEEE international conference on computer vision (ICCV), pp 1431–1439. https://doi.org/10.1109/ICCV.2015.168

  • Chen X, Mottaghi R, Liu X, Fidler S, Urtasun R, Yuille A (2014) Detect what you can: Detecting and representing objects using holistic models and body parts. In: Proceedings of the 2014 IEEE conference on computer vision and pattern recognition, IEEE Computer Society, USA, pp 1979–1986, https://doi.org/10.1109/CVPR.2014.254

  • Chen L, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587

  • Cichon J, Gan WB (2015) Branch-specific dendritic ca2+ spikes cause persistent synaptic plasticity. Nature 520(7546):180–185

    Article  Google Scholar 

  • Collobert R, Weston J (2008) A unified architecture for natural language processing: Deep neural networks with multitask learning. In: Proceedings of the 25th international conference on machine learning, pp 160–167

  • Cordts M, Omran M, Ramos S, Rehfeld T, Enzweiler M, Benenson R, Franke U, Roth S, Schiele B (2016) The cityscapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 3213–3223

  • Dhar P, Singh RV, Peng K, Wu Z, Chellappa R (2019) Learning without memorizing. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 5133–5141. https://doi.org/10.1109/CVPR.2019.00528

  • Douillard A, Chen Y, Dapogny A, Cord M (2021) Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 4040–4050

  • Everingham M, Gool LV, Williams C, Winn J, Zisserman A (2010) The pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  • Everingham M, Eslami S, Gool LV, Williams C, Winn J, Zisserman A (2015) The pascal visual object classes challenge: a retrospective. Int J Comput Vis 111(1):98–136

    Article  Google Scholar 

  • Fayek HM, Cavedon L, Wu HR (2020) Progressive learning: a deep learning framework for continual learning. Neural Netw 128:345–357. https://doi.org/10.1016/j.neunet.2020.05.011

    Article  MATH  Google Scholar 

  • Fernando C, Banarse D, Blundell C, Zwols Y, Ha D, Rusu AA, Pritzel A, Wierstra D (2017) Pathnet: evolution channels gradient descent in super neural networks. arXiv preprint arXiv:1701.08734

  • French RM (2003) Catastrophic interference in connectionist networks. In: Nadel L (Ed) Encyclopedia of cognitive science, vol 1, pp 431–435

  • Fu J, Zheng H, Mei T (2017) Look closer to see better: recurrent attention convolutional neural network for fine-grained image recognition. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 4476–4484. https://doi.org/10.1109/CVPR.2017.476

  • Furlanello T, Zhao J, Saxe AM, Itti L, Tjan BS (2016) Active long term memory networks. arXiv preprint arXiv:1606.02355

  • Fusi S, Drew PJ, Abbott LF (2005) Cascade models of synaptically stored memories. Neuron 45(4):599–611

    Article  Google Scholar 

  • Gais S, Albouy G, Boly M, Dang-Vu TT, Darsaud A, Desseilles M, Rauchs G, Schabus M, Sterpenich V, Vandewalle G (2007) Sleep transforms the cerebral trace of declarative memories. Proc Natl Acad Sci 104(47):18778–18783

    Article  Google Scholar 

  • Garcia-Garcia A, Orts-Escolano S, Oprea S, Villena-Martinez V, Garcia-Rodriguez J (2017) A review on deep learning techniques applied to semantic segmentation. arXiv preprint arXiv:1704.06857

  • Gepperth A, Karaoguz C (2016) A bio-inspired incremental learning architecture for applied perceptual problems. Cogn Comput 8(5):924–934

    Article  Google Scholar 

  • Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems. https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf

  • Hayashi-Takagi A, Yagishita S, Nakamura M, Shirai F, Wu Y, Loshbaugh A, Kuhlman B, Hahn K, Kasai H (2015) Labelling and optical erasure of synaptic memory traces in the motor cortex. Nature 525(7569):333–338. https://doi.org/10.1038/nature15257

    Article  Google Scholar 

  • Hayes TL, Kemker R, Cahill ND, Kanan C (2018) New metrics and experimental paradigms for continual learning. In: 2018 IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW), pp 2031–2034. https://doi.org/10.1109/CVPRW.2018.00273

  • He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284

    Article  Google Scholar 

  • He X, Jaeger H (2018) Overcoming catastrophic interference using conceptor-aided backpropagation. In: International conference on learning representations. https://openreview.net/forum?id=B1al7jg0b

  • He K, Zhang X, Ren S, Sun J (2016a) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  • He K, Zhang X, Ren S, Sun J (2016b) Identity mappings in deep residual networks. Computer Vision—ECCV 2016. Springer, Cham, pp 630–645

    Chapter  Google Scholar 

  • He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: 2017 IEEE international conference on computer vision (ICCV), pp 2980–2988. https://doi.org/10.1109/ICCV.2017.322

  • Hinton GE, Plaut DC (1987) Using fast weights to deblur old memories. In: Proceedings of the 9th annual conference of the cognitive science society, Erlbaum, pp 177–186

  • Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. Comput Sci 3(4):212–223

    Google Scholar 

  • Hinton G, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. Comput Sci 14(7):38–39

    Google Scholar 

  • Hoens TR, Polikar R, Chawla NV (2012) Learning from streaming data with concept drift and imbalance: an overview. Prog Artif Intell 1(1):89–101

    Article  Google Scholar 

  • Hou S, Pan X, Loy CC, Wang Z, Lin D (2019) Learning a unified classifier incrementally via rebalancing. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 831–839. https://doi.org/10.1109/CVPR.2019.00092

  • Hu X, Tang K, Miao C, Hua XS, Zhang H (2021) Distilling causal effect of data in class-incremental learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 3957–3966

  • Japkowicz N, Stephen S (2002) The class imbalance problem: a systematic study. Intell Data Anal 6(5):429–449

    Article  MATH  Google Scholar 

  • Javed K, White M (2019) Meta-learning representations for continual learning. In: Wallach H, Larochelle H, Beygelzimer A, d’ Alché-Buc F, Fox E, Garnett R (eds) Advances in neural information processing systems, Curran Associates, Inc., vol 32. https://proceedings.neurips.cc/paper/2019/file/f4dd765c12f2ef67f98f3558c282a9cd-Paper.pdf

  • Jung H, Ju J, Jung M, Kim J (2016) Less-forgetting learning in deep neural networks. arXiv preprint arXiv:1607.00122

  • Kemker R, Kanan C (2017) Fearnet: Brain-inspired model for incremental learning. arXiv preprint arXiv:1711.10563

  • Kemker R, Mcclure M, Abitino A, Hayes T, Kanan C (2017) Measuring catastrophic forgetting in neural networks. In: AAAI conference on artificial intelligence, pp 3390–3398

  • Kirkpatrick J, Pascanu R, Rabinowitz N, Veness J, Desjardins G, Rusu AA, Milan K, Quan J, Ramalho T, Grabska-Barwinska A (2017) Overcoming catastrophic forgetting in neural networks. Proc Natl Acad Sci 114(13):3521–3526

    Article  MathSciNet  MATH  Google Scholar 

  • Kitamura T, Ogawa SK, Roy DS, Okuyama T, Morrissey M, Smith LM, Redondo RL, Tonegawa S (2017) Engrams and circuits crucial for systems consolidation of a memory. Science 356(6333):73–78

    Article  Google Scholar 

  • Klingner M, Bär A, Donn P, Fingscheidt T (2020) Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd international conference on intelligent transportation systems (ITSC), pp 1–8. https://doi.org/10.1109/ITSC45102.2020.9294483

  • Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. University of Toronto, Tech Rep, Computer Science Department, p 1

  • Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems, vol 25. Curran Associates, Inc.. https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf

  • Kumaran D, Hassabis D, McClelland JL (2016) What learning systems do intelligent agents need? complementary learning systems theory updated. Trends Cogn Sci 20(7):512–534. https://doi.org/10.1016/j.tics.2016.05.004

    Article  Google Scholar 

  • Lange S, Grieser G (2002) On the power of incremental learning. Theoret Comput Sci 2(288):277–307

    Article  MathSciNet  MATH  Google Scholar 

  • Lecun Y, Cortes C (2010) The mnist database of handwritten digits. http://yann.lecun.com/exdb/mnist/

  • LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324. https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  • Lee SW, Kim JH, Jun J, Ha JW, Zhang BT (2017) Overcoming catastrophic forgetting by incremental moment matching. In: Advances in neural information processing systems, pp 4652–4662

  • Lee J, Joo D, Hong HG, Kim J (2020) Residual continual learning. Proc AAAI Conf Artif Intell 34(4):4553–4560

    Google Scholar 

  • Lesort T, Gepperth A, Stoian A, Filliat D (2019) Marginal replay vs conditional replay for continual learning. In: International conference on artificial neural networks, Springer, pp 466–480

  • Lesort T, Lomonaco V, Stoian A, Maltoni D, Filliat D, Díaz-Rodríguez N (2020) Continual learning for robotics: definition, framework, learning strategies, opportunities and challenges. Inf Fusion 58:52–68

    Article  Google Scholar 

  • Li Z, Hoiem D (2017) Learning without forgetting. IEEE Trans Pattern Anal Mach Intell 40(12):2935–2947

    Article  Google Scholar 

  • Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: common objects in context. In: European conference on computer vision, pp 740–755

  • Liu X, Gao J, He X, Deng l, Duh K, Wang YY (2015) Representation learning using multi-task deep neural networks for semantic classification and information retrieval. In: European conference on computer vision, NAACL, pp 912–921. https://doi.org/10.3115/v1/N15-1092

  • Liu X, Masana M, Herranz L, Van de Weijer J, López AM, Bagdanov AD (2018) Rotate your networks: better weight consolidation and less catastrophic forgetting. In: 2018 24th international conference on pattern recognition (ICPR), pp 2262–2268. https://doi.org/10.1109/ICPR.2018.8545895

  • Liu Y, Parisot S, Slabaugh G, Jia X, Leonardis A, Tuytelaars T (2020a) More classifiers, less forgetting: a generic multi-classifier paradigm for incremental learning. In: Computer Vision—ECCV 2020, Springer, pp 699–716. https://doi.org/10.1007/978-3-030-58574-7_42

  • Liu Y, Su Y, Liu AA, Schiele B, Sun Q (2020b) Mnemonics training: multi-class incremental learning without forgetting. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp 12245–12254. https://doi.org/10.1109/cvpr42600.2020.01226

  • Lomonaco V, Maltoni D (2017) Core50: a new dataset and benchmark for continuous object recognition. In: Levine S, Vanhoucke V, Goldberg K (eds) Proceedings of the 1st Annual Conference on Robot Learning, PMLR, Proceedings of machine learning research, vol 78, pp 17–26. https://proceedings.mlr.press/v78/lomonaco17a.html

  • Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 3431–3440

  • Lopez-Paz D, Ranzato MA (2017) Gradient episodic memory for continual learning. In: Advances in neural information processing systems, pp 6470–6479

  • Losing V, Hammer B, Wersing H (2018) Incremental on-line learning: a review and comparison of state of the art algorithms. Neurocomputing 275(JAN.31):1261–1274

    Article  Google Scholar 

  • MacKay David JC (1992) A practical bayesian framework for backpropagation networks. Neural Comput 4(3):448–472

    Article  Google Scholar 

  • Maltoni D, Lomonaco V (2019) Continuous learning in single-incremental-task scenarios. Neural Netw 116:56–73. https://doi.org/10.1016/j.neunet.2019.03.010

    Article  Google Scholar 

  • Maracani A, Michieli U, Toldo M, Zanuttigh P (2021) RECALL: replay-based continual learning in semantic segmentation. In: 2021 IEEE/CVF international conference on computer vision (ICCV). IEEE, pp 7026–7035. https://doi.org/10.1109/iccv48922.2021.00694

  • Mcclelland J, Mcnaughton B, O’Reilly R (1995) Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol Rev 102(3):419–57

    Article  Google Scholar 

  • McCloskey M, Cohen NJ (1989) Catastrophic interference in connectionist networks: the sequential learning problem. In: Psychology of learning and motivation, vol 24, Elsevier, pp 109–165

  • Mel M, Michieli U, Zanuttigh P (2020) Incremental and multi-task learning strategies for coarse-to-fine semantic segmentation. Technologies 8(1):1. https://doi.org/10.3390/technologies8010001

    Article  Google Scholar 

  • Mermillod M, Bugaiska A, Bonin P (2013) The stability-plasticity dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effects. Front Psychol 4(504):1–3. https://doi.org/10.3389/fpsyg.2013.00504

    Article  Google Scholar 

  • Michieli U, Zanuttigh P (2019) Incremental learning techniques for semantic segmentation. In: 2019 IEEE/CVF international conference on computer vision workshop (ICCVW), pp 3205–3212, https://doi.org/10.1109/ICCVW.2019.00400

  • Michieli U, Zanuttigh P (2021) Knowledge distillation for incremental learning in semantic segmentation. Comput Vis Image Underst 205:103167. https://doi.org/10.1016/j.cviu.2021.103167

    Article  Google Scholar 

  • Mirza M, Osindero S (2014) Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784

  • Mottaghi R, Chen X, Liu X, Cho N, Lee S, Fidler S, Urtasun R, Yuille AL (2014) The role of context for object detection and semantic segmentation in the wild. IEEE Trans Magn 47(5):1302–1305

    Google Scholar 

  • Muhlbaier M, Topalis A, Polikar R (2009) Learn\(^{++}\) .nc: combining ensemble of classifiers with dynamically weighted consult-and-vote for efficient incremental learning of new classes. IEEE Trans Neural Netw 20(1):152–168

    Article  Google Scholar 

  • Nadal JP, Toulouse G, Changeux JP, Dehaene S (1986) Networks of formal neurons and memory palimpsests. Europhys Lett 1(10):535–542

    Article  Google Scholar 

  • Netzer Y, Wang T, Coates A, Bissacco A, Wu B, Ng AY (2011) Reading digits in natural images with unsupervised feature learning. In: NIPS workshop on deep learning and unsupervised feature learning 2011, pp 1–9. http://ufldl.stanford.edu/housenumbers/nips2011_housenumbers.pdf

  • Nguyen CV, Li Y, Bui TD, Turner RE (2018) Variational continual learning. arXiv preprint arXiv:1710.10628

  • Odena A, Olah C, Shlens J (2017) Conditional image synthesis with auxiliary classifier GANs. In: Precup D, Teh YW (eds) Proceedings of the 34th international conference on machine learning, PMLR, Proceedings of machine learning research, vol 70, pp 2642–2651. https://proceedings.mlr.press/v70/odena17a.html

  • Ozdemir F, Goksel O (2019) Extending pretrained segmentation networks with additional anatomical structures. Int J Comput Assist Radiol Surg 14:1187–1195. https://doi.org/10.1007/s11548-019-01984-4

    Article  Google Scholar 

  • Ozdemir F, Fuernstahl P, Goksel O (2018) Learn the new, keep the old: extending pretrained models with new anatomy and images. In: Frangi AF, Schnabel JA, Davatzikos C, Alberola-López C, Fichtinger G (eds) Medical image computing and computer assisted intervention—MICCAI 2018. Springer, Cham, pp 361–369

    Chapter  Google Scholar 

  • Paik JK, Katsaggelos AK (1992) Image restoration using a modified hopfield network. IEEE Trans Image Process 1(1):49–63

    Article  Google Scholar 

  • Pan P, Swaroop S, Immer A, Eschenhagen R, Turner R, Khan MEE (2020) Continual deep learning by functional regularisation of memorable past. In: Larochelle H, Ranzato M, Hadsell R, Balcan MF, Lin H (eds) Advances in neural information processing systems, Curran Associates, Inc., vol 33, pp 4453–4464. https://proceedings.neurips.cc/paper/2020/file/2f3bbb9730639e9ea48f309d9a79ff01-Paper.pdf

  • Parisi GI, Tani J, Weber C, Wermter S (2017) Lifelong learning of human actions with deep neural network self-organization. Neural Netw 96:137–149

    Article  Google Scholar 

  • Parisi GI, Tani J, Weber C, Wermter S (2018) Lifelong learning of spatiotemporal representations with dual-memory recurrent self-organization. Front Neurorobot 12:78. https://doi.org/10.3389/fnbot.2018.00078

    Article  Google Scholar 

  • Parisi G, Kemker R, Part J, Kanan C, Wermter S (2019) Continual lifelong learning with neural networks: a review. Neural Netw 113:54–71. https://doi.org/10.1016/j.neunet.2019.01.012

    Article  Google Scholar 

  • Rajasegaran J, Khan S, Hayat M, Khan FS, Shah M (2020) iTAML: an incremental task-agnostic meta-learning approach. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), IEEE, pp 13588–13597. https://doi.org/10.1109/cvpr42600.2020.01360

  • Ratcliff R (1990) Connectionist models of recognition memory. Psychol Rev 97(2):285–308

    Article  Google Scholar 

  • Rebuffi S, Kolesnikov A, Sperl G, Lampert CH (2017) icarl: Incremental classifier and representation learning. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 5533–5542. https://doi.org/10.1109/CVPR.2017.587

  • Ren S, He K, Girshick R, Sun J (2017) Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  • Robins A (1995) Catastrophic forgetting, rehearsal and pseudorehearsal. Connect Sci 7(2):123–146

    Article  Google Scholar 

  • Ruping S (2001) Incremental learning with support vector machines. In: Proceedings 2001 IEEE international conference on data mining, pp 641–642. https://doi.org/10.1109/ICDM.2001.989589

  • Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    Article  MathSciNet  Google Scholar 

  • Rusu AA, Rabinowitz NC, Desjardins G, Soyer H, Kirkpatrick J, Kavukcuoglu K, Pascanu R, Hadsell R (2016) Progressive neural networks. arXiv preprint arXiv:1606.04671

  • Schwarz J, Luketina J, Czarnecki WM, Grabska-Barwinska A, Teh YW, Pascanu R, Hadsell R (2018) Progress & compress: a scalable framework for continual learning. In: International conference on machine learning, pp 4535–4544

  • Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: visual explanations from deep networks via gradient-based localization. In: 2017 IEEE international conference on computer vision (ICCV), pp 618–626. https://doi.org/10.1109/ICCV.2017.74

  • Serra J, Suris D, Miron M, Karatzoglou A (2018) Overcoming catastrophic forgetting with hard attention to the task. In: International conference on machine learning, PMLR, pp 4548–4557

  • Shin H, Lee JK, Kim J, Kim J (2017) Continual learning with deep generative replay. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems, vol 30, pp 2290–2999. https://proceedings.neurips.cc/paper/2017/file/0efbe98067c6c73dba1250d2beaa81f9-Paper.pdf

  • Shmelkov K, Schmid C, Alahari K (2017) Incremental learning of object detectors without catastrophic forgetting. In: 2017 IEEE international conference on computer vision (ICCV), pp 3420–3429. https://doi.org/10.1109/ICCV.2017.368

  • Soltoggio Andrea (2015) Short-term plasticity as cause-effect hypothesis testing in distal reward learning. Biol Cybern 109(1):75–94

    Article  MathSciNet  MATH  Google Scholar 

  • Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  • Swaroop S, Nguyen CV, Bui TD, Turner RE (2019) Improving and understanding variational continual learning. arXiv preprint arXiv:1905.02099

  • Tasar O, Tarabalka Y, Alliez P (2019) Incremental learning for semantic segmentation of large-scale remote sensing data. IEEE J Sel Topics Appl Earth Observ Remote Sens 12(9):3524–3537. https://doi.org/10.1109/JSTARS.2019.2925416

    Article  Google Scholar 

  • Terekhov AV, Montone G, O’Regan JK (2015) Knowledge transfer in deep block-modular neural networks. In: Biomimetic and biohybrid systems, pp 268–279

  • Titsias MK, Schwarz J, de G Matthews AG, Pascanu R, Teh YW (2019) Functional regularisation for continual learning using gaussian processes. arXiv preprint arXiv:1901.11356

  • Wang YX, Ramanan D, Hebert M (2017) Growing a brain: fine-tuning by increasing model capacity. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2471–2480

  • Welinder P, Branson S, Wah C, Schroff F, Belongie S, Perona P (2010) Caltech-ucsd birds 200. Tech. rep., California Institute of Technology, http://vision.caltech.edu/visipedia/CUB-200.html

  • Welling M (2009) Herding dynamical weights to learn. In: Proceedings of the 26th annual international conference on machine learning, pp 1121–1128

  • Wu C, Herranz L, Liu X, Wang Y, Weijer Jvd, Raducanu B (2018) Memory replay gans: Learning to generate images from new categories without forgetting. In: Advances in neural information processing systems, pp 5966–5976

  • Wu Y, Chen Y, Wang L, Ye Y, Liu Z, Guo Y, Fu Y (2019) Large scale incremental learning. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 374–382. https://doi.org/10.1109/CVPR.2019.00046

  • Xiao H, Rasul K, Vollgraf R (2017) Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747

  • Yang G, Pan F, Gan WB (2009) Stably maintained dendritic spines are associated with lifelong memories. Nature 462:920–924. https://doi.org/10.1038/nature08577

    Article  Google Scholar 

  • Yoon J, Yang E, Lee J, Hwang SJ (2017) Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547

  • Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? In: Proceedings of the 27th International conference on neural information processing systems, MIT Press, Cambridge, MA, USA, NIPS’14, vol 2, pp 3320–3328

  • Zenke F, Agnes EJ, Gerstner W (2015) Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks. Nat Commun 6(1):1–13

    Article  Google Scholar 

  • Zenke F, Poole B, Ganguli S (2017) Continual learning through synaptic intelligence. Int Conf Mach Learn 70:3987–3995

    Google Scholar 

  • Zhai M, Chen L, Tung F, He J, Nawhal M, Mori G (2019) Lifelong gan: continual learning for conditional image generation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 2759–2768

  • Zhang H, Dana K, Shi J, Zhang Z, Wang X, Tyagi A, Agrawal A (2018a) Context encoding for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7151–7160

  • Zhang S, Wen L, Bian X, Lei Z, Li SZ (2018b) Single-shot refinement neural network for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4203–4212

  • Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2881–2890

  • Zhou G, Sohn K, Lee H (2012) Online incremental feature learning with denoising autoencoders. In: Artificial intelligence and statistics, PMLR, pp 1453–1461

  • Zhou B, Zhao H, Puig X, Fidler S, Barriuso A, Torralba A (2017) Scene parsing through ade20k dataset. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 633–641

  • Zhu JY, Zhang R, Pathak D, Darrell T, Efros A, Wang O, Shechtman E (2017) Toward multimodal image-to-image translation. In: Conference on neural information processing systems, pp 465–476

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Nos. 61806206, 61772530, 62172417, 62106268), and the Natural Science Foundation of Jiangsu Province (Nos. BK20180639, BK20201346), the Six Talent Peaks Project in Jiangsu Province (Nos. 2015-DZXX-010, 2018-XYDXX-044).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, 221116, China

    Hao Liu, Yong Zhou, Bing Liu, Jiaqi Zhao, Rui Yao & Zhiwen Shao

  2. Mine Digitization Engineering Research Center of Ministry of Education of the People’s Republic of China, Xuzhou, 221116, China

    Hao Liu, Yong Zhou, Bing Liu, Jiaqi Zhao, Rui Yao & Zhiwen Shao

Authors
  1. Hao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiaqi Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rui Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhiwen Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Zhou.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, H., Zhou, Y., Liu, B. et al. Incremental learning with neural networks for computer vision: a survey. Artif Intell Rev 56, 4557–4589 (2023). https://doi.org/10.1007/s10462-022-10294-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-022-10294-2

Keywords

Search

Navigation