Skip to main content
SpringerLink
Log in

Real Time Data Augmentation Using Fractional Linear Transformations in Continual Learning

  • Conference paper
  • First Online:
Medical Image Learning with Limited and Noisy Data (MILLanD 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13559))

Included in the following conference series:

  • 488 Accesses

Abstract

Over recent years, deep learning algorithms have gained prominence in medical image analysis research. Like other connectionist systems, such networks have been found to be prone to catastrophic forgetting effects. This makes generalization a challenge as new additions to prediction requirements at runtime would invariably require retraining on not only the new dataset, but also substantial portions of older task data. This is a difficult task in clinical imaging where retention of datasets over extended time is challenged by legal and infrastructure constraints. Thus, there is a requirement of algorithmic designs that address for-getting as a part of base and incremental task learning. This has been cast as an incremental learning problem recently. We propose a novel approach to the incremental class addition problem, where a retention of limited numbers of exemplars of old classes helps reduce forgetting instead of large scale data storage, using a strategy of incremental time augmentation with Mobius transformations and weighted distillation objectives to correct evolving class imbalance effects.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y. An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv:1312.6211 (2013)

  2. Ravishankar, H., et al.: Understanding the mechanisms of deep transfer learning for medical images. In: Carneiro, G., et al. (eds.) LABELS/DLMIA -2016. LNCS, vol. 10008, pp. 188–196. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46976-8_20

  3. Patra, A., Chakraborti, T.: Learn more, forget less: cues from human brain. In: Proceedings of the Asian Conference on Computer Vision (2020)

    Google Scholar 

  4. Li, Z., Hoiem, D.: Learning without forgetting. IEEE Trans. Pattern Anal. Mach. Intell. 40(12), 2935–2947 (2017)

    Article  Google Scholar 

  5. Kemker, R., Kanan, C.: Fearnet: brain-inspired model for incremental learning. arXiv:1711.10563 (2017)

  6. Patra, A., Cai, Y., Chatelain, P., Sharma, H., Drukker, L., Papageorghiou, A.T., Noble, J.A.: Multimodal continual learning with sonographer eye-tracking in fetal ultrasound. In: International Workshop on Advances in Simplifying Medical Ultrasound, pp. 14–24 (2020)

    Google Scholar 

  7. Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. In: Proceedings of the National Academy of Sciences, pp. 3521–3526 (2017)

    Google Scholar 

  8. Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. In: NIPS 2014 Deep Learning Workshop (2014)

    Google Scholar 

  9. Patra, A., Noble, J.A.: Multi-anatomy localization in fetal echocardiography videos. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 1761–1764 (2019)

    Google Scholar 

  10. Rebuffi, S.A., Kolesnikov, A., Sperl, G., Lampert, C.H.: iCaRl: incremental classifier and representation learning. In: Proceedings of the IEEE CVPR, pp. 2001–2010 (2017)

    Google Scholar 

  11. Patra, A., et al.: Efficient ultrasound image analysis models with sonographer gaze assisted distillation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11767, pp. 394–402. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32251-9_43

  12. Hou, S., Pan, X., Loy, C.C., Wang, Z., Lin, D.: Lifelong learning via progressive distillation and retrospection. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Yair (eds.) ECCV 2018. LNCS, vol. 11207, pp. 452–467. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01219-9_27

    Chapter  Google Scholar 

  13. Ozdemir, F., Fuernstahl, P., Goksel, O.: Learn the new, keep the old: Extending pretrained models with new anatomy and images. In: Frangi, A., Schnabel, J., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) International Conference on Medical Image Computing and Computer-Assisted Intervention. MICCAI 2018, LNIP, vol. 11073. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_42

  14. Zhang, J.,Wang, Y.: Continually modeling Alzheimer’s disease progression via deep multi- order preserving weight consolidation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. LNIP, vol. 11765, pp. 850–859. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_94

  15. Kim, H.E., Kim, S., Lee, J.: Keep and learn: continual learning by constraining the latent space for knowledge preservation in neural networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, LNCS. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_59

  16. Patra, A., Noble, J.A.: Hierarchical class incremental learning of anatomical structures in fetal echocardiography videos. IEEE J. Biomed. Health Inf. 24 (2020)

    Google Scholar 

  17. Cubuk, E. D., Zoph, B., Mane, D., Vasudevan, V. and Le, Q. V., Autoaugment: Learning augmentation strategies from data. In: Proceedings of the IEEE Conference on Computer ViSion and Pattern Recognition, pp. 113–123 (2019)

    Google Scholar 

  18. Ahlfors, L., Möbius Transformations in Several Dimensions. University of Minnesota, 1989

    Google Scholar 

  19. Özdemir, N., Iskender, B.B., Özgür, N.Y.: Complex valued neural network with Mö-bius activation function. Commun. Nonlinear Sci. Numer. Simul. 16(12), 4698–4703 (2011)

    Article  MathSciNet  Google Scholar 

  20. Zammit-Mangion, A., Ng, T.L.J., Vu, Q., Filippone, M.: Deep compositional spatial models. arXiv preprint arXiv:1906.02840 (2019)

  21. Zhou, S., Zhang, J., Jiang, H., Lundh, T., Ng, A.Y.: Data augmentation with Mobius transformations. arXiv preprint arXiv:2002.02917 (2020)

  22. Ganea, O., Bécigneul, G., Hofmann, T.: Hyperbolic neural networks. In: Advances in Neural Information Processing Systems, pp. 5345–5355 (2018)

    Google Scholar 

  23. Islam, M.A., Anderson, D.T., Pinar, A., Havens, T.C., Scott, G., Keller, J.M.: Enabling explainable fusion in deep learning with fuzzy integral neural networks. IEEE Trans. Fuzzy Syst 28 (2019)

    Google Scholar 

  24. Kather, J., Weis, C., Bianconi, F., Melchers, S.M., Schad, L.R., Gaiser, T., Marx, A., Zöllner, F.G.: Multi-class texture analysis in colorectal cancer histology.’ In Scientific re-ports (2016)

    Google Scholar 

  25. Liouville, J.: Extension au cas des trois dimensions de la question du tracé géographique. Note VI, pp. 609–617 (1850)

    Google Scholar 

  26. Kingma, D.P., Adam, J.B.: A method for stochastic optimization. arXiv:1412.6980

  27. DeVries, T., Taylor, G.W.: Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552 (2017)

  28. Tang, Z., Peng, X., Li, T., Zhu, Y., Metaxas, D.N.: Adatransform: adaptive data transfor-mation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2998–3006 (2019)

    Google Scholar 

  29. Ho, D., Liang, E., Stoica, I., Abbeel, P., Chen, X.: Population based augmentation: efficient learning of augmentation policy schedules. arXiv preprint arXiv:1905.05393 (2019)

  30. Cubuk, E.D., Zoph, B., Shlens, J., Le, Q.V.: Randaugment: practical data augmentation with no separate search. arXiv preprint arXiv:1909.13719 (2019)

  31. Omar, H.A., Patra, A., Domingos, J.S., Leeson, P., Noblel, A.J.: Automated myocardial wall motion classification using handcrafted features vs a deep cnn-based mapping. In: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 3140–3143. IEEE (2018)

    Google Scholar 

  32. Lee, C.S., Lee, A.Y.: Clinical applications of continual learning in machine learning. In: The Lancet Digital Health 2.6, pp. 279–e281 (2020)

    Google Scholar 

  33. Patra, A., Noble, J.A.: Sequential anatomy localization in fetal echocardiography videos. arXiv preprint arXiv:1810.11868 (2019)

  34. Patra, A., Noble, J.A.: Incremental learning of fetal heart anatomies using interpretable saliency maps. In: Zheng, Y., Williams, B., Chen, K. (eds.) Medical Image Understanding and Analysis. MIUA 2019. Communications in Computer and Information Science, CCIS. vol. 1065, pp. 129–141. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-39343-4_11

  35. Patra, A., Huang, W., Noble, J.A.: Learning spatio-temporal aggregation for fetal heart analysis in ultrasound video. In: Cardoso, M.J., et al. (eds.) DLMIA/ML-CDS -2017. LNCS, vol. 10553, pp. 276–284. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67558-9_32

  36. Chakraborti, T., Patra, A., Noble, J.A.: Contrastive fairness in machine learning. IEEE Lett. Comput. Soc. 3(2), 38–41 (2020)

    Article  Google Scholar 

  37. Omar, H.A., Domingos, J.S., Patra, A., Leeson, P., Noble, J.A.: Improving visual detection of wall motion abnormality with echocardiographic image enhancing methods. In: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 1128–1131. IEEE (2018)

    Google Scholar 

  38. Food, Drug Administration, et al.: Proposed regulatory framework for modifications to arti-ficial intelligence/machine learning (AI/ML)-based software as a medical device (SaMD) - discussion paper (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Data and Translational Sciences, UCB Biopharma UK, Slough, SL1 3WE, UK

    Arijit Patra

Authors
  1. Arijit Patra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arijit Patra .

Editor information

Editors and Affiliations

  1. National Institutes of Health, Bethesda, MD, USA

    Ghada Zamzmi

  2. National Institutes of Health, Bethesda, MD, USA

    Sameer Antani

  3. Northwestern University, Chicago, IL, USA

    Ulas Bagci

  4. Children's National Hospital, Washington, WA, USA

    Marius George Linguraru

  5. National Institutes of Health, Bethesda, MD, USA

    Sivaramakrishnan Rajaraman

  6. National Institutes of Health, Bethesda, MD, USA

    Zhiyun Xue

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Patra, A. (2022). Real Time Data Augmentation Using Fractional Linear Transformations in Continual Learning. In: Zamzmi, G., Antani, S., Bagci, U., Linguraru, M.G., Rajaraman, S., Xue, Z. (eds) Medical Image Learning with Limited and Noisy Data. MILLanD 2022. Lecture Notes in Computer Science, vol 13559. Springer, Cham. https://doi.org/10.1007/978-3-031-16760-7_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-16760-7_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-16759-1

  • Online ISBN: 978-3-031-16760-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation