Skip to main content

Advertisement

SpringerLink
Log in

LAMA: Lesion-Aware Mixup Augmentation for Skin Lesion Segmentation

  • Published:
Journal of Imaging Informatics in Medicine Aims and scope Submit manuscript

Abstract

Deep learning can exceed dermatologists’ diagnostic accuracy in experimental image environments. However, inaccurate segmentation of images with multiple skin lesions can be seen with current methods. Thus, information present in multiple-lesion images, available to specialists, is not retrievable by machine learning. While skin lesion images generally capture a single lesion, there may be cases in which a patient’s skin variation may be identified as skin lesions, leading to multiple false positive segmentations in a single image. Conversely, image segmentation methods may find only one region and may not capture multiple lesions in an image. To remedy these problems, we propose a novel and effective data augmentation technique for skin lesion segmentation in dermoscopic images with multiple lesions. The lesion-aware mixup augmentation (LAMA) method generates a synthetic multi-lesion image by mixing two or more lesion images from the training set. We used the publicly available International Skin Imaging Collaboration (ISIC) 2017 Challenge skin lesion segmentation dataset to train the deep neural network with the proposed LAMA method. As none of the previous skin lesion datasets (including ISIC 2017) has considered multiple lesions per image, we created a new multi-lesion (MuLe) segmentation dataset utilizing publicly available ISIC 2020 skin lesion images with multiple lesions per image. MuLe was used as a test set to evaluate the effectiveness of the proposed method. Our test results show that the proposed method improved the Jaccard score 8.3% from 0.687 to 0.744 and the Dice score 5% from 0.7923 to 0.8321 over a baseline model on MuLe test images. On the single-lesion ISIC 2017 test images, LAMA improved the baseline model’s segmentation performance by 0.08%, raising the Jaccard score from 0.7947 to 0.8013 and the Dice score 0.6% from 0.8714 to 0.8766. The experimental results showed that LAMA improved the segmentation accuracy on both single-lesion and multi-lesion dermoscopic images. The proposed LAMA technique warrants further study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Algorithm 1
Algorithm 2
Algorithm 3
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availability

The test set, publicly available, is referenced on the description of the image data sets in the Introduction.

References

  1. R. L. Siegel, K. D. Miller, N. S. Wagle, and A. Jemal, Cancer statistics, 2023, CA Cancer J Clin, vol. 73, no. 1, pp. 17–48, 2023, https://doi.org/10.3322/caac.21763.

    Article  PubMed  Google Scholar 

  2. H. Pehamberger, M. Binder, A. Steiner, and K. Wolff, In vivo epiluminescence microscopy: Improvement of early diagnosis of melanoma, Journal of Investigative Dermatology, vol. 100, no. 3 SUPPL., pp. S356–S362, 1993, https://doi.org/10.1038/jid.1993.63.

    Article  Google Scholar 

  3. H. P. Soyer, G. Argenziano, R. Talamini, and S. Chimenti, Is Dermoscopy Useful for the Diagnosis of Melanoma?, Arch Dermatol, vol. 137, no. 10, pp. 1361–1363, Oct. 2001, https://doi.org/10.1001/archderm.137.10.1361.

    Article  CAS  PubMed  Google Scholar 

  4. R. P. Braun, H. S. Rabinovitz, M. Oliviero, A. W. Kopf, and J. H. Saurat, Pattern analysis: a two-step procedure for the dermoscopic diagnosis of melanoma, Clin Dermatol, vol. 20, no. 3, pp. 236–239, May 2002, https://doi.org/10.1016/S0738-081X(02)00216-X.

    Article  PubMed  Google Scholar 

  5. A. Krizhevsky, I. Sutskever, and G. Hinton, ImageNet Classification with Deep Convolutional Neural Networks, in Advances in Neural Information and Processing Systems (NIPS), 2012, pp. 1097–1105.

  6. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, Rethinking the inception architecture for computer vision, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818–2826.

  7. K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556, 2014.

  8. I. Goodfellow et al., Generative adversarial networks, Commun ACM, vol. 63, no. 11, pp. 139–144, 2020.

    Article  MathSciNet  Google Scholar 

  9. K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.

  10. A. Dosovitskiy et al., An image is worth 16x16 words: Transformers for image recognition at scale, arXiv preprint arXiv:2010.11929, 2020.

  11. O. Ronneberger, P. Fischer, and T. Brox, U-Net: Convolutional Networks for Biomedical Image Segmentation. [Online]. Available: http://lmb.informatik.uni-freiburg.de/

  12. M. Tan and Q. Le, Efficientnet: Rethinking model scaling for convolutional neural networks, in International conference on machine learning, 2019, pp. 6105–6114.

  13. A. Esteva et al., Dermatologist-level classification of skin cancer with deep neural networks, Nature, vol. 542, no. 7639, pp. 115–118, 2017, https://doi.org/10.1038/nature21056.

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  14. V. Gulshan et al., Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs, JAMA, vol. 316, no. 22, pp. 2402–2410, 2016.

    Article  PubMed  Google Scholar 

  15. S. Sornapudi et al., Deep learning nuclei detection in digitized histology images by superpixels, J Pathol Inform, vol. 9, no. 1, p. 5, 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  16. G. Litjens et al., A survey on deep learning in medical image analysis, Med Image Anal, vol. 42, pp. 60–88, 2017, https://doi.org/10.1016/j.media.2017.07.005.

    Article  PubMed  Google Scholar 

  17. L. K. Ferris et al., Computer-aided classification of melanocytic lesions using dermoscopic images, J Am Acad Dermatol, vol. 73, no. 5, pp. 769–776, Nov. 2015, https://doi.org/10.1016/J.JAAD.2015.07.028.

    Article  PubMed  Google Scholar 

  18. M. A. Marchetti et al., Results of the 2016 International Skin Imaging Collaboration International Symposium on Biomedical Imaging challenge: Comparison of the accuracy of computer algorithms to dermatologists for the diagnosis of melanoma from dermoscopic images, J Am Acad Dermatol, vol. 78, no. 2, pp. 270-277.e1, Feb. 2018, https://doi.org/10.1016/j.jaad.2017.08.016.

    Article  PubMed  Google Scholar 

  19. H. A. Haenssle et al., Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists, Annals of Oncology, vol. 29, no. 8, pp. 1836–1842, 2018, https://doi.org/10.1093/annonc/mdy166.

    Article  CAS  PubMed  Google Scholar 

  20. N. C. F. Codella et al., Deep Learning Ensembles for Melanoma Recognition in Dermoscopy Images, IBM J. Res. Dev., vol. 61, no. 4–5, pp. 5:1–5:15, Jul. 2017. https://doi.org/10.1147/JRD.2017.2708299.

  21. S. Pathan, K. G. Prabhu, and P. C. Siddalingaswamy, Techniques and algorithms for computer aided diagnosis of pigmented skin lesions—A review, Biomed Signal Process Control, vol. 39, pp. 237–262, Jan. 2018, https://doi.org/10.1016/J.BSPC.2017.07.010.

    Article  Google Scholar 

  22. T. Majtner, S. Yildirim-Yayilgan, and J. Y. Hardeberg, Combining deep learning and hand-crafted features for skin lesion classification, 2016 6th International Conference on Image Processing Theory, Tools and Applications, IPTA 2016, 2017. https://doi.org/10.1109/IPTA.2016.7821017.

  23. N. Codella, J. Cai, M. Abedini, R. Garnavi, A. Halpern, and J. R. Smith, Deep Learning, Sparse Coding, and SVM for Melanoma Recognition in Dermoscopy Images BT - Machine Learning in Medical Imaging, L. Zhou, L. Wang, Q. Wang, and Y. Shi, Eds., Cham: Springer International Publishing, 2015, pp. 118–126.

  24. I. González-Díaz, DermaKNet: Incorporating the Knowledge of Dermatologists to Convolutional Neural Networks for Skin Lesion Diagnosis, IEEE J Biomed Health Inform, vol. 23, no. 2, pp. 547–559, 2019, https://doi.org/10.1109/JBHI.2018.2806962.

    Article  PubMed  Google Scholar 

  25. J. R. Hagerty et al., Deep Learning and Handcrafted Method Fusion: Higher Diagnostic Accuracy for Melanoma Dermoscopy Images, IEEE J Biomed Health Inform, vol. 23, no. 4, pp. 1385–1391, 2019, https://doi.org/10.1109/JBHI.2019.2891049.

    Article  PubMed  Google Scholar 

  26. A. K. Nambisan et al., Improving Automatic Melanoma Diagnosis Using Deep Learning-Based Segmentation of Irregular Networks, Cancers (Basel), vol. 15, no. 4, 2023. https://doi.org/10.3390/cancers15041259.

  27. G. Celebi, Emre M.; Wen, Quan; Iyatomi, Hitoshi; Shimizu, Kouhei; Zhou, Huiyu; Schaefer, A State-of-the-Art on Lesion Border Detection in Dermoscopy Images, in Dermoscopy Image Analysis, J. S. Celebi, M. Emre; Mendonca, Teresa; Marques, Ed., Boca Raton: CRC Press, 2015, pp. 97–129. [Online]. Available: https://doi.org/10.1201/b19107

  28. N. K. Mishra et al., Automatic lesion border selection in dermoscopy images using morphology and color features, Skin Research and Technology, vol. 25, no. 4, pp. 544–552, 2019.

    Article  PubMed  Google Scholar 

  29. M. E. Celebi, H. Iyatomi, G. Schaefer, and W. v Stoecker, Lesion border detection in dermoscopy images, Computerized Medical Imaging and Graphics, vol. 33, no. 2, pp. 148–153, 2009. https://doi.org/10.1016/j.compmedimag.2008.11.002.

  30. M. A. Al-masni, M. A. Al-antari, M. T. Choi, S. M. Han, and T. S. Kim, Skin lesion segmentation in dermoscopy images via deep full resolution convolutional networks, Comput Methods Programs Biomed, vol. 162, pp. 221–231, 2018, https://doi.org/10.1016/j.cmpb.2018.05.027.

    Article  PubMed  Google Scholar 

  31. P. Tschandl, C. Sinz, and H. Kittler, Domain-specific classification-pretrained fully convolutional network encoders for skin lesion segmentation, Comput Biol Med, vol. 104, pp. 111–116, 2019, https://doi.org/10.1016/j.compbiomed.2018.11.010.

    Article  PubMed  Google Scholar 

  32. Y. Yuan and Y. C. Lo, Improving Dermoscopic Image Segmentation With Enhanced Convolutional-Deconvolutional Networks, IEEE J Biomed Health Inform, vol. 23, no. 2, pp. 519–526, 2019, https://doi.org/10.1109/JBHI.2017.2787487.

    Article  MathSciNet  PubMed  Google Scholar 

  33. F. Xie, J. Yang, J. Liu, Z. Jiang, Y. Zheng, and Y. Wang, Skin lesion segmentation using high-resolution convolutional neural network, Comput Methods Programs Biomed, vol. 186, p. 105241, 2020, https://doi.org/10.1016/j.cmpb.2019.105241.

    Article  PubMed  Google Scholar 

  34. Ş. Öztürk and U. Özkaya, Skin Lesion Segmentation with Improved Convolutional Neural Network, J Digit Imaging, vol. 33, no. 4, pp. 958–970, 2020, https://doi.org/10.1007/s10278-020-00343-z.

    Article  PubMed  PubMed Central  Google Scholar 

  35. N. Lama, J. Hagerty, A. Nambisan, R. J. Stanley, and W. V. Stoecker, Skin Lesion Segmentation in Dermoscopic Images with Noisy Data, J Digit Imaging, 2023, https://doi.org/10.1007/s10278-023-00819-8.

    Article  PubMed  Google Scholar 

  36. T. DeVries and G. W. Taylor, Improved regularization of convolutional neural networks with cutout, arXiv preprint arXiv:1708.04552, 2017.

  37. H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, mixup: Beyond empirical risk minimization, arXiv preprint arXiv:1710.09412, 2017.

  38. Y. Tokozume, Y. Ushiku, and T. Harada, Between-class learning for image classification, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 5486–5494.

  39. S. Yun, D. Han, S. J. Oh, S. Chun, J. Choe, and Y. Yoo, Cutmix: Regularization strategy to train strong classifiers with localizable features, in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 6023–6032.

  40. N. C. F. Codella et al., Skin lesion analysis toward melanoma detection: A challenge at the 2017 International symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC), Proceedings - International Symposium on Biomedical Imaging, vol. 2018-April, pp. 168–172, 2018. https://doi.org/10.1109/ISBI.2018.8363547.

  41. V. Rotemberg et al., A patient-centric dataset of images and metadata for identifying melanomas using clinical context, Sci Data, vol. 8, no. 1, p. 34, 2021.

    Article  PubMed  PubMed Central  Google Scholar 

  42. N. Lama et al., ChimeraNet: U-Net for Hair Detection in Dermoscopic Skin Lesion Images, J Digit Imaging, no. 0123456789, 2022. https://doi.org/10.1007/s10278-022-00740-6.

  43. S. Ioffe and C. Szegedy, Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift.

  44. A. Buslaev, V. I. Iglovikov, E. Khvedchenya, A. Parinov, M. Druzhinin, and A. A. Kalinin, Albumentations: fast and flexible image augmentations, Information, vol. 11, no. 2, p. 125, 2020.

    Article  Google Scholar 

  45. D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980, 2014.

  46. C. H. Sudre, W. Li, T. Vercauteren, S. Ourselin, and M. Jorge Cardoso, Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations, in Deep learning in medical image analysis and multimodal learning for clinical decision support, Springer, 2017, pp. 240–248.

  47. F. Navarro, M. Escudero-Viñolo, and J. Bescós, Accurate Segmentation and Registration of Skin Lesion Images to Evaluate Lesion Change, IEEE J Biomed Health Inform, vol. 23, no. 2, pp. 501–508, 2019, doi: 10.1109/ and Sustainable Development, GTSD 2020, pp. 366–371, 2020. https://doi.org/10.1109/GTSD50082.2020.9303084.

  48. P. Shan, Y. Wang, C. Fu, W. Song, and J. Chen, Automatic skin lesion segmentation based on FC-DPN, Comput Biol Med, vol. 123, no. April, p. 103762, 2020. https://doi.org/10.1016/j.compbiomed.2020.103762.

  49. R. Kaymak, C. Kaymak, and A. Ucar, Skin lesion segmentation using fully convolutional networks: A comparative experimental study, Expert Syst Appl, vol. 161, p. 113742, 2020, https://doi.org/10.1016/j.eswa.2020.113742.

    Article  Google Scholar 

  50. D. K. Nguyen, T. T. Tran, C. P. Nguyen, and V. T. Pham, Skin Lesion Segmentation based on Integrating EfficientNet and Residual block into U-Net Neural Network, Proceedings of 2020 5th International Conference on Green Technology Access, vol. 10, no. September, pp. 94007–94018, 2022. https://doi.org/10.1109/ACCESS.2022.3204280.

  51. K. Zafar et al., Skin lesion segmentation from dermoscopic images using convolutional neural network, Sensors (Switzerland), vol. 20, no. 6, pp. 1–14, 2020, https://doi.org/10.3390/s20061601.

    Article  Google Scholar 

  52. M. Goyal, A. Oakley, P. Bansal, D. Dancey, and M. H. Yap, Skin Lesion Segmentation in Dermoscopic Images with Ensemble Deep Learning Methods, IEEE Access, vol. 8, pp. 4171–4181, 2020, https://doi.org/10.1109/ACCESS.2019.2960504.

    Article  Google Scholar 

  53. X. Tong, J. Wei, B. Sun, S. Su, Z. Zuo, and P. Wu, Ascu-net: Attention gate, spatial and channel attention u-net for skin lesion segmentation, Diagnostics, vol. 11, no. 3, 2021. https://doi.org/10.3390/diagnostics11030501.

  54. P. Chen, S. Huang, and Q. Yue, Skin Lesion Segmentation Using Recurrent Attentional Convolutional Networks, IEEE JBHI.2018.2825251.

  55. H. Ashraf, A. Waris, M. F. Ghafoor, S. O. Gilani, and I. K. Niazi, Melanoma segmentation using deep learning with test-time augmentations and conditional random fields, Sci Rep, vol. 12, no. 1, pp. 1–16, 2022, https://doi.org/10.1038/s41598-022-07885-y.

    Article  CAS  Google Scholar 

  56. A. Scope, et al. The ugly duckling sign: agreement between observers. Archives of Dermatology, vol. 144, no. 1, pp. 58–64. https://doi.org/10.1001/archdermatol.2007.15

  57. M. Zafar, M. I. Sharif, M. I. Sharif, S. Kadry, S. A. C. Bukhari, and H. T. Rauf, Skin Lesion Analysis and Cancer Detection Based on Machine/Deep Learning Techniques: A Comprehensive Survey. Life (Basel), 2023 Jan 4;13(1):146. https://doi.org/10.3390/life13010146.

    Article  ADS  PubMed  Google Scholar 

  58. K. M. Hosny, D. Elshoura, E. R. Mohamed, E. Vrochidou and G. A. Papakostas, Deep Learning and Optimization-Based Methods for Skin Lesions Segmentation: A Review, in IEEE Access, vol. 11, pp. 85467-85488, 2023. https://doi.org/10.1109/ACCESS.2023.3303961.

    Article  Google Scholar 

  59. G. Subhashini & A. Chandrasekar (2023) Hybrid deep learning technique for optimal segmentation and classification of multi-class skin cancer, The Imaging Science Journal, https://doi.org/10.1080/13682199.2023.2241794

    Article  Google Scholar 

  60. BAGHERI, FATEMEH; TAROKH, MOHAMMAD JAFAR; and ZIARATBAN, MAJID (2022) Skin lesion segmentation by using object detection networks, DeepLab3+, and active contours, Turkish Journal of Electrical Engineering and Computer Sciences: Vol. 30: No. 7, Article 2. https://doi.org/10.55730/1300-0632.3951

  61. Öztürk, Ş., Özkaya, U. Skin Lesion Segmentation with Improved Convolutional Neural Network. J Digit Imaging 33, 958–970 (2020). https://doi.org/10.1007/s10278-020-00343-z

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

National Institutes of Health, R43 CA153927- 01, William Van Stoecker, R44 CA101639-02A2, William Van Stoecker.

Author information

Authors and Affiliations

  1. Missouri University of Science & Technology, Rolla, MO, 65409, USA

    Norsang Lama, Ronald Joe Stanley, Akanksha Maurya, Anand Nambisan & Thanh Phan

  2. S&A Technologies, Rolla, MO, 65401, USA

    Binita Lama, Jason Hagerty & William Van Stoecker

Authors
  1. Norsang Lama
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ronald Joe Stanley
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Binita Lama
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Akanksha Maurya
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Anand Nambisan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jason Hagerty
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Thanh Phan
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. William Van Stoecker
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

We confirm that all authors contributed to the study conception and design. The experiments were conducted by Norsang Lama. Data collection, analysis, and validation were performed by all authors. The first draft of the manuscript was written by Norsang Lama, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Ronald Joe Stanley.

Ethics declarations

Ethics Approval

NA.

Consent to Participate

NA.

Consent for Publication

NA.

Conflict of Interest

The authors declare no competing interests.

Disclaimer

Its contents are solely the authors’ responsibility and do not necessarily represent the official views of the NIH or NSF.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Article Highlights

• Current skin lesion segmentation algorithms lack accuracy for images with multiple lesions.

• LAMA is an algorithm for generating synthetic skin lesion patches for training image segmentation.

• LAMA yields multiple-lesion training images with minimal lesion overlap.

• Results show improved segmentation accuracy for both single- and multiple-lesion images.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lama, N., Stanley, R.J., Lama, B. et al. LAMA: Lesion-Aware Mixup Augmentation for Skin Lesion Segmentation. J Digit Imaging. Inform. med. (2024). https://doi.org/10.1007/s10278-024-01000-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10278-024-01000-5

Keywords

Search

Navigation