Skip to main content
SpringerLink
Log in

Synthetic Data as a Tool to Combat Racial Bias in Medical AI: Utilizing Generative Models for Optimizing Early Detection of Melanoma in Fitzpatrick Skin Types IV–VI

  • Conference paper
  • First Online:
Medical Imaging and Computer-Aided Diagnosis (MICAD 2022)

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 810))

  • 85 Accesses

Abstract

Assistive tools to aid in skin cancer detection are experiencing an unprecedented rise with the accessibility of robust and accurate deep learning models. However, in the present applications, only a negligible number of dermatology images come from patients with Fitzpatrick skin types IV–VI, representing brown, dark brown or black skin, respectively. In this study, we demonstrate the utilization of Zero-Shot Text-to-Image autoregressive models to generate synthetic medical data for improved balance in training CAD classification models with minimized racial bias. Synthetically generated images of skin lesions were assessed by an experienced dermatologist using the ABCD rule and differential diagnostics, and subsequently validated using a pre-trained ResNet50V2 multi-class classification model.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.isic-archive.com/.

  2. 2.

    https://github.com/openai/dalle-2-preview/blob/main/system-card.md.

References

  1. Lerner, B., Stewart, L., Horowitz, D. & Carvajal, R. Mucosal Melanoma: new insights and therapeutic options for a unique and aggressive disease. Oncology (08909091). 31 (2017)

    Google Scholar 

  2. D’Arcy, C., Holman, J. & Armstrong, B. Pigmentary traits, ethnic origin, benign nevi, and family history as risk factors for cutaneous malignant melanoma. Journal Of The National Cancer Institute. 72, 257–266 (1984)

    Google Scholar 

  3. McGovern, V., Mihm Jr, M., Bailly, C., Booth, J., Clark Jr, W., Cochran, A., Hardy, E., Hicks, J., Levene, A., Lewis, M. & Others The classification of malignant melanoma and its histologic reporting. Cancer. 32, 1446–1457 (1973)

    Google Scholar 

  4. Rastrelli, M., Tropea, S., Rossi, C. & Alaibac, M. Melanoma: epidemiology, risk factors, pathogenesis, diagnosis and classification. In Vivo. 28, 1005–1011 (2014)

    Google Scholar 

  5. Lopes, F., Sleiman, M., Sebastian, K., Bogucka, R., Jacobs, E. & Adamson, A. UV exposure and the risk of cutaneous melanoma in skin of color: a systematic review. JAMA Dermatology. 157, 213–219 (2021)

    Google Scholar 

  6. Gohara, M. Skin cancer in skins of color: Skin cancer. Journal Of Drugs In Dermatology. 7, 441–445 (2008)

    Google Scholar 

  7. Wu, X., Eide, M., King, J., Saraiya, M., Huang, Y., Wiggins, C., Barnholtz-Sloan, J., Martin, N., Cokkinides, V., Miller, J. & Others. Racial and ethnic variations in incidence and survival of cutaneous melanoma in the United States, 1999-2006. Journal Of The American Academy Of Dermatology. 65, S26-e1 (2011)

    Google Scholar 

  8. Gupta, A., Bharadwaj, M. & Mehrotra, R. Skin cancer concerns in people of color: risk factors and prevention. Asian Pacific Journal Of Cancer Prevention: APJCP. 17, 5257 (2016)

    Google Scholar 

  9. Rigel, D., Friedman, R. & Kopf, A. The incidence of malignant melanoma in the United States: issues as we approach the 21st century. Journal Of The American Academy Of Dermatology. 34, 839–847 (1996)

    Google Scholar 

  10. Coit, D., Andtbacka, R., Bichakjian, C., Dilawari, R., DiMaio, D., Guild, V., Halpern, A., Hodi, F., Kashani-Sabet, M., Lange, J. & Others Melanoma. Journal Of The National Comprehensive Cancer Network. 7, 250–275 (2009)

    Google Scholar 

  11. Hida, T., Kamiya, T., Kawakami, A., Ogino, J., Sohma, H., Uhara, H. & Jimbow, K. Elucidation of melanogenesis cascade for identifying pathophysiology and therapeutic approach of pigmentary disorders and melanoma. International Journal Of Molecular Sciences. 21, 6129 (2020)

    Google Scholar 

  12. Bastian, B. The molecular pathology of melanoma: an integrated taxonomy of melanocytic neoplasia. Annual Review Of Pathology: Mechanisms Of Disease. 9 pp. 239–271 (2014)

    Google Scholar 

  13. Apalla, Z., Nashan, D., Weller, R. & Castellsagué, X. Skin cancer: epidemiology, disease burden, pathophysiology, diagnosis, and therapeutic approaches. Dermatology And Therapy. 7, 5–19 (2017)

    Google Scholar 

  14. Shortliffe, E. & Sepúlveda, M. Clinical decision support in the era of artificial intelligence. Jama. 320, 2199–2200 (2018)

    Google Scholar 

  15. Jain, A., Way, D., Gupta, V., Gao, Y., Oliveira Marinho, G., Hartford, J., Sayres, R., Kanada, K., Eng, C., Nagpal, K. & Others Development and assessment of an artificial intelligence-based tool for skin condition diagnosis by primary care physicians and nurse practitioners in teledermatology practices. JAMA Network Open. 4, e217249–e217249 (2021)

    Google Scholar 

  16. Liu, Y., Jain, A., Eng, C., Way, D., Lee, K., Bui, P., Kanada, K., Oliveira Marinho, G., Gallegos, J., Gabriele, S. & Others A deep learning system for differential diagnosis of skin diseases. Nature Medicine. 26, 900–908 (2020)

    Google Scholar 

  17. Nahar, V., Ford, M., Jacks, S., Thielen, S., Johnson, A., Brodell, R. & Bass, M. Sun-related behaviors among individuals previously diagnosed with non-melanoma skin cancer. Indian Journal Of Dermatology, Venereology And Leprology. 81 pp. 568 (2015)

    Google Scholar 

  18. Sober, A. & Burstein, J. Precursors to skin cancer. Cancer. 75, 645–650 (1995)

    Google Scholar 

  19. Carli, P., Massi, D., Giorgi, V. & Giannotti, B. Clinically and dermoscopically featureless melanoma: when prevention fails. Journal Of The American Academy Of Dermatology. 46, 957–959 (2002)

    Google Scholar 

  20. Li, Y., Esteva, A., Kuprel, B., Novoa, R., Ko, J. & Thrun, S. Skin cancer detection and tracking using data synthesis and deep learning. ArXiv Preprint ArXiv:1612.01074. (2016)

  21. Liu, W., Hill, D., Gibbs, A., Tempany, M., Howe, C., Borland, R., Morand, M. & Kelly, J. What features do patients notice that help to distinguish between benign pigmented lesions and melanomas?: the ABCD (E) rule versus the seven-point checklist. Melanoma Research. 15, 549–554 (2005)

    Google Scholar 

  22. Esteva, A., Kuprel, B., Novoa, R., Ko, J., Swetter, S., Blau, H. & Thrun, S. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 542, 115–118 (2017)

    Google Scholar 

  23. Mar, V. & Soyer, H. Artificial intelligence for melanoma diagnosis: how can we deliver on the promise?. Annals Of Oncology. 29, 1625–1628 (2018)

    Google Scholar 

  24. Sun, M., Kentley, J., Mehta, P., Dusza, S., Halpern, A. & Rotemberg, V. Accuracy of commercially available smartphone applications for the detection of melanoma. British Journal Of Dermatology. 186, 744–746 (2022)

    Google Scholar 

  25. Combalia, M., Codella, N., Rotemberg, V., Carrera, C., Dusza, S., Gutman, D., Helba, B., Kittler, H., Kurtansky, N., Liopyris, K. & Others Validation of artificial intelligence prediction models for skin cancer diagnosis using dermoscopy images: the 2019 International Skin Imaging Collaboration Grand Challenge. The Lancet Digital Health. 4, e330–e339 (2022)

    Google Scholar 

  26. Haenssle, H., Fink, C., Schneiderbauer, R., Toberer, F., Buhl, T., Blum, A., Kalloo, A., Hassen, A., Thomas, L., Enk, A. & Others Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Annals Of Oncology. 29, 1836–1842 (2018)

    Google Scholar 

  27. Masood, A., Al-Jumaily, A. & Anam, K. Self-supervised learning model for skin cancer diagnosis. 2015 7th International IEEE/EMBS Conference On Neural Engineering (NER). pp. 1012–1015 (2015)

    Google Scholar 

  28. Castiglioni, I., Rundo, L., Codari, M., Di Leo, G., Salvatore, C., Interlenghi, M., Gallivanone, F., Cozzi, A., D’Amico, N. & Sardanelli, F. AI applications to medical images: From machine learning to deep learning. Physica Medica. 83 pp. 9–24 (2021)

    Google Scholar 

  29. Ghassemi, M., Oakden-Rayner, L. & Beam, A. The false hope of current approaches to explainable artificial intelligence in health care. The Lancet Digital Health. 3, e745–e750 (2021)

    Google Scholar 

  30. FDA Proposed regulatory framework for modifications to artificial intelligence/machine learning (AI/ML)-based software as a medical device (SaMD). (Department of Health, 2019)

    Google Scholar 

  31. Tjoa, E. & Guan, C. A survey on explainable artificial intelligence (xai): Toward medical xai. IEEE Transactions On Neural Networks And Learning Systems. 32, 4793–4813 (2020)

    Google Scholar 

  32. Handa, A., Patraucean, V., Badrinarayanan, V., Stent, S. & Cipolla, R. Understanding real world indoor scenes with synthetic data. Proceedings Of The IEEE Conference On Computer Vision And Pattern Recognition. pp. 4077–4085 (2016)

    Google Scholar 

  33. Jaderberg, M., Simonyan, K., Vedaldi, A. & Zisserman, A. Synthetic data and artificial neural networks for natural scene text recognition. ArXiv Preprint ArXiv:1406.2227. (2014)

  34. Benaim, A., Almog, R., Gorelik, Y., Hochberg, I., Nassar, L., Mashiach, T., Khamaisi, M., Lurie, Y., Azzam, Z., Khoury, J. & Others Analyzing medical research results based on synthetic data and their relation to real data results: systematic comparison from five observational studies. JMIR Medical Informatics. 8, e16492 (2020)

    Google Scholar 

  35. El Emam, K., Mosquera, L., Fang, X., El-Hussuna, A. & Others Utility Metrics for Evaluating Synthetic Health Data Generation Methods: Validation Study. JMIR Medical Informatics. 10, e35734 (2022)

    Google Scholar 

  36. Shin, H., Tenenholtz, N., Rogers, J., Schwarz, C., Senjem, M., Gunter, J., Andriole, K. & Michalski, M. Medical image synthesis for data augmentation and anonymization using generative adversarial networks. International Workshop On Simulation And Synthesis In Medical Imaging. pp. 1–11 (2018)

    Google Scholar 

  37. Peng, Y., Chen, S., Qin, A., Chen, M., Gao, X., Liu, Y., Miao, J., Gu, H., Zhao, C., Deng, X. & Others Magnetic resonance-based synthetic computed tomography images generated using generative adversarial networks for nasopharyngeal carcinoma radiotherapy treatment planning. Radiotherapy And Oncology. 150 pp. 217–224 (2020)

    Google Scholar 

  38. Onishi, Y., Teramoto, A., Tsujimoto, M., Tsukamoto, T., Saito, K., Toyama, H., Imaizumi, K. & Fujita, H. Automated pulmonary nodule classification in computed tomography images using a deep convolutional neural network trained by generative adversarial networks. BioMed Research International. 2019 (2019)

    Google Scholar 

  39. Han, X., Zhang, L., Zhou, K. & Wang, X. ProGAN: Protein solubility generative adversarial nets for data augmentation in DNN framework. Computers & Chemical Engineering. 131 pp. 106533 (2019)

    Google Scholar 

  40. Narayanan, H., Dingfelder, F., Butté, A., Lorenzen, N., Sokolov, M. & Arosio, P. Machine learning for biologics: opportunities for protein engineering, developability, and formulation. Trends In Pharmacological Sciences. 42, 151–165 (2021)

    Google Scholar 

  41. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. & Bengio, Y. Generative adversarial networks. arXiv e-prints. ArXiv Preprint ArXiv:1406.2661. 1406 (2014)

  42. Limeros, S., Majchrowska, S., Zoubi, M., Rosén, A., Suvilehto, J., Sjöblom, L. & Kjellberg, M. GAN-based generative modelling for dermatological applications-comparative study. ArXiv Preprint ArXiv:2208.11702. (2022)

  43. Chen, S., Shi, D., Sadiq, M. & Cheng, X. Image denoising with generative adversarial networks and its application to cell image enhancement. IEEE Access. 8 pp. 82819–82831 (2020)

    Google Scholar 

  44. Khalifa, N., Taha, M., Hassanien, A. & Elghamrawy, S. Detection of coronavirus (covid-19) associated pneumonia based on generative adversarial networks and a fine-tuned deep transfer learning model using chest x-ray dataset. ArXiv Preprint ArXiv:2004.01184. (2020)

  45. Jin, Q., Cui, H., Sun, C., Meng, Z. & Su, R. Free-form tumor synthesis in computed tomography images via richer generative adversarial network. Knowledge-Based Systems. 218 pp. 106753 (2021)

    Google Scholar 

  46. Wang, K., Gou, C., Duan, Y., Lin, Y., Zheng, X. & Wang, F. Generative adversarial networks: introduction and outlook. IEEE/CAA Journal Of Automatica Sinica. 4, 588–598 (2017)

    Google Scholar 

  47. Kim, T., Cha, M., Kim, H., Lee, J. & Kim, J. Learning to discover cross-domain relations with generative adversarial networks. International Conference On Machine Learning. pp. 1857–1865 (2017)

    Google Scholar 

  48. Choi, Y., Choi, M., Kim, M., Ha, J., Kim, S. & Choo, J. Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. Proceedings Of The IEEE Conference On Computer Vision And Pattern Recognition. pp. 8789–8797 (2018)

    Google Scholar 

  49. Li, X., Luo, M., Ji, S., Zhang, L. & Lu, M. Evaluating generative adversarial networks based image-level domain transfer for multi-source remote sensing image segmentation and object detection. International Journal Of Remote Sensing. 41, 7343–7367 (2020)

    Google Scholar 

  50. Larochelle, H., Erhan, D. & Bengio, Y. Zero-Data Learning of New Tasks. Proceedings Of The 23rd National Conference On Artificial Intelligence - Volume 2. pp. 646–651 (2008)

    Google Scholar 

  51. Mensink, T., Gavves, E. & Snoek, C. Costa: Co-occurrence statistics for zero-shot classification. Proceedings Of The IEEE Conference On Computer Vision And Pattern Recognition. pp. 2441–2448 (2014)

    Google Scholar 

  52. Radford, A., Kim, J., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G. & Sutskever, I. Learning Transferable Visual Models From Natural Language Supervision. (arXiv, 2021)

    Google Scholar 

  53. Ramesh, A., Dhariwal, P., Nichol, A., Chu, C. & Chen, M. Hierarchical Text-Conditional Image Generation with CLIP Latents. (arXiv,2022)

    Google Scholar 

  54. Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M. & Sutskever, I. Zero-shot text-to-image generation. International Conference On Machine Learning. pp. 8821–8831 (2021)

    Google Scholar 

  55. Kapelyukh, I., Vosylius, V. & Johns, E. DALL-E-Bot: Introducing Web-Scale Diffusion Models to Robotics. ArXiv Preprint ArXiv:2210.02438. (2022)

  56. Croitoru, F., Hondru, V., Ionescu, R. & Shah, M. Diffusion models in vision: A survey. ArXiv Preprint ArXiv:2209.04747. (2022)

  57. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I. & Amodei, D. Language Models are Few-Shot Learners. Advances In Neural Information Processing Systems. 33 pp. 1877–1901 (2020)

    Google Scholar 

  58. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A., Kaiser, Ł. & Polosukhin, I. Attention is all you need. Advances In Neural Information Processing Systems. 30 (2017)

    Google Scholar 

  59. Nachbar, F., Stolz, W., Merkle, T., Cognetta, A., Vogt, T., Landthaler, M., Bilek, P., Braun-Falco, O. & Plewig, G. The ABCD rule of dermatoscopy: high prospective value in the diagnosis of doubtful melanocytic skin lesions. Journal Of The American Academy Of Dermatology. 30, 551–559 (1994)

    Google Scholar 

  60. Jensen, J. & Elewski, B. The ABCDEF rule: combining the “ABCDE rule” and the “ugly duckling sign” in an effort to improve patient self-screening examinations. The Journal Of Clinical And Aesthetic Dermatology. 8, 15 (2015)

    Google Scholar 

  61. Wolner, Z., Yélamos, O., Liopyris, K., Rogers, T., Marchetti, M. & Marghoob, A. Enhancing skin cancer diagnosis with dermoscopy. Dermatologic Clinics. 35, 417–437 (2017)

    Google Scholar 

  62. He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. Proceedings Of The IEEE Conference On Computer Vision And Pattern Recognition. pp. 770–778 (2016)

    Google Scholar 

  63. Yamazaki, M., Kasagi, A., Tabuchi, A., Honda, T., Miwa, M., Fukumoto, N., Tabaru, T., Ike, A. & Nakashima, K. Yet another accelerated sgd: Resnet-50 training on imagenet in 74.7 seconds. ArXiv Preprint ArXiv:1903.12650. (2019)

  64. Tschandl, P., Rosendahl, C. & Kittler, H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data. 5, 1–9 (2018)

    Google Scholar 

Download references

Conflict of interest

In relation to this study, we declare the following conflicts of interest: the research was funded by Carebot s.r.o.

Author information

Authors and Affiliations

  1. Carebot s.r.o., Prague, Czech Republic

    Daniel Kvak, Marek Biroš & Robert Hrubý

  2. Faculty of Medicine, Masaryk University, Brno, Czech Republic

    Eva Březinová

  3. First Department of Dermatovenerology, St. Anne’s University Hospital, Brno, Czech Republic

    Eva Březinová

  4. Faculty of Mathematics and Physics, Charles University, Prague, Czech Republic

    Marek Biroš

  5. Faculty of Nuclear Sciences and Physical Engineering, Czech Technical University, Prague, Czech Republic

    Robert Hrubý

Authors
  1. Daniel Kvak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eva Březinová
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marek Biroš
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Robert Hrubý
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Kvak .

Editor information

Editors and Affiliations

  1. Department of Computer Sciences and Engineering, Shanghai Jiao Tong University, Shanghai, China

    Ruidan Su

  2. Department of Informatics, University of Leicester, Leicester, UK

    Yudong Zhang

  3. College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, Guangdong, China

    Han Liu

  4. Computational Medicine, University of Manchester, Manchester, UK

    Alejandro F Frangi

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kvak, D., Březinová, E., Biroš, M., Hrubý, R. (2023). Synthetic Data as a Tool to Combat Racial Bias in Medical AI: Utilizing Generative Models for Optimizing Early Detection of Melanoma in Fitzpatrick Skin Types IV–VI. In: Su, R., Zhang, Y., Liu, H., F Frangi, A. (eds) Medical Imaging and Computer-Aided Diagnosis. MICAD 2022. Lecture Notes in Electrical Engineering, vol 810. Springer, Singapore. https://doi.org/10.1007/978-981-16-6775-6_26

Download citation

  • DOI: https://doi.org/10.1007/978-981-16-6775-6_26

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-16-6774-9

  • Online ISBN: 978-981-16-6775-6

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation