Abstract
Longitudinal tracking of skin lesions - finding correspondence, changes in morphology, and texture - is beneficial to the early detection of melanoma. However, it has not been well investigated in the context of full-body imaging. We propose a novel framework combining geometric and texture information to localize skin lesion correspondence from a source scan to a target scan in total body photography (TBP). Body landmarks or sparse correspondence are first created on the source and target 3D textured meshes. Every vertex on each of the meshes is then mapped to a feature vector characterizing the geodesic distances to the landmarks on that mesh. Then, for each lesion of interest (LOI) on the source, its corresponding location on the target is first coarsely estimated using the geometric information encoded in the feature vectors and then refined using the texture information. We evaluated the framework quantitatively on both a public and a private dataset, for which our success rates (at 10 mm criterion) are comparable to the only reported longitudinal study. As full-body 3D capture becomes more prevalent and has higher quality, we expect the proposed method to constitute a valuable step in the longitudinal tracking of skin lesions.
M. Kazhdan and M. Armand—Co-senior authors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abbasi, N.R., et al.: Early diagnosis of cutaneous melanoma: revisiting the ABCD criteria. JAMA 292(22), 2771–2776 (2004)
Bogo, F., Romero, J., Peserico, E., Black, M.J.: Automated detection of new or evolving melanocytic lesions using a 3D body model. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8673, pp. 593–600. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10404-1_74
Datar, M., Lyu, I., Kim, S.H., Cates, J., Styner, M.A., Whitaker, R.: Geodesic distances to landmarks for dense correspondence on ensembles of complex shapes. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8150, pp. 19–26. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40763-5_3
Guo, Y., Sohel, F., Bennamoun, M., Lu, M., Wan, J.: Rotational projection statistics for 3D local surface description and object recognition. Int. J. Comput. Vision 105(1), 63–86 (2013)
Halpern, A.C.: Total body skin imaging as an aid to melanoma detection. In: Seminars in Cutaneous Medicine and Surgery, vol. 22, pp. 2–8 (2003)
Kim, H., Kim, J., Kam, J., Park, J., Lee, S.: Deep virtual markers for articulated 3D shapes. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11615–11625 (2021)
Korotkov, K., et al.: An improved skin lesion matching scheme in total body photography. IEEE J. Biomed. Health Inform. 23(2), 586–598 (2018)
Korotkov, K., Quintana, J., Puig, S., Malvehy, J., Garcia, R.: A new total body scanning system for automatic change detection in multiple pigmented skin lesions. IEEE Trans. Med. Imaging 34(1), 317–338 (2014)
Li, Y., Esteva, A., Kuprel, B., Novoa, R., Ko, J., Thrun, S.: Skin cancer detection and tracking using data synthesis and deep learning. arXiv preprint arXiv:1612.01074 (2016)
Mcgregor, B.: Automatic registration of images of pigmented skin lesions. Pattern Recogn. 31(6), 805–817 (1998)
Mirzaalian, H., Hamarneh, G., Lee, T.K.: A graph-based approach to skin mole matching incorporating template-normalized coordinates. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2152–2159. IEEE (2009)
Mirzaalian, H., Lee, T.K., Hamarneh, G.: Uncertainty-based feature learning for skin lesion matching using a high order MRF optimization framework. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012. LNCS, vol. 7511, pp. 98–105. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33418-4_13
Mirzaalian, H., Lee, T.K., Hamarneh, G.: Spatial normalization of human back images for dermatological studies. IEEE J. Biomed. Health Inform. 18(4), 1494–1501 (2013)
Mirzaalian, H., Lee, T.K., Hamarneh, G.: Skin lesion tracking using structured graphical models. Med. Image Anal. 27, 84–92 (2016)
Mitchel, T.W., Rusinkiewicz, S., Chirikjian, G.S., Kazhdan, M.: Echo: extended convolution histogram of orientations for local surface description. In: Computer Graphics Forum, vol. 40, pp. 180–194. Wiley Online Library (2021)
Perednia, D.A., White, R.G.: Automatic registration of multiple skin lesions by use of point pattern matching. Comput. Med. Imaging Graph. 16(3), 205–216 (1992)
Roning, J., Riech, M.: Registration of nevi in successive skin images for early detection of melanoma. In: Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No. 98EX170), vol. 1, pp. 352–357. IEEE (1998)
Saint, A., Ahmed, E., Cherenkova, K., Gusev, G., Aouada, D., Ottersten, B., et al.: 3DBodyTex: textured 3D body dataset. In: 2018 International Conference on 3D Vision (3DV), pp. 495–504. IEEE (2018)
Saint, A., Cherenkova, K., Gusev, G., Aouada, D., Ottersten, B., et al.: Bodyfitr: robust automatic 3D human body fitting. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 484–488. IEEE (2019)
Salti, S., Tombari, F., Di Stefano, L.: Shot: unique signatures of histograms for surface and texture description. Comput. Vis. Image Underst. 125, 251–264 (2014)
Strakowska, M., Kociołek, M.: Skin lesion matching algorithm for application in full body imaging systems. In: Pietka, E., Badura, P., Kawa, J., Wieclawek, W. (eds.) ITIB 2022. Advances in Intelligent Systems and Computing, vol. 1429, pp. 222–233. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-09135-3_19
Strzelecki, M.H., Strąkowska, M., Kozłowski, M., Urbańczyk, T., Wielowieyska-Szybińska, D., Kociołek, M.: Skin lesion detection algorithms in whole body images. Sensors 21(19), 6639 (2021)
Tombari, F., Salti, S., Di Stefano, L.: Unique signatures of histograms for local surface description. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6313, pp. 356–369. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15558-1_26
Tombari, F., Salti, S., Di Stefano, L.: A combined texture-shape descriptor for enhanced 3D feature matching. In: 2011 18th IEEE International Conference on Image Processing, pp. 809–812. IEEE (2011)
White, R.G., Perednia, D.A.: Automatic derivation of initial match points for paired digital images of skin. Comput. Med. Imaging Graph. 16(3), 217–225 (1992)
Zhao, M., Kawahara, J., Abhishek, K., Shamanian, S., Hamarneh, G.: Skin3D: detection and longitudinal tracking of pigmented skin lesions in 3D total-body textured meshes. Med. Image Anal. 77, 102329 (2022)
Acknowledgments
The research was in part supported by the Intramural Research Program (IRP) of the NIH/NICHD, Phase I of NSF STTR grant 2127051, and Phase I NIH/NIBIB STTR grant R41EB032304.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Huang, WL., Tashayyod, D., Kang, J., Gandjbakhche, A., Kazhdan, M., Armand, M. (2023). Skin Lesion Correspondence Localization in Total Body Photography. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14226. Springer, Cham. https://doi.org/10.1007/978-3-031-43990-2_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-43990-2_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43989-6
Online ISBN: 978-3-031-43990-2
eBook Packages: Computer ScienceComputer Science (R0)
