Advertisement

HDR-Like Image Generation to Mitigate Adverse Wound Illumination Using Deep Bi-directional Retinex and Exposure Fusion

Conference paper
  • 146 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12722)

Abstract

Periodic assessment is necessary to evaluate the healing prog-ress of chronic wounds. Image analyses using computer vision algorithms have recently emerged as a viable alternative that has been demonstrated by prior work. However, the performance of such image analysis methods degrade on captured in adverse illumination, which is common in many indoor environments. To mitigate these lighting problems, High Dynamic Range (HDR) image enhancement techniques can be used to mitigate over- and under-exposure issues and preserve the details of scenes captured in non-ideal illumination. In this paper, we address over- and under-exposure simultaneously using a deep learning-based bi-directional illumination enhancement network that is able to generate over- and under-exposed images that are then fused into a final image with enhanced illumination. In rigorous evaluations using metrics including structure similarity, peak signal-noise ratio and changes in segmentation accuracy, our proposed method outperformed the state-of-the-art (SSIM scores \(0.76\pm 0.04\)/\(0.69\pm 0.08\) on bright/dark images, PSNR scores \(28.60\pm 0.70\) on dark images and DSC scores \(0.76\pm 0.09\)/\(0.74\pm 0.09\) on bright/dark images).

Keywords

Image enhancement Wound assessment Deep learning Image segmentation 

References

  1. 1.
    Martinengo, L., et al.: Prevalence of chronic wounds in the general population: systematic review and meta-analysis of observational studies. Ann. Epidemiol. 29, 8–15 (2019)CrossRefGoogle Scholar
  2. 2.
    Richmond, N.A., Maderal, A.D., Vivas, A.C.: Evidence-based management of common chronic lower extremity ulcers. Dermatol. Ther. 26(3), 187–196 (2013)CrossRefGoogle Scholar
  3. 3.
    Wang, L., Pedersen, P.C., Strong, D.M., Tulu, B., Agu, E., Ignotz, R.: Smartphone-based wound assessment system for patients with diabetes. IEEE Trans. Biomed. Eng. 62(2), 477–488 (2014)CrossRefGoogle Scholar
  4. 4.
    Kuang, B., et al.: Assessment of a smartphone-based application for diabetic foot ulcer measurement. Wound Repair and Regeneration (2021)Google Scholar
  5. 5.
    Wang, L.: System designs for diabetic foot ulcer image assessment. System 2016, 03–07 (2016)Google Scholar
  6. 6.
    Myszkowski, K., Mantiuk, R., Krawczyk, G.: High dynamic range video. Syn. Lect. Comput. Graph. Anim. 1(1), 1–158 (2008)Google Scholar
  7. 7.
    Reinhard, E., Heidrich, W., Debevec, P., Pattanaik, S., Ward, G., Myszkowski, K.: High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting. Morgan Kaufmann, Burlington (2010)Google Scholar
  8. 8.
    Debevec, P.E., Malik, J.: Recovering high dynamic range radiance maps from photographs. In: ACM SIGGRAPH 2008 Classes, pp. 1–10 (2008)Google Scholar
  9. 9.
    Licciardo, G.D., Cappetta, C., Di Benedetto, L.: Dynamic range enhancement for medical image processing. In: 2017 7th IEEE International Workshop on Advances in Sensors and Interfaces (IWASI), pp. 219–223. IEEE (2017)Google Scholar
  10. 10.
    Eilertsen, G., Kronander, J., Denes, G., Mantiuk, R.K., Unger, J.: HDR image reconstruction from a single exposure using deep CNNs. ACM Trans. Graph. (TOG) 36(6), 1–15 (2017)CrossRefGoogle Scholar
  11. 11.
    Marnerides, D., Bashford-Rogers, T., Hatchett, J., Debattista, K.: Expandnet: a deep convolutional neural network for high dynamic range expansion from low dynamic range content. In: Computer Graphics Forum, vol. 37, no. 2, pp. 37–49. Wiley Online Library (2018)Google Scholar
  12. 12.
    Y.-L. Liu, et al.: Single-image HDR reconstruction by learning to reverse the camera pipeline. In: Proceedings IEEE CVPR, pp. 1651–1660 (2020)Google Scholar
  13. 13.
    Endo, Y., Kanamori, Y., Mitani, J.: Deep reverse tone mapping. ACM Trans. Graph. 36(6), 177–181 (2017)CrossRefGoogle Scholar
  14. 14.
    Bachoo, A.K.: Real-time exposure fusion on a mobile computer (2009)Google Scholar
  15. 15.
    Kim, J.H., Lee, S., Jo, S., Kang, S.-J.: End-to-end differentiable learning to HDR image synth. for multi-exposure images. arXiv preprint arXiv:2006.15833 (2020)
  16. 16.
    Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the CVPR, pp. 1125–1134 (2017)Google Scholar
  17. 17.
    Mertens, T., Kautz, J., Van Reeth, F.: Exposure fusion: a simple and practical alternative to high dynamic range photography. Comput. Graph. Forum 28(1), 161–171 (2009). Wiley Online LibraryGoogle Scholar
  18. 18.
    Song, M., Tao, D., Chen, C., Bu, J., Luo, J., Zhang, C.: Probabilistic exposure fusion. IEEE Trans. Image Process. 21(1), 341–357 (2011)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Li, H., et al.: Denoising scanner effects from multimodal MRI data using linked independent component analysis. Neuroimage 208, 116388 (2020)CrossRefGoogle Scholar
  20. 20.
    Vonikakis, V., Bouzos, O., Andreadis, I.: Multi-exposure image fusion based on illumination estimation. In: Proceedings of IASTED SIPA, pp. 135–142 (2011)Google Scholar
  21. 21.
    Banterle, F., Ledda, P., Debattista, K., Chalmers, A.: Inverse tone mapping. In: Proceedings of the 4th International Conference on Computer Graphics and Interactive Techniques in Australasia and Southeast Asia, pp. 349–356 (2006)Google Scholar
  22. 22.
    Grossberg, M.D., Nayar, S.K.: What is the space of camera response functions? In: Proceedings of IEEE CVPR, vol. 2, pp. II-602. IEEE (2003)Google Scholar
  23. 23.
    Zhang, J., Lalonde, J.-F.: Learning high dynamic range from outdoor panoramas. In: Proceedings IEEE International Conference on Computer Vision, pp. 4519–4528 (2017)Google Scholar
  24. 24.
    Kalantari, N.K., Ramamoorthi, R.: Deep high dynamic range imaging of dynamic scenes. ACM Trans. Graph. 36(4), 144–151 (2017)CrossRefGoogle Scholar
  25. 25.
    Banterle, F., Ledda, P., Debattista, K., Bloj, M., Artusi, A., Chalmers, A.: A psychophysical evaluation of inverse tone mapping techniques. In: Computer Graphics Forum, vol. 28, no. 1, pp. 13–25 (2009). Wiley Online LibraryGoogle Scholar
  26. 26.
    Kovaleski, R.P., Oliveira, M.M.: High-quality reverse tone mapping for a wide range of exposures. In: Proceedings of SIBGRAPI, pp. 49–56. IEEE (2014)Google Scholar
  27. 27.
    Wei, C., Wang, W., Yang, W., Liu, J.: Deep retinex decomposition for low-light enhancement. arXiv preprint arXiv:1808.04560 (2018)
  28. 28.
    Khunteta, A., Ghosh, D., et al.: Fuzzy rule-based image exposure level estimation and adaptive gamma correction for contrast enhancement in dark images. In: IEEE International Conference on Signal Processing, vol. 1, pp. 667–672. IEEE (2012)Google Scholar
  29. 29.
    Iyer, A.B.: Let there be light...characterizing the effects of adverse lighting on semantic segmentation of wound images and mitigation using a deep retinex model. Masters thesis, Worcester Polytechnic Institute (2020)Google Scholar
  30. 30.
    Vinnie venous insufficiency leg model. https://vatainc.com/product/vinnie-venous-insufficiency-leg-model. Accessed 10 May 2021
  31. 31.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  32. 32.
    Hore, A., Ziou, D.: Image quality metrics: PSNR vs. SSIM. In: 2010 20th International Conference on Pattern Recognition, pp. 2366–2369. IEEE (2010)Google Scholar
  33. 33.
    Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26(3), 297–302 (1945)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2021

Authors and Affiliations

  1. 1.Department of Computer ScienceWorcester Polytechnic InstituteWorcesterUSA
  2. 2.Department of RadiologyUniversity of Massachusetts Medical SchoolWorcesterUSA

Personalised recommendations