Skip to main content
SpringerLink
Log in

Color Rendering in Medical Extended-Reality Applications

  • Original Paper
  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Cross-platform development of medical applications in extended-reality (XR) head-mounted displays (HMDs) often relies on game engines with rendering capabilities currently not standardized in the context of medical visualizations. Many aspects of the visualization pipeline including the characterization of color have yet to be consistently defined across rendering models and platforms. We examined the transfer of color properties from digital objects, through the rendering and image processing steps, to the RGB values sent to the display device. Five rendering pipeline configurations within the Unity engine were evaluated using 24 digital color patches. In the second experiment, the same configurations were evaluated with a tissue slide sample image. Measurements of the change in color associated with each configuration were characterized using the CIE 1976 color difference (\({\Delta}{\text{E}}\)). We found that the distribution of \({\Delta}{\text{E}}\) for the first experiment ranges from zero, as in the case using an Unlit Shader, to 25.97, as in the case using default configurations. The default Unity configuration consistently returned the highest \({\Delta}{\text{E}}\) across all 24 colors and also the largest range of color differences. In the second experiment, \({\Delta}{\text{E}}\)E ranged from 7.49 to 34.18. The Unlit configuration resulted in the highest \({\Delta}{\text{E}}\) in three of four selected pixels in the tissue sample image. Changes in color image properties associated with texture import settings were then evaluated in a third experiment using the TG18-QC test pattern. Differences in pixel values were found in all nine of the investigated texture import settings. The findings provide an initial characterization of color transfer and a basis for future work on standardization, consistency, and optimization of color in medical XR applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Valcasara N, Unreal Engine Game Development Blueprints. Packt Publishing, 2015.

  2. Kim SL, Suk HJ, Kang JH, Jung JM, Laine TH, Westlin J, Using Unity 3D to facilitate mobile augmented reality game development, 2014 IEEE World Forum on Internet of Things (WF-IoT), 2014. https://doi.org/10.1109/WF-IoT.2014.6803110

  3. Burley B, Physically-based shading at Disney. In ACM SIGGRAPH 2012 Courses, ACM, SIGGRAPH 2012.

  4. Cook RL, Torrance KE, A reflectance model for computer graphics. ACM Trans. Graph. 1, 5, pp.7-24, 1982.

    Article  Google Scholar 

  5. Walter B, Marschner SR, Li HS, Torrance KE, Microfacet models for refraction through rough surfaces, The Eurographics Association, Proceedings of the 18th Eurographics Conference on Rendering Techniques, pp.195-206, 2007.

  6. Blinn JF, Models of light reflection for computer synthesized pictures. Proceedings of the 4th Annual Conference on Computer Graphics and Interactive Techniques, Association for Computing Machinery, New York, NY, pp.192-198, 1977. https://doi.org/10.1145/563858.563893

  7. Schlick C, An inexpensive BRDF model for physically-based rendering, Computer Graphics Forum, 13(3), pp.223-246, 1994.

    Article  Google Scholar 

  8. Cheng WC, Saleheen F, Badano A, Assessing color performance of whole-slide imaging scanners for digital pathology, Color Research & Application, 44(3), pp.322-334, 2019. https://doi.org/10.1002/col.22365

    Article  Google Scholar 

  9. Samei E, Badano A, Chakraborty D, Compton K, Cornelius C, Corrigan K, Flynn MJ, Hemminger B, Hangiandreou N, Johnson J, and others, Assessment of display performance for medical imaging systems: executive summary of AAPM TG18 report, Medical Physics, 32(4), pp.1205-1225, 2005.

  10. Unity. https://unity3d.com/. Accessed 15 Jan 2020

  11. Wheeler G, Deng S, Toussaint N, Pushparajah K, Schnabel JA, Simpson JM, Gomez A, Virtual interaction and visualisation of 3D medical imaging data with VTK and Unity, Healthcare technology letters, 5(5), pp.148–153, 2018. https://doi.org/10.1049/htl.2018.5064

    Article  PubMed  PubMed Central  Google Scholar 

  12. Seif M, Umeda R, Uehara Y, Higa H, A Data Conversion for Medical Training System, IEEJ 2016 Conference, 2016.

  13. Wang R, Yao J, Wang L, Liu X, Wang H, Zheng L, A surgical training system for four medical punctures based on virtual reality and haptic feedback, 2017 IEEE Symposium on 3D User Interfaces (3DUI), pp.215-216, 2017. https://doi.org/10.1109/3DUI.2017.7893348

  14. Peters TM, Linte CA, Yaniv Z, Williams J, Mixed and augmented reality in medicine, CRC, 2018.

  15. Escobar-Castillejos D, Noguez J, Neri L, Magana A, Benes B, A Review of Simulators with Haptic Devices for Medical Training, Journal of Medical Systems, 40(104), 2016. https://doi.org/10.1007/s10916-016-0459-8

  16. Unity Technologies, Unity Manual 2019.2: Standard Shader. https://docs.unity3d.com/Manual/shader-StandardShader.html. Accessed 15 Jan 2020.

  17. Lagarde S, The High Definition Render Pipeline: Focused on visual quality. https://blogs.unity3d.com/2018/03/16/the-high-definition-render-pipeline-focused-on-visual-quality/. Accessed 15 Jan 2020.

  18. Unity Technologies, Unity Manual 2019.2: Global Illuminance. https://docs.unity3d.com/Manual/GIIntro.html. Accessed 15 Jan 2020.

  19. Khor WS, Baker B, Amin K, Chan A, Patel K, Woong J, Augmented and virtual reality in surgery the digital surgical environment: applications, limitations and legal pitfalls, Annals of translational medicine, 4(23), 2016.

  20. Chen L, Day TW, Tang W, John NW, Recent developments and future challenges in medical mixed reality, 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp.123-135, 2017.

  21. Vávra P, Roman J, Zonča P, Ihnát P, Němec M, Kumar J, Habib N, El-Gendi A, Recent development of augmented reality in surgery: a review, Journal of healthcare engineering, 2017.

  22. Unity Technologies, Unity Manual 2019.2: Textures. https://docs.unity3d.com/Manual/class-TextureImporter.html. Accessed 15 Jan 2020.

  23. International Electrotechnical Commission and others, Multimedia systems and equipment-Colour measurement and management-Part 2-1: Colour management-Default RGB colour space-sRGB, IEC 61966-2-1, 1999.

  24. Ohta N, Robertson A, Colorimetry: fundamentals and applications. John Wiley & Sons, 2016.

  25. Unity Technologies, Unity Scripting API 2019.2: Texture Importer Platform Settings. docs.unity3d.com/ScriptReference/TextureImporterPlatformSettings.html. Accessed 15 Jan 2020.

  26. Dille S, Fuhrmann A, Fischer G, Real-time tone mapping- An evaluation of color-accurate methods for luminance compression, Tagungsband des 22. Workshop Farbbildverarbeitung, Ilmenau, Germany, 2016.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. FDA, Silver Spring, Maryland, USA

    Andrea Seung Kim, Wei-Chung Cheng, Ryan Beams & Aldo Badano

Authors
  1. Andrea Seung Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei-Chung Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ryan Beams
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aldo Badano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aldo Badano.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, A.S., Cheng, WC., Beams, R. et al. Color Rendering in Medical Extended-Reality Applications. J Digit Imaging 34, 16–26 (2021). https://doi.org/10.1007/s10278-020-00392-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-020-00392-4

Keywords

Search

Navigation