LumiPath – Towards Real-Time Physically-Based Rendering on Embedded Devices
With the increasing computational power of today’s workstations, real-time physically-based rendering is within reach, rapidly gaining attention across a variety of domains. These have expeditiously applied to medicine, where it is a powerful tool for intuitive 3D data visualization. Embedded devices such as optical see-through head-mounted displays (OST HMDs) have been a trend for medical augmented reality. However, leveraging the obvious benefits of physically-based rendering remains challenging on these devices because of limited computational power, memory usage, and power consumption. We navigate the compromise between device limitations and image quality to achieve reasonable rendering results by introducing a novel light field that can be sampled in real-time on embedded devices. We demonstrate its applications in medicine and discuss limitations of the proposed method. An open-source version of this project is available at https://github.com/lorafib/LumiPath which provides full insight on implementation and exemplary demonstrational material.
KeywordsLight field Fibonacci Augmented reality
The Titan V used for this research was donated by the Nvidia Corporation. The authors would like to thank Benjamin Keinert for helping to understand the inverse Fibonacci mapping and Arian Mehrfard for his help in acquiring screenshots.
Supplementary material 1 (mp4 60444 KB)
- 2.Coretti, M.: Study and implementation of a decomposable virtual skull in 54 anatomically correct elements. ThesisGoogle Scholar
- 3.Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M.F.: The lumigraph. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1996, pp. 43–54. ACM, New York (1996)Google Scholar
- 4.Hajek, J., et al.: Closing the calibration loop: an inside-out-tracking paradigm for augmented reality in orthopedic surgery. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 299–306. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_35CrossRefGoogle Scholar
- 5.Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: CVPR, pp. 5967–5976. IEEE (2017)Google Scholar
- 6.Jolicoeur-Martineau, A.: The relativistic discriminator: a key element missing from standard GAN. arXiv preprint arXiv:1807.00734 (2018)
- 10.McMillan, L., Bishop, G.: Plenoptic modeling: an image-based rendering system. In: SIGGRAPH, pp. 39–46. Citeseer (1995)Google Scholar
- 11.Pharr, M., Jakob, W., Humphreys, G.: Physically Based Rendering: From Theory to Implementation. Morgan Kaufmann, Burlington (2016)Google Scholar
- 14.Zhou, S.K., Engel, K.: Method and system for volume rendering based on 3D image filtering and real-time cinematic rendering. US Patent 9,984,493 (2018)Google Scholar