Advertisement

Computational Visual Media

, Volume 4, Issue 3, pp 267–276 | Cite as

Instantaneous foveated preview for progressive Monte Carlo rendering

  • Matias K. Koskela
  • Kalle V. Immonen
  • Timo T. Viitanen
  • Pekka O. Jääskeläinen
  • Joonas I. Multanen
  • Jarmo H. Takala
Open Access
Research Article
  • 92 Downloads

Abstract

Progressive rendering, for example Monte Carlo rendering of 360° content for virtual reality headsets, is a time-consuming task. If the 3D artist notices an error while previewing the rendering, they must return to editing mode, make the required changes, and restart rendering. We propose the use of eye-tracking-based optimization to significantly speed up previewing of the artist’s points of interest. The speed of the preview is further improved by sampling with a distribution that closely follows the experimentally measured visual acuity of the human eye, unlike the piecewise linear models used in previous work. In a comprehensive user study, the perceived convergence of our proposed method was 10 times faster than that of a conventional preview, and often appeared to be instantaneous. In addition, the participants rated the method to have only marginally more artifacts in areas where it had to start rendering from scratch, compared to conventional rendering methods that had already generated image content in those areas.

Keywords

foveated rendering progressive rendering Monte Carlo rendering preview 360° content 

Notes

Acknowledgements

The authors would like to thank the creators of the 3D models used in the user study: Christophe Seux for Classroom, Anat Grynberg and Greg Ward for Conference, Marko Dabrovic for Sibenik (License: CC BY-NC), and Frank Meinl for Crytek Sponza (License: CC BY 3.0). In addition, the authors would like to thank Heli Väätäjä, Chelsea Kelling, and Otto Kauhanen for helpful discussions.

Supplementary material

Supplementary material, approximately 101 MB.

References

  1. [1]
    Pharr, M.; Jakob, W.; Humphreys, G. Physically Based Rendering: From Theory to Implementation, 2nd edn. Morgan Kaufmann, 2010.Google Scholar
  2. [2]
    Strasburger, H.; Rentschler, I.; Jüttner, M. Peripheral vision and pattern recognition: A review. Journal of Vision Vol. 11, No. 5, 13, 2011.CrossRefGoogle Scholar
  3. [3]
    Koskela, M.; Viitanen, T.; Jääskeläinen, P.; Takala, J. Foveated path tracing. In: Advances in Visual Computing. Lecture Notes in Computer Science, Vol. 10072. Bebis, G.; Boyle, R.; Parvin, B. et al. Eds. Springer Cham, 723–732, 2016CrossRefGoogle Scholar
  4. [4]
    Koskela, M.; Immonen, K.; Viitanen, T.; Jääskeläinen, P.; Multanen, J.; Takala, J. Foveated instant preview for progressive rendering. In: Proceedings of the SIGGRAPH Asia 2017 Technical Briefs, Article No. 10, 2017.Google Scholar
  5. [5]
    Weier, M.; Stengel, M.; Roth, T.; Didyk, P.; Eisemann, E.; Eisemann, M.; Grogorick, S.; Hinkenjann, A.; Kruijff, E.; Magnor, M.; Myszkowski, K.; Slusallek, P. Perception-driven accelerated rendering. Computer Graphics Forum Vol. 36, No. 2, 611–643, 2017CrossRefGoogle Scholar
  6. [6]
    Shibata, T. Head mounted display. Displays Vol. 23, Nos. 1–2, 57–64, 2002CrossRefGoogle Scholar
  7. [7]
    Lee, E. C.; Park, K. R. A robust eye gaze tracking method based on a virtual eyeball model. Machine Vision and Applications Vol. 20, No. 5, 319–337, 2009CrossRefGoogle Scholar
  8. [8]
    Guenter, B.; Finch, M.; Drucker, S.; Tan, D.; Snyder, J. Foveated 3D graphics. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 164, 2012.Google Scholar
  9. [9]
    Vaidyanathan, K.; Salvi, M.; Toth, R.; Foley, T.; Akenine-Möller, T.; Nilsson, J.; Munkberg, J.; Hasselgren, J.; Sugihara, M.; Clarberg, P.; Janczak, T.; Lefohn, A. Coarse pixel shading. In: Proceedings of High Performance Graphics, 9–18, 2014Google Scholar
  10. [10]
    Stengel, M.; Grogorick, S.; Eisemann, M.; Magnor, M. Adaptive image-space sampling for gaze-contingent real-time rendering. Computer Graphics Forum Vol. 35, No. 4, 129–139, 2016CrossRefGoogle Scholar
  11. [11]
    Weier, M.; Roth, T.; Kruijff, E.; Hinkenjann, A.; Pérard-Gayot, A.; Slusallek, P.; Li, Y. Foveated real-time ray tracing for head-mounted displays. Computer Graphics Forum Vol. 35, No. 7, 289–298, 2016CrossRefGoogle Scholar
  12. [12]
    Murphy, H. A.; Duchowski, A. T.; Tyrrell, R. A. Hybrid image/model-based gaze-contingent rendering. ACM Transactions on Applied Perception Vol. 5, No. 4, Article No. 22, 2009.Google Scholar
  13. [13]
    Reddy, M. Perceptually optimized 3D graphics. IEEE Computer Graphics and Applications Vol. 21, No. 5, 68–75, 2001CrossRefGoogle Scholar
  14. [14]
    Pohl, D.; Zhang, X.; Bulling, A. Combining eye tracking with optimizations for lens astigmatism in modern wideangle HMDs. In: Proceedings of the IEEE Virtual Reality, 269–270, 2016Google Scholar
  15. [15]
    Roth, T.; Weier, M.; Maiero, J.; Hinkenjann, A.; Li, Y. Guided high-quality rendering. In: Advances in Visual Computing. Lecture Notes in Computer Science, Vol. 9475. Bebis, G.; Boyle, R.; Parvin, B. et al. Eds. Springer Cham, 115–125, 2015Google Scholar
  16. [16]
    Pixar. Renderman 20 documentation: Rendering efficiently. 2017. Available at https://doi.org/renderman.pixar.com/resources/RenderMan_20/tutorialRenderingEfficiently.html.Google Scholar
  17. [17]
    The community of LuxRender. LuxRender documentation: Refine brush. 2013. Available at https://doi.org/www.luxrender.net/wiki/Refine_Brush.
  18. [18]
    Duchowski, A. T.; Bate, D.; Stringfellow, P.; Thakur, K.; Melloy, B. J.; Gramopadhye, A. K. On spatiochromatic visual sensitivity and peripheral color LOD management. ACM Transactions on Applied Perception Vol. 6, No. 2, Article No. 9, 2009.Google Scholar
  19. [19]
    Viitanen, T.; Koskela, M.; Immonen, K.; Mäkitalo, M.; Jääskeläinen, P.; Takala, J. Sparse sampling for real-time ray tracing. In: Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, Vol. 1, 295–302, 2018.CrossRefGoogle Scholar
  20. [20]
    Devroye, L. Non-Uniform Random Variate Generation. Springer-Verlag, 1986.CrossRefzbMATHGoogle Scholar
  21. [21]
    Weisstein, E. Lambert W-function. Available at https://doi.org/mathworld.wolfram.com/LambertW-Function.html.
  22. [22]
    MacQuarrie, A.; Steed, A. Cinematic virtual reality: Evaluating the effect of display type on the viewing experience for panoramic video. In: Proceedings of the IEEE Virtual Reality, 45–54, 2017.Google Scholar
  23. [23]
    Stark, R.; Israel, J. H.; Wöhler, T. Towards hybrid modelling environments—Merging desktop-CAD and virtual reality-technologies. CIRP Annals Vol. 59, No. 1, 179–182, 2010.CrossRefGoogle Scholar
  24. [24]
  25. [25]
  26. [26]
    FOVE Inc. Tech specs. Available at https://doi.org/www.getfov.com/.

Copyright information

© The Author(s) 2018

Authors and Affiliations

  • Matias K. Koskela
    • 1
  • Kalle V. Immonen
    • 1
  • Timo T. Viitanen
    • 1
  • Pekka O. Jääskeläinen
    • 1
  • Joonas I. Multanen
    • 1
  • Jarmo H. Takala
    • 1
  1. 1.Tampere University of TechnologyTampereFinland

Personalised recommendations