Supervised Learning of How to Blend Light Transport Simulations

  • Hisanari Otsu
  • Shinichi Kinuwaki
  • Toshiya Hachisuka
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 241)


Light transport simulation is a popular approach for rendering photorealistic images. However, since different algorithms have different efficiencies depending on input scene configurations, a user would try to find the most efficient algorithm based on trials and errors. This selection of an algorithm can be cumbersome because a user needs to know technical details of each algorithm. We propose a framework which blends the results of two different rendering algorithms, such that a blending weight per pixel becomes automatically larger for a more efficient algorithm. Our framework utilizes a popular machine learning technique, regression forests, for analyzing statistics of outputs of rendering algorithms and then generating an appropriate blending weight for each pixel. The key idea is to determine blending weights based on classification of path types. This idea is inspired by the same common practice in movie industries; an artist composites multiple rendered images where each image contains only a part of light transport paths (e.g., caustics) rendered by a specific algorithm. Since our framework treats each algorithm as a black-box, we can easily combine very different rendering algorithms as long as they eventually generate the same results based on light transport simulation. The blended results with our algorithm are almost always more accurate than taking the average, and no worse than the results with an inefficient algorithm alone.


Light transport simulation Machine learning Regression forest 


  1. 1.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefGoogle Scholar
  2. 2.
    Georgiev, I., Krivanek, J., Davidovic, T., Slusallek, P.: Light transport simulation with vertex connection and merging. ACM Trans. Graph. (Proceedings of SIGGRAPH Asia) 31(6) (2012). Article 192CrossRefGoogle Scholar
  3. 3.
    Gritz, L., Stein, C., Kulla, C., Conty, A.: Open shading language. ACM SIGGRAPH 2010 Talks (2010). Article 33Google Scholar
  4. 4.
    Hachisuka, T., Jensen, H.W.: Stochastic progressive photon mapping. ACM Trans. Graph. (Proceedings of SIGGRAPH Asia) 28(5) (2009). Article 141Google Scholar
  5. 5.
    Hachisuka, T., Ogaki, S., Jensen, H.W.: Progressive photon mapping. ACM Trans. Graph. (Proceedings of SIGGRAPH Asia) 27(5) (2008). Article 130Google Scholar
  6. 6.
    Hachisuka, T., Pantaleoni, J., Jensen, H.W.: A path space extension for robust light transport simulation. ACM Trans. Graph. (Proceedings of SIGGRAPH Asia) 31(6) (2012). Article 191CrossRefGoogle Scholar
  7. 7.
    Havran, V.: Heuristic ray shooting algorithms. Ph.D. thesis, Department of Computer Science and Engineering, Faculty of Electrical Engineering, Czech Technical University in Prague, November (2000)Google Scholar
  8. 8.
    Haykin, S.: Neural Networks: A Comprehensive Foundation, 2nd edn. Prentice Hall PTR, Upper Saddle River (1998)zbMATHGoogle Scholar
  9. 9.
    Heckbert, P.S.: Adaptive radiosity textures for bidirectional ray tracing. Comput. Graph. (Proceedings of SIGGRAPH) 24(4), 145–154 (1990)CrossRefGoogle Scholar
  10. 10.
    Hinton, G.E., Osindero, S., Teh, Y.-W.: A fast learning algorithm for deep belief nets. Neural comput. 18(7), 1527–1554 (2006)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Jakob, W.: Mitsuba renderer. (2010)
  12. 12.
    Jakob, W., Marschner, S.: Manifold exploration: a Markov chain Monte Carlo technique for rendering scenes with difficult specular transport. ACM Trans. Graph. (Proceedings of SIGGRAPH) 31(4) (2012)CrossRefGoogle Scholar
  13. 13.
    Jakob, W., Regg, C., Jarosz, W.: Progressive expectation–maximization for hierarchical volumetric photon mapping. Comput. Graph. Forum (Proceedings of the Eurographics Symposium on Rendering) 30(4) (2011)CrossRefGoogle Scholar
  14. 14.
    Jensen, H.W.: Global illumination using photon maps. In: Proceedings of the Eurographics Symposium on Rendering, pp. 21–30 (1996)Google Scholar
  15. 15.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R.B., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. CoRR arXiv:1408.5093 (2014)
  16. 16.
    Kajiya, J.T.: The rendering equation. Comput. Graph. (Proceedings of SIGGRAPH) 143–150 (1986)CrossRefGoogle Scholar
  17. 17.
    Kalantari, N.K., Bako, S., Sen, P.: A machine learning approach for filtering Monte Carlo noise. ACM Trans. Graph. (TOG) (Proceedings of SIGGRAPH 2015) 34(4) (2015)CrossRefGoogle Scholar
  18. 18.
    Kaplanyan, A.S., Dachsbacher, C.: Path space regularization for holistic and robust light transport. Comput. Graph. Forum (Proceedings of the Eurographics Symposium on Rendering) 32(2), 63–72 (2013)CrossRefGoogle Scholar
  19. 19.
    Ladický, L., Jeong, S., Solenthaler, B., Pollefeys, M., Gross, M.: Data-driven fluid simulations using regression forests. ACM Trans. Graph. (Proceedings of SIGGRAPH Asia) 34(6), 199 (2015)Google Scholar
  20. 20.
    Lafortune, E., Willems, Y.D.: Bi-directional path-tracing. In: Proceedings of Compugraphics, pp. 145–153 (1993)Google Scholar
  21. 21.
    MacDonald, J.D., Booth, K.S.: Heuristics for ray tracing using space subdivision. Visual Comput. 6(3), 153–166 (1990)CrossRefGoogle Scholar
  22. 22.
    Nalbach, O., Arabadzhiyska, E., Mehta, D., Seidel, H.-P., Ritschel, T.: Deep shading: convolutional neural networks for screen-space shading. Comput. Graph. Forum (Proceedings of the EGSR 2017) 36(4) (2017)CrossRefGoogle Scholar
  23. 23.
    Ren, P., Wang, J., Gong, M., Lin, S., Tong, X., Guo, B.: Global illumination with radiance regression functions. ACM Trans. Graph. (Proceedings of SIGGRAPH) 32(4), 130:1–130:12 (2013)CrossRefGoogle Scholar
  24. 24.
    Ren, S., Cao, X., Wei, Y., Sun, J.: Face alignment at 3000 fps by regressing local binary features. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)Google Scholar
  25. 25.
    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vision (IJCV) 115(3), 211–252 (2015). Scholar
  26. 26.
    Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-time human pose recognition in parts from a single depth images. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011)Google Scholar
  27. 27.
    Tang, D., Yu, T.-H., Kim, T.-K.: Real-time articulated hand pose estimation using semi-supervised transductive regression forests. In: The IEEE International Conference on Computer Vision (ICCV) (2013)Google Scholar
  28. 28.
    Veach, E.: Robust Monte Carlo methods for light transport simulation. Ph.D. thesis, Stanford University, USA (1998). AAI9837162Google Scholar
  29. 29.
    Veach, E., Guibas, L.J.: Bidirectional estimator for light transport. In: Proceedings of the Eurographics Symposium on Rendering, pp. 147–162 (1994)Google Scholar
  30. 30.
    Veach, E., Guibas, L.J.: Optimally combining sampling techniques for Monte Carlo rendering. In: Proceedings of SIGGRAPH ’95, pp. 419–428 (1995)Google Scholar
  31. 31.
    Veach, E., Guibas, L.J.: Metropolis light transport. Proceedings of SIGGRAPH 97, 65–76 (1997)Google Scholar
  32. 32.
    Vorba, J., Karlík, O., Šik, M., Ritschel, T., Křivánek, J.: On-line learning of parametric mixture models for light transport simulation. ACM Trans. Graph. (Proceedings of SIGGRAPH) 33(4) (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Hisanari Otsu
    • 1
  • Shinichi Kinuwaki
    • 2
  • Toshiya Hachisuka
    • 1
  1. 1.The University of TokyoBunkyo-ku, TokyoJapan
  2. 2.Bunkyo-ku, TokyoJapan

Personalised recommendations