Skip to main content
SpringerLink
Log in

Photorealistic rendering: a survey on evaluation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This article is a systematic collection of existing methods and techniques for evaluating rendering category in the field of computer graphics. The motive for doing this study was the difficulty of selecting appropriate methods for evaluating and validating specific results reported by many researchers. This difficulty lies in the availability of numerous methods and lack of robust discussion of them. To approach such problems, the features of well-known methods are critically reviewed to provide researchers with backgrounds on evaluating different styles in photo-realistic rendering part of computer graphics. There are many ways to evaluating a research. For this article, classification and systemization method is use. After reviewing the features of different methods, their future is also discussed. Finally, dome pointers are proposed as to the likely future issues in evaluating the research on realistic rendering. It is expected that this analysis helps researchers to overcome the difficulties of evaluation not only in research, but also in application.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Ackroyd S, Hughes JA (1981) Data collection in context. Longman, London

    Google Scholar 

  2. Aittala M (2010) Inverse lighting and photorealistic rendering for augmented reality. Vis Comput 26(6-8):669–678

    Article  Google Scholar 

  3. Alhajhamad H, Sunar MS, Kolivand H (2015) Automatic estimation of illumination features for indoor photorealistic rendering in augmented reality. In: International conference on intelligent software methodologies, tools, and techniques. Springer, pp 541–554

  4. Ali HH, Kolivand H, Sunar MS (2017) Soft bilateral filtering shadows using multiple image-based algorithms. Multimed Tools Appl 76(2):2591–2608

    Article  Google Scholar 

  5. Ando R, Thürey N, Wojtan C (2013) Highly adaptive liquid simulations on tetrahedral meshes. ACM Trans Graph (TOG) 32(4):103

    Article  MATH  Google Scholar 

  6. Annen T, Mertens T, Bekaert P, Seidel H-P, Kautz J (2007) Convolution shadow maps. In: Proceedings of the 18th Eurographics conference on rendering techniques. Eurographics Association, pp 51–60

  7. Annen T, Dong Z, Mertens T, Bekaert P, Seidel H, Kautz J (2008) Real-time, all-frequency shadows in dynamic scenes. ACM Trans Graph (Proc ACM SIGGRAPH 2008) 27(3):1–34

    Article  Google Scholar 

  8. Annen T, Mertens T, Seidel HP, Flerackers E, Kautz J (2008) Exponential shadow maps. In: Proceedings of graphics interface 2008. Canadian Information Processing Society, pp 155–161

  9. Audet S, Okutomi M (2009) A user-friendly method to geometrically calibrate projector-camera systems. In: 2009 IEEE Computer society conference on computer vision and pattern recognition Workshops, 2009. CVPR Workshops. IEEE, pp 47–54

  10. Baran I, Chen J, Ragan-Kelley J, Durand F, Lehtinen J (2010) A hierarchical volumetric shadow algorithm for single scattering. ACM Trans Graph (TOG) 29(6):178. aCM

    Article  Google Scholar 

  11. Barringer R, Akenine-Möller T (2013) A 4: asynchronous adaptive anti-aliasing using shared memory. ACM Trans Graph (TOG) 32(4):100

    Article  Google Scholar 

  12. Billeter M, Sintorn E, Assarsson U (2010) Real time volumetric shadows using polygonal light volumes. In: Proceedings of the conference on high performance graphics, pp 39–45

  13. Bojsen-Hansen M, Wojtan C (2013) Liquid surface tracking with error compensation. ACM Trans Graph (TOG) 32:4

    Article  MATH  Google Scholar 

  14. Chinthammit W, Merritt T, Pedersen S, Williams A, Visentin D, Rowe R, Furness T (2014) Ghostman: augmented reality application for telerehabilitation and remote instruction of a novel motor skill. BioMed Res Int, 2014

  15. Crow FC (1977) The aliasing problem in computer-generated shaded images. Commun ACM 20(11):799–805

    Article  Google Scholar 

  16. Dachsbacher C, Stamminger M (2005) Reflective shadow maps. In: Proceedings of the 2005 symposium on interactive 3D graphics and games. ACM, pp 203–231

  17. De Rousiers C, Bousseau A, Subr K, Holzschuch N, Ramamoorthi R (2011) Real-time rough refraction. In: Symposium on interactive 3D graphics and games. ACM, pp PAGE–7

  18. Debevec P, scenes Rendering synthetic objects into real (1998) Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In: SIGGRAPH ’98 Conf. Proc., pp 189–198

  19. Dimitrov R (2007) Cascaded shadow maps. NVIDIA, Technical Report

  20. Dobashi Y, Yamamoto T, Nishita T (2002) Interactive rendering of atmospheric scattering effects using graphics hardware. In: Proceedings of the ACM SIGGRAPH/EUROGRAPHICS conference on graphics hardware, pp 99–107

  21. Donnelly W, Lauritzen A (2006) Variance shadow maps. In: Proceedings of the 2006 ACM SIGGRAPH symposium on interactive 3D graphics and games, pp 161–165

  22. Eisemann E, Décoret X (2006) Fast scene voxelization and applications. In: Proceedings of the 2006 symposium on interactive 3D graphics and games. ACM, pp 71–78

  23. Engelhardt T, Dachsbacher C (2010) Epipolar sampling for shadows and crepuscular rays in participating media with single scattering. In: Proceedings of the 2010 ACM SIGGRAPH symposium on interactive 3D graphics and games. ACM, pp 119–125

  24. Fernando R (2005) Percentage-closer soft shadows. In: ACM SIGGRAPH 2005 Sketches. ACM, pp 35

  25. Figueiredo LHd, Velho L et al (2012) Realistic shadows for mobile augmented reality. In: 2012 14th Symposium on virtual and augmented reality (SVR). IEEE, pp 36–45

  26. Fournier A, Gunawan AS, Romanzin C (1993) Common illumination between real and computer generated scenes. In Graphics interface. Canadian Information Processing Society, pp 254–254

  27. Glaister A (2008) Windows advanced rasterization platform (WARP) guide. MSDN

  28. Grasset R, Langlotz T, Kalkofen D, Tatzgern M, Schmalstieg D (2012) Image-driven view management for augmented reality browsers. In: 2012 IEEE International symposium on mixed and augmented reality (ISMAR). IEEE, pp 177–186

  29. Grosch T (2005) Differential photon mapping-consistent augmentation of photographs with correction of all light paths. In: Eurographics, pp 53–56

  30. Grosch T, Eble T, Mueller S (2007) Consistent interactive augmentation of live camera images with correct near-field illumination. In: Proceedings of the 2007 ACM symposium on virtual reality software and technology. ACM, pp 125–132

  31. Gruber L, Richter-Trummer T, Schmalstieg D (2012) Real-time photometric registration from arbitrary geometry. In: 2012 IEEE International symposium on mixed and augmented reality (ISMAR). IEEE, pp 119–128

  32. Haller M, Drab S, Hartmann W (2003) A real-time shadow approach for an augmented reality application using shadow volumes. Proc VRST 03:56–65

    Google Scholar 

  33. Hensley J, Scheuermann T, Coombe G, Singh M, Lastra A (2005) Fast summed-area table generation and its applications. Comput Graph Forum 24 (3):547–555

    Article  Google Scholar 

  34. Hewitt M (2007) How to search and critically evaluate research literature. The NIHR RDS for the East Midlands/Yorkshire & the Humber

  35. Hosek L, Wilkie A (2012) An analytic model for full spectral sky-dome radiance. ACM Trans Graph (TOG) 31(4):95. aCM

    Article  Google Scholar 

  36. Izadi S, Kim D, Hilliges O, Molyneaux D, Newcombe R, Kohli P, Shotton J, Hodges S, Freeman D, Davison A et al (2011) Kinectfusion: real-time 3d reconstruction and interaction using a moving depth camera. In: Proceedings of the 24th annual ACM symposium on user interface software and technology. ACM, pp 559–568

  37. Jansen J, Bavoil L (2010) Fourier opacity mapping. In: Proceedings of the 2010 ACM SIGGRAPH symposium on interactive 3D graphics and games. ACM, pp 165–172

  38. Jensen HW (2001) Realistic image synthesis using photon mapping. AK Peters, Ltd.

  39. Jensen B, Laursen J, Madsen J, Pedersen T (2009) Simplifying real time light source tracking and credible shadow generation for augmented reality. Institute for Media Technology Aalborg University

  40. Jia N, Luo D, Zhang (2013) Distorted shadow mapping. In: Proceedings of the 19th ACM symposium on virtual reality software and technology. ACM, pp 209–214

  41. Jiang D, Zhao Y, Sahli H, Zhang Y (2013) Speech driven photo realistic facial animation based on an articulatory dbn model and aam features. Multimed Tools Appl, 1–19

  42. Jimenez J, Gutierrez D, Yang J, Reshetov A, Demoreuille P, Berghoff T, Perthuis C, Yu H, McGuire M, Lottes T et al (2011) Filtering approaches for real-time anti-aliasing. ACM SIGGRAPH Courses 2(3):4

    Google Scholar 

  43. Kán P, Kaufmann H (2012) High-quality reflections, refractions, and caustics in augmented reality and their contribution to visual coherence. In: 2012 IEEE International symposium on mixed and augmented reality (ISMAR). IEEE, pp 99–108

  44. Kanbara M, Yokoya N (2004) Real-time estimation of light source environment for photorealistic augmented reality. In: Proceedings of the 17th international conference on pattern recognition, pp 911–914. Cambridge

  45. Kavan L, Collins S, zára J, O’Sullivan C (2008) Geometric skinning with approximate dual quaternion blending. ACM Trans Graph (TOG) 27(4):105

    Article  Google Scholar 

  46. Keller A (1997) Instant radiosity. In: Proceedings of the 24th annual conference on computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., pp 49–56

  47. Keller M, Lefloch D, Lambers M, Izadi S, Weyrich T, Kolb A (2013) Real-time 3d reconstruction in dynamic scenes using point-based fusion. In: 2013 International conference on 3DTV-conference. IEEE, pp 1–8

  48. Kitchenham B (2004) Procedures for performing systematic reviews, Keele, UK. Keele Univ 33(2004):1–26

    Google Scholar 

  49. Knecht M, Traxler C, Mattausch O, Purgathofer W, Wimmer M (2010) Differential instant radiosity for mixed reality. In: 2010 9th IEEE International symposium on mixed and augmented reality (ISMAR). IEEE, pp 99–107

  50. Kolivand H (2013) Shadow and sky colour rendering technique in augmented reality environments, PhD Thesis, Universiti Teknologi, Malaysia

  51. Kolivand H, Sunar M (2012) Real-time outdoor rendering using hybrid shadow maps, International. J Innov Comput Inf Control (IJICIC) 18(10B):7169–7184

    Google Scholar 

  52. Kolivand H, Noh Z, Sunar MS (2014) A quadratic spline approximation using detail multi-layer for soft shadow generation in augmented reality. Multimed Tools Appl 73(3)

  53. Kolivand H, Sunar M (2014) Covering photometric properties of outdoor components with the effects of sky color in mixed reality. Multimed Tools Appl 72(3):2143–2162

    Article  Google Scholar 

  54. Kolivand H, Sunar MS (2014) Anti-aliasing in image based shadow generation techniques: a comprehensive survey. Multimed Tools Appl, 1–27

  55. Kolivand H, Sunar MS (2014) Realistic real-time outdoor rendering in augmented reality. PloS One 9(9):e108334

    Article  Google Scholar 

  56. Kolivand H, Billinghurst M, Sunar MS (2016) Livephantom: retrieving virtual world light data to real environments. PloS One 11(12):e0166424

    Article  Google Scholar 

  57. Lauritzen A, McCool M (2008) Layered variance shadow maps. In: GI ’08: Proceedings of graphics interface 2008 Toronto, Ontario, Canada). Canadian Information Processing Society, pp 139–146

  58. Lavoué G, Liu H, Myszkowski K, Lin W (2016) Quality assessment and perception in computer graphics. IEEE Comput Graph Appl 36(4):21–22

    Article  Google Scholar 

  59. Lensing P, Broll W (2012) Instant indirect illumination for dynamic mixed reality scenes. In: 2012 IEEE International symposium on mixed and augmented reality (ISMAR). IEEE, pp 109–118

  60. Levine S, Wang JM, Haraux A, Popovic Z, Koltun V (2012) Continuous character control with low-dimensional embeddings. ACM Trans Graph (TOG) 31(4):28

    Article  Google Scholar 

  61. Li D, Sueda S, Neog DR, Pai DK (2013) Thin skin elastodynamics. ACM Trans Graph (TOG) 32(4):49

    MATH  Google Scholar 

  62. Lieberknecht S, Benhimane S, Meier P, Navab N (2009) A dataset and evaluation methodology for template-based tracking algorithms. In: 8th IEEE International symposium on mixed and augmented reality, 2009. ISMAR 2009. IEEE, pp 145–151

  63. Liu N, Pang M (2009) A survey of shadow rendering algorithms: projection shadows and shadow volumes. In: Second international workshop on computer science and engineering, pp 488–492

  64. Liu Y, Qin X, Xing G, Peng Q (2010) A new approach to outdoor illumination estimation based on statistical analysis for augmented reality. Comput Anim Virt Worlds 21(3–4):321–330. Wiley Online Library

    Google Scholar 

  65. Liu Y, Qin X, Xing G, Peng Q (2010) A new approach to outdoor illumination estimation based on statistical analysis for augmented reality. Comput Anim Virt Worlds 21(3-4):321–330

    Google Scholar 

  66. Liu Y, Qin X, Xu S, Nakamae E, Peng Q (2009) Light source estimation of outdoor scenes for mixed reality. Vis Comput 25(5):637–646

    Article  Google Scholar 

  67. Lokovic T, Veach E (2000) Deep shadow maps. In: Proceedings of the 27th annual conference on computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., pp 385–392

  68. Macklin M, Muller M (2013) Position based fluids. Proc SIGGRAPH 32 (4):2013. ACM

    MATH  Google Scholar 

  69. Madsen CB, Lal BB (2013) Estimating outdoor illumination conditions based on detection of dynamic shadows. Computer vision, imaging and computer graphics. Theory and applications. Springer, pp 33–52

  70. Magnenat-Thalmann N, Laperrire R, Thalmann D et al (1988) Joint-dependent local deformations for hand animation and object grasping. In: Proceedings on graphics interface’88. Citeseer

  71. Malterud K (2001) Qualitative research: standards, challenges, and guidelines. Lancet 358(9280):483–488

    Article  Google Scholar 

  72. Martin T, Tan T-S (2004) Anti-aliasing and continuity with trapezoidal shadow maps. In: Proceedings of 15th Eurographics symposium on rendering. Norrköping, pp 153–160

  73. Maule M, Comba JL, Torchelsen R, Bastos R (2012) Transparency and anti-aliasing techniques for real-time rendering. In: 2012 25th SIBGRAPI Conference on graphics, patterns and images tutorials (SIBGRAPI-T). IEEE, pp 50–59

  74. Max NL (1986) Atmospheric illumination and shadows. In: Computer graphics (Proc. SIGGRAPH ’86). ACM, New York, pp 117–124

  75. Mehta SU, Ramamoorthi BWR, Durand F (2013) Axis-aligned filtering for interactive physically based diffuse indirect lighting. ACM Transactions on Graphics (TOG). ACM

  76. Mehta SU, Wang B, Ramamoorthi R (2012) Axis-aligned filtering for interactive sampled soft shadows. ACM Trans Graph (TOG) 31(6):163. aCM

    Article  Google Scholar 

  77. Monaghan JJ (1992) Smoothed particle hydrodynamics. Ann Rev Astron Astrophys 30:543–574

    Article  Google Scholar 

  78. Newcombe RA, Davison AJ, Izadi S, Kohli P, Hilliges O, Shotton J, Molyneaux D, Hodges S, Kim D, Fitzgibbon A (2011) Kinectfusion: Real-time dense surface mapping and tracking. In: 2011 10th IEEE International symposium on mixed and augmented reality (ISMAR). IEEE, pp 127–136

  79. Nowrouzezahrai D, Geiger S, Mitchell K, Sumner R, Jarosz W, Gross M (2011) Light factorization for mixed-frequency shadows in augmented reality. In: 2011 10th IEEE International symposium on mixed and augmented reality (ISMAR). IEEE, pp 173–179

  80. Oliveira MM, Brauwers M (2007) Real-time refraction through deformable objects. In: Proceedings of the 2007 symposium on interactive 3D graphics and games. ACM, pp 89–96

  81. Pan M, Wang R, Chen W, Zhou K, Bao H (2009) Fast, sub-pixel antialiased shadow maps. Comput Graph Forum 28(7):1927–1934

    Article  Google Scholar 

  82. Park Y, Lepetit V, Woo W (2009) Esm-blur: handling & rendering blur in 3d tracking and augmentation. In: 8th IEEE International symposium on mixed and augmented reality, 2009. ISMAR 2009. IEEE, 163–166

  83. Park Y, Lepetit V, Woo W (2012) Handling motion-blur in 3d tracking and rendering for augmented reality. IEEE Trans Vis Comput Graph 18(9):1449–1459

    Article  Google Scholar 

  84. Patton MQ (1982) Practical evaluation. Sage Newbury Park, CA

  85. Patton MQ (1990) Qualitative evaluation and research methods. SAGE Publications, inc

  86. Patton MQ (2005) Qualitative research. Wiley Online Library

  87. Pellacini F (2010) envylight: an interface for editing natural illumination. ACM Trans Graph (TOG) 29(4):34

    Article  Google Scholar 

  88. Perez R, Seals R, Michalsky J (1993) All-weather model for sky luminance distribution - preliminary configuration and validation. Sol Energy 50:235–245

    Article  Google Scholar 

  89. Pharr M, Humphreys G (2010) Physically based rendering: from theory to implementation. Morgan Kaufmann

  90. Popper K (2002) The logic of scientific discovery. Routledge

  91. Rademacher P, Lengyel J, Cutrell E, Whitted T (2001) Measuring the perception of visual realism in images. In: Rendering techniques 2001. Springer, pp 235–247

  92. Reeves W, Salesin D, Cook PL (1987) Rendering antialiased shadows with depth maps. Comput Graph (Proc SIGGRAPH 87 21(4):557–562

    Google Scholar 

  93. Roth H, Vona M (2012) Moving volume kinectfusion. In: BMVC, pp 1–11

  94. Salvi M, Vidimce K, Lauritzen A, Lefohn A (2010) Adaptive volumetric shadow maps. Comput Graph Forum 29(4):1289–1296. Wiley Online Library

    Article  Google Scholar 

  95. Shen L, Guennebaud G, Yang B, Feng J (2011) Predicted virtual soft shadow maps with high quality filtering. Comput Graph Forum 30(2):493–502. Wiley Online Library

    Article  Google Scholar 

  96. Solenthaler B, Pajarola R (2009) Predictive-corrective incompressible sph. In: ACM Transactions on graphics (TOG), vol 28, no 3. ACM, pp 40

  97. Sugano N, Kato H, Tachibana K (2003) The effects of shadow representation of virtual objects in augmented reality. In: The Second IEEE and ACM international symposium on mixed and augmented reality 2003. Proceedings. IEEE, pp 76–83

  98. Sunar M (2001) Sky colour modelling, Master Thesis, University of Hull

  99. Takagi A, Yamazaki S, Saito Y, Taniguchi N (2000) Development of a stereo video see-through hmd for ar systems. In: IEEE and ACM International symposium on augmented reality, 2000. (ISAR 2000). IEEE, pp 68–77

  100. Tian Y, Long Y, Xia D, Yao H, Zhang J (2015) Handling occlusions in augmented reality based on 3d reconstruction method. Neurocomputing 156:96–104

    Article  Google Scholar 

  101. Wang L, Zhou S, Ke W, Popescu V (2014) Gears: a general and efficient algorithm for rendering shadows. Comput Graph Forum 33(6):264–275. Wiley Online Library

    Article  Google Scholar 

  102. Williams L (1978) Casting curved shadows on curved surfaces. SIGGRAPH ’78 12(3):270–274

    Article  Google Scholar 

  103. Wu C, Liu Y, Dai Q, Wilburn B (2011) Fusing multiview and photometric stereo for 3d reconstruction under uncalibrated illumination. IEEE Trans Vis Comput Graph 17(8):1082–1095

    Article  Google Scholar 

  104. Wyman C, Ramsey S (2008) Interactive volumetric shadows in participating media with single-scattering. In: IEEE Symposium on interactive ray tracing, 2008. RT. IEEE, pp 87–92

  105. Wyman C, Hoetzlein R, Lefohn A (2015) Frustum-traced irregular z-buffers: fast, sub-pixel accurate hard shadows. In: ACM SIGGRAPH 2015 Talks. ACM, p 69

  106. Xiao J, Moriyama T, Kanade T, Cohn JF (2003) Robust full-motion recovery of head by dynamic templates and re-registration techniques. Int J Imaging Syst Technol 13(1):85–94

    Article  Google Scholar 

  107. Xie L, Sun N, Fan B (2013) A statistical parametric approach to video-realistic text-driven talking avatar. Multimed Tools Appl, 1–20

  108. Xing G (2012) A practical approach for real-time illumination estimation of outdoor videos. Comput Graph 36:857–865

    Article  Google Scholar 

  109. Xing G, Liu Y, Qin X, Peng Q (2011) On-line illumination estimation of outdoor scenes based on area selection for augmented reality. In: 2011 12th International conference on computer-aided design and computer graphics (CAD/Graphics). IEEE, pp 43–442

  110. Zheng F, Schubert R, Welch G (2012) A general approach for closed-loop registration in ar. In: 2012 IEEE International symposium on mixed and augmented reality (ISMAR). IEEE, pp 335–336

  111. Zhou G, Zhu D, Wei Y, Wang Z, Zhou Y (2016) Real-time online learning of gaussian mixture model for opacity mapping. Neurocomputing

  112. Zokai S, Esteve J, Genc Y, Navab N (2003) Multiview paraperspective projection model for diminished reality. In: The Second IEEE and ACM international symposium on mixed and augmented reality. 2003 Proceedings. IEEE, 217–226

Download references

Acknowledgements

This research is a collaboration between Liverpool John Moores University and Universiti Teknologi Malaysia.

Author information

Authors and Affiliations

  1. Department of Computer Science, Liverpool John Moores University, Liverpool, L3 3AF, UK

    Hoshang Kolivand

  2. MaGIC-X (Media and Games Innovation Centre of Excellence) UTM-IRDA Digital Media, Centre Universiti Teknologi, Malaysia, 81310, Skudai Johor, Malaysia

    Mohd Shahrizal Sunar & Riyadh Al-Rousan

  3. EDENZ Colleges, Auckland, New Zealand

    Samira Y. Kakh

  4. Faculty of Informatics and Computing, Universiti Sultan Zainal Abidin, Kuala Terengganu, Malaysia

    Ismahafezi Ismail

Authors
  1. Hoshang Kolivand
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohd Shahrizal Sunar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Samira Y. Kakh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Riyadh Al-Rousan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ismahafezi Ismail
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hoshang Kolivand.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kolivand, H., Sunar, M.S., Kakh, S.Y. et al. Photorealistic rendering: a survey on evaluation. Multimed Tools Appl 77, 25983–26008 (2018). https://doi.org/10.1007/s11042-018-5834-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-5834-7

Keywords

Search

Navigation