Mirrors in Computer Graphics, Computer Vision and Time-of-Flight Imaging

  • Ilya Reshetouski
  • Ivo Ihrke
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8200)

Abstract

Mirroring is one of the fundamental light/surface interactions occurring in the real world. Surfaces often cause specular reflection, making it necessary to design robust geometry recovery algorithms for many practical situations. In these applications the specular nature of the surface is a challenge. On the other side, mirrors, with their unique reflective properties, can be used to improve our sensing modalities, enabling applications such as surround, stereo and light field imaging. In these scenarios the specular interactions are highly desirable. Both of these aspects, the utilization and circumvention of mirrors are present in a significant amount of publications in different scientific areas. These publications are covering a large number of different problem statements as well as many different approaches to solutions. In the chapter we will focus on a collection and classification of the work in this area.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Smith, W.J.: Modern Optical Engineering, 4th edn. McGraw Hill Professional (2008)Google Scholar
  2. 2.
    Reshetouski, I., Manakov, A., Seidel, H.P., Ihrke, I.: Three-Dimensional Kaleidoscopic Imaging. In: Proc. CVPR, pp. 353–360 (2011)Google Scholar
  3. 3.
    Sossinsky, A.B.: Geometries, vol. 64. American Mathematical Soc. (2012)Google Scholar
  4. 4.
    Gluckman, J., Nayar, S.K.: Rectified Catadioptric Stereo Sensors. In: Proc. CVPR, pp. 380–387 (2000)Google Scholar
  5. 5.
    Wu, H.H.P., Chang, S.H.: Design of Stereoscopic Viewing System Based on a Compact Mirror and Dual Monitor. Optical Engineering 49(2), 027401-1–027401-6 (2010)Google Scholar
  6. 6.
    Gluckman, J., Nayar, S.K.: A Real-Time Catadioptric Stereo System Using Planar Mirrors. In: Proc. of Image Understanding Workshop (1998)Google Scholar
  7. 7.
    Gluckman, J., Nayar, S.: Rectified Catadioptric Stereo Sensors. IEEE Trans. PAMI 24(2), 224–236 (2002)CrossRefGoogle Scholar
  8. 8.
    Mitsumoto, H., Tamura, S., Okazaki, K., Kajimi, N., Fukui, Y.: 3-D Reconstruction Using Mirror Images Based on a Plane Symmetry Recovering Method. IEEE Trans. PAMI 14, 941–946 (1992)CrossRefGoogle Scholar
  9. 9.
    Ihrke, I., Stich, T., Gottschlich, H., Magnor, M., Seidel, H.P.: Fast Incident Light Field Acquisition and Rendering. WSCG 16(1-3), 25–32 (2008)Google Scholar
  10. 10.
    Murray, D.W.: Recovering Range using Virtual Multi-Camera Stereo. CVIU 61(2), 285–291 (1995)Google Scholar
  11. 11.
    Hu, B., CRV: It’s All Done with Mirrors: Calibration-and-Correspondence-Free 3D Reconstruction. In: Proc. CRV, pp. 148–154 (2009)Google Scholar
  12. 12.
    Sen, P., Chen, B., Garg, G., Marschner, S.R., Horowitz, M., Levoy, M., Lensch, H.P.A.: Dual photography. ACM TOG 24, 745–755 (2005)CrossRefGoogle Scholar
  13. 13.
    Fuchs, M., Kächele, M., Rusinkiewicz, S.: Design and Fabrication of Faceted Mirror Arrays for Light Field Capture. In: Proc. VMV, pp. 1–8 (2012)Google Scholar
  14. 14.
    Levoy, M., Chen, B., Vaish, V., Horowitz, M., McDowall, I., Bolas, M.: Synthetic Aperture Confocal Imaging. ACM TOG 23, 825–834 (2004)CrossRefGoogle Scholar
  15. 15.
    Mukaigawa, Y., Tagawa, S., Kim, J., Raskar, R., Matsushita, Y., Yagi, Y.: Hemispherical confocal imaging using turtleback reflector. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part I. LNCS, vol. 6492, pp. 336–349. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  16. 16.
    Weyrich, T., Peers, P., Matusik, W., Rusinkiewicz, S.: Fabricating Microgeometry for Custom Surface Reflectance. ACM TOG 28(3), 32:1–32:6 (2009)Google Scholar
  17. 17.
    Aggarwal, M., Ahuja, N.: Split aperture imaging for high dynamic range. In: Proc. ICCV, vol. 2, pp. 10–17 (2001)Google Scholar
  18. 18.
    Tan, K.H., Hua, H., Ahuja, N.: Multiview Panoramic Cameras Using Mirror Pyramids. IEEE Trans. PAMI 26(7), 941–946 (2004)CrossRefGoogle Scholar
  19. 19.
    Harvey, A.R., Fletcher-Holmes, D.W., Gorman, A.: Spectral Imaging in a Snapshot. In: Proc. SPIE, vol. 5694, pp. 1–10 (2005)Google Scholar
  20. 20.
    Gao, L., Kester, R.T., Tkaczyk, T.S.: Compact Image Slicing Spectrometer (ISS) for Hyperspectral Fluorescence Microscopy. Optics Express 17(15), 12293–12308 (2009)CrossRefGoogle Scholar
  21. 21.
    Gorman, A., Fletcher-Holmes, D.W., Harvey, A.R.: Generalization of the Lyot Filter and its Application to Snapshot Spectral Imaging. Optics Express 18(6), 5602–5608 (2010)CrossRefGoogle Scholar
  22. 22.
    McGuire, M., Matusik, W., Pfister, H., Chen, B., Hughes, J.F., Nayar, S.K.: Optical Splitting Trees for High-Precision Monocular Imaging. IEEE Comput. Graph. & Appl. 27(2), 32–42 (2007)CrossRefGoogle Scholar
  23. 23.
    Wetzstein, G., Ihrke, I., Lanman, D., Heidrich, W.: Computational plenoptic imaging. Computer Graphics Forum 30(8), 2397–2426 (2011)CrossRefGoogle Scholar
  24. 24.
    Zhou, C., Nayar, S.: Computational Cameras: Convergence of Optics and Processing. IEEE Trans. IP 20(12), 3322–3340 (2011)Google Scholar
  25. 25.
    Lanman, D., Crispell, D., Taubin, G.: Surround Structured Lighting for Full Object Scanning. In: Proc. 3DIM, pp. 107–116 (2007)Google Scholar
  26. 26.
    Huang, P.H., Lai, S.H.: Contour-Based Structure from Reflection. In: Proc. CVPR, pp. 165–178 (2006)Google Scholar
  27. 27.
    Forbes, K., Nicolls, F., de Jager, G., Voigt, A.: Shape-from-Silhouette with Two Mirrors and an Uncalibrated Camera. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 165–178. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  28. 28.
    Lanman, D., Crispell, D., Taubin, G.: Surround Structured Lighting: 3-D Scanning with Orthographic Illumination. CVIU (113), 1107–1117 (2009)Google Scholar
  29. 29.
    Ying, X., Peng, K., Ren, R., Zha, H.: Geometric Properties of Multiple Reflections in Catadioptric Camera with Two Planar Mirrors. In: Proc. CVPR, pp. 1–8 (2010)Google Scholar
  30. 30.
    Gluckman, J., Nayar, S.K.: Ego-Motion and Omnidirectional Cameras. In: Proc. ICCV, pp. 999–1005 (1998)Google Scholar
  31. 31.
    Gluckman, J., Nayar, S.K.: Real-Time Omnidirectional and Panoramic Stereo. In: Proc. of Image Understanding Workshop (1998)Google Scholar
  32. 32.
    Nayar, S.: Catadioptric Omnidirectional Camera. In: Proc. CVPR, pp. 482–488 (1997)Google Scholar
  33. 33.
    Chahl, J.S., Srinivasan, M.V.: Reflective Surfaces for Panoramic Imaging. Appl. Optics 36(31), 8275–8285 (1997)CrossRefGoogle Scholar
  34. 34.
    Hicks, R.A., Bajcsy, R.: Catadioptric Sensors that Approximate Wide-angle Perspective Projections. In: Proc. CVPR, pp. 1–7 (2000)Google Scholar
  35. 35.
    Hicks, R.A.: Designing a Mirror to Realize a Given Projection. JOSA 22(2), 323–330 (2005)CrossRefGoogle Scholar
  36. 36.
    Srinivasan, M.V.: A New Class of Mirrors for Wide-Angle Imaging. In: Proc. OMNIVIS, pp. 1–8 (2003)Google Scholar
  37. 37.
    Swaminathan, R., Grossberg, M.D., Nayar, S.K.: Designing Mirrors for Catadioptric Systems that Minimize Image Errors. In: Proc. OMNIVIS, pp. 1–8 (2004)Google Scholar
  38. 38.
    Swaminathan, R., Grossberg, M.D., Nayar, S.K.: Framework for Designing Catadioptric Projection and Imaging Systems. In: Proc. OMNIVIS, pp. 1–8 (2003)Google Scholar
  39. 39.
    Baker, S., Nayar, S.K.: A Theory of Catadioptric Image Formation. In: Proc. ICCV, pp. 35–42 (1998)Google Scholar
  40. 40.
    Baker, S., Nayar, S.K.: A Theory of Single-Viewpoint Catadioptric Image Formation. IJCV 35(2), 175–196 (1999)CrossRefGoogle Scholar
  41. 41.
    Zhang, L., Nayar, S.K.: Projection Defocus Analysis for Scene Capture and Image Display. ACM TOG 25, 907–915 (2006)CrossRefGoogle Scholar
  42. 42.
    Kuthirummal, S., Nayar, S.K.: Multiview Radial Catadioptric Imaging for Scene Capture. ACM TOG 25(3) (2006)Google Scholar
  43. 43.
    Unger, J., Wenger, A., Hawkins, T., Gardner, A., Debevec, P.: Capturing and Rendering with Incident Light Fields. In: Proc. EGWR, pp. 141–149 (2003)Google Scholar
  44. 44.
    Taguchi, Y., Agrawal, A., Veeraraghavan, A., Ramalingam, S., Raskar, R.: Axial-Cones: Modeling Spherical Catadioptric Cameras for Wide-Angle Light Field Rendering. ACM Trans. Graph. 29, 172:1–172:8 (2010)Google Scholar
  45. 45.
    Lanman, D., Crispell, D., Wachs, M., Taubin, G.: Spherical Catadioptric Arrays: Construction, Multi-View Geometry, and Calibration. In: Proc. 3DPVT, pp. 81–88 (2006)Google Scholar
  46. 46.
    Ding, Y., Yu, J., Sturm, P.: Multiperspective Stereo Matching and Volumetric Reconstruction. In: Proc. ICCV, pp. 1827–1834 (2009)Google Scholar
  47. 47.
    Nayar, S.K.: Sphereo: Determining Depth using Two Specular Spheres and a Single Camera. In: SPIE Optics, Illumination, and Image Sensing for Machine Vision III, vol. 1005, pp. 245–254 (1988)Google Scholar
  48. 48.
    Lensch, H.P.A., Kautz, J., Goesele, M., Heidrich, W., Seidel, H.P.: Image-based Reconstruction of Spatial Appearance and Geometric Detail. ACM TOG 22(2), 234–257 (2003)CrossRefGoogle Scholar
  49. 49.
    Nene, S., Nayar, S.: Stereo with Mirrors. In: Proc. ICCV, pp. 1087–1094 (1998)Google Scholar
  50. 50.
    Nayar, S., Peri, V.: Folded Catadioptric Cameras. In: Proc. CVPR, vol. 2, pp. 217–223 (1999)Google Scholar
  51. 51.
    Jang, G., Kim, S., Kweon, I.: Single Camera Catadioptric Stereo System. In: Proc. OMNIVIS, pp. 1–8 (2005)Google Scholar
  52. 52.
    Lanman, D., Wachs, M., Taubin, G., Cukierman, F.: Reconstructing a 3D Line from a Single Catadioptric Image. In: Proc. 3DPVT, pp. 1–8 (2006)Google Scholar
  53. 53.
    Bimber, O., Iwai, D., Wetzstein, G., Grundhfer, A.: The Visual Computing of Projector-Camera Systems. CGF 27(8), 2219–2245 (2008)Google Scholar
  54. 54.
    Fasano, A., Callieri, M., Cignoni, P., Scopigno, R.: Exploiting mirrors for laser stripe 3d scanning. In: Proc. of 4th International Conference on 3D Digital Imaging and Modeling (3DIM 2003), Banff, Canada (2003)Google Scholar
  55. 55.
    Ihrke, I., Reshetouski, I., Manakov, A., Tevs, A., Wand, M., Seidel, H.P.: A Kaleidoscopic Approach to Surround Geometry and Reflectance Acquisition. In: Proceedings of IEEE International Workshop on Computational Cameras and Displays, pp. 29–36 (2012)Google Scholar
  56. 56.
    Fujii, K., Grossberg, M., Nayar, S.: A Projector-Camera System with Real-Time Photometric Adaptation for Dynamic Environments. In: Proc. CVPR, vol. 2, pp. 1180–1187 (2005)Google Scholar
  57. 57.
    Garg, G.G., Talvala, E.V., Levoy, M., Lensch, H.P.A.: Symmetric Photography: Exploiting Data-sparseness in Reflectance Fields. In: Proc. EGSR, pp. 251–262 (June 2006)Google Scholar
  58. 58.
    Ghosh, A., Achutha, S., Heidrich, W., O’Toole, M.: BRDF Acquisition with Basis Illumination. In: Proc. ICCV, pp. 183–197 (2007)Google Scholar
  59. 59.
    Ghosh, A., Heidrich, W., Achutha, S., O’Toole, M.: A Basis Illumination Approach to BRDF Measurement. IJCV 90(2), 183–197 (2010)CrossRefGoogle Scholar
  60. 60.
    Holroyd, M., Lawrence, J., Zickler, T.: A Coaxial Optical Scanner for Synchronous Acquisition of 3D Geometry and Surface Reflectance. ACM TOG 29(4), article no. 99 (2010)Google Scholar
  61. 61.
    Han, J.Y., Perlin, K.: Measuring Bidirectional Texture Reflectance with a Kaleidoscope. In: Proc. SIGGRAPH, pp. 741–748 (2003)Google Scholar
  62. 62.
    Bangay, S., Radloff, J.D.: Kaleidoscope Configurations for Reflectance Measurement. In: Proc. AFRIGRAPH, pp. 161–170 (2004)Google Scholar
  63. 63.
    Ding, Y., Xiao, J., Tan, K.H., Yu, J.: Catadioptric Projectors. In: Proc. CVPR, pp. 2528–2535 (2009)Google Scholar
  64. 64.
    Dana, K.: BRDF/BTF Measurement Device. In: Proc. ICCV, pp. 460–466 (2001)Google Scholar
  65. 65.
    Dana, K.J., Wang, J.: Device for Convenient Measurement of Spatially Varying Bidirectional Reflectance. JOSA 21, 1–12 (2004)CrossRefGoogle Scholar
  66. 66.
    Mukaigawa, Y., Sumino, K., Yagi, Y.: High-Speed Measurement of BDRF Using an Ellipsoidal Mirror and a Projector. In: Proc. CVPR, pp. 1–8 (2007)Google Scholar
  67. 67.
    Mukaigawa, Y., Sumino, K., Yagi, Y.: Multiplexed Illumination for Measuring BRDF Using an Ellipsoidal Mirror and a Projector. In: Yagi, Y., Kang, S.B., Kweon, I.S., Zha, H. (eds.) ACCV 2007, Part II. LNCS, vol. 4844, pp. 246–257. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  68. 68.
    Bouguet, J.Y.: Camera Calibration Toolbox for Matlab (2005)Google Scholar
  69. 69.
    Mariottini, G.L., Scheggi, S., Morbidi, F., Prattichizzo, D.: Planar Catadioptric Stereo: Single and Multi-View Geometry for Calibration and Localization. In: Proc. ICRA, pp. 2711–2716 (2009)Google Scholar
  70. 70.
    Ramalingam, S., Bouaziz, S., Sturm, P., Torr, P.H.: The Light-Path Less Traveled. In: Proc. CVPR, pp. 3145–3152 (2011)Google Scholar
  71. 71.
    Reshetouski, I., Manakov, A., Bandhari, A., Raskar, R., Seidel, H.P., Ihrke, I.: Discovering the Structure of a Planar Mirror System from Multiple Observations of a Single Point. In: Proc. CVPR, pp. 89–96 (2013)Google Scholar
  72. 72.
    Ihrke, I., Kutulakos, K.N., Lensch, H.P.A., Magnor, M., Heidrich, W.: Transparent and Specular Object Reconstruction. CGF 29(8), 2400–2426 (2010)Google Scholar
  73. 73.
    Sturm, P., Ramalingam, S., Tardif, J.P., Gasparini, S., Barreto, J.: Camera Models and Fundamental Concepts used in Geometric Computer Vision. In: Foundations and Trends in Computer Graphics and Vision, vol. 6, pp. 1–183. Now Publisher Inc. (2010)Google Scholar
  74. 74.
    Swaminathan, R., Grossberg, M.D., Nayar, S.K.: A Perspective on Distortions. In: Proc. CVPR, pp. 594–601 (2003)Google Scholar
  75. 75.
    Yu, J., McMillan, L., Sturnm, P.: Multi-Prespective Modelling, Rendering and Imaging. CGF 29(1), 227–246 (2010)Google Scholar
  76. 76.
    Swaminathan, R., Grossberg, M.D., Nayar, S.K.: Caustics of Catadioptric Cameras. In: Proc. ICCV, pp. 2–9 (2001)Google Scholar
  77. 77.
    Geyer, C., Danillidis, K.: Catadioptric Projective Geometry. IJCV 45(3), 223–243 (2001)CrossRefMATHGoogle Scholar
  78. 78.
    Geyer, C., Danillidis, K.: Paracatadioptric Camera Calibration. IEEE Trans. PAMI 24, 687–695 (2002)CrossRefGoogle Scholar
  79. 79.
    Agrawal, A., Taguchi, Y., Ramalingam, S.: Analytical Forward Projection for Axial Non-Central Dioptric & Catadioptric Cameras. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 129–143. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  80. 80.
    Agrawal, A., Taguchi, Y., Ramalingam, S.: Beyond Alhazen’s Problem: Analytical Projection Model for Non-Central Catadioptric Cameras with Quadric Mirrors. In: Proc. CVPR, pp. 2993–3000 (2011)Google Scholar
  81. 81.
    Geyer, C., Danillidis, K.: Catadioptric Camera Calibration. In: Proc. ICCV, vol. 1, pp. 398–404 (1999)Google Scholar
  82. 82.
    Ding, Y., Yu, J.: Recovering Shape Characteristics on Near-Flat Specular Surfaces. In: Proc. CVPR, pp. 1–8 (2008)Google Scholar
  83. 83.
    Ding, Y., Yu, J., Sturm, P.F.: Recovering Specular Surfaces using Curved Line Images. In: Proc. CVPR, pp. 2326–2333 (2009)Google Scholar
  84. 84.
    Tarini, M., Lensch, H.P.A., Goesele, M., Seidel, H.P.: 3D Acquisition of Mirroring Objects. Graphical Models 67(4), 233–259 (2005)CrossRefGoogle Scholar
  85. 85.
    Bonfort, T., Sturm, P., Gargallo, P.: General Specular Surface Triangulation. In: Narayanan, P.J., Nayar, S.K., Shum, H.-Y. (eds.) ACCV 2006. LNCS, vol. 3852, pp. 872–881. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  86. 86.
    Liu, M., Hartley, R., Salzmann, M.: Mirror Surface Reconstruction from a Single Image. In: Proc. CVPR, pp. 129–136 (June 2013)Google Scholar
  87. 87.
    Kang, S.B.: Catadioptric Self-Calibration. In: Proc. CVPR, pp. 201–207 (2000)Google Scholar
  88. 88.
    Svoboda, T., Pajdla, T., Hlaváč, V.: Epipolar Geometry for Panoramic Cameras. In: Burkhardt, H.-J., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1406, pp. 218–231. Springer, Heidelberg (1998)Google Scholar
  89. 89.
    Svoboda, T., Pajdla, T.: Epipolar Geometry for Central Catadioptric Cameras. IJCV 49(1), 23–37 (2002)CrossRefMATHGoogle Scholar
  90. 90.
    Agrawal, A., Ramalingam, S.: Single Image Calibration of Multi-Axial Imaging Systems. In: Proc. CVPR, pp. 1399–1406 (2013)Google Scholar
  91. 91.
    Kutulakos, K.N., Steger, E.: A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation. IJCV 76(1), 13–29 (2008)CrossRefGoogle Scholar
  92. 92.
    Scheuing, J., Yang, B.: Disambiguation of tdoa estimates in multi-path multi-source environments (datemm). IEEE ICASSP 4, 837–840 (2006)Google Scholar
  93. 93.
    Ajdler, T., Sbaiz, L., Vetterli, M.: The Plenacoustic Function and its Sampling. IEEE Transactions on Signal Processing 54(10), 3790–3804 (2006)CrossRefGoogle Scholar
  94. 94.
    Allen, J., Berkley, D.: Image method for efficiently simulating small-room acoustics. The Journal of the Acoustical Society of America 65(4), 943–950 (1979)CrossRefGoogle Scholar
  95. 95.
    Borish, J.: Extension of the Image Model to Arbitrary Polyhedra. Journal of the Acoustic Society of America 75(6), 1827–1836 (1984)CrossRefGoogle Scholar
  96. 96.
    Ribeiro, F., Florencio, D., Ba, D., Zhang, C.: Geometrically Constrained Room Modeling With Compact Microphone Arrays. IEEE Transactions on Audio, Speech, and Language Processing 20(5), 1449–1460 (2012)CrossRefGoogle Scholar
  97. 97.
    Antonacci, F., Filos, J., Thomas, M.R.P., Habets, E., Sarti, A., Naylor, P.A., Tubaro, S.: Inference of room geometry from acoustic impulse responses. IEEE Trans. on Audio, Speech and Language Processing 20(10), 2683–2695 (2012)CrossRefGoogle Scholar
  98. 98.
    Tervo, S., Tossavainen, T.: 3D Room Geometry Estimation from Measured Impulse Responses. In: Proc. ICASSP, pp. 513–516 (2012)Google Scholar
  99. 99.
    Kirmani, A., Hutchison, T., Davis, J., Raskar, R.: Looking around the Corner using Transient Imaging. In: Proc. ICCV, pp. 159–166 (2009)Google Scholar
  100. 100.
    Heide, F., Hullin, M., Gregson, J., Heidrich, W.: Low-Budget Transient Imaging using Photonic Mixer Devices. ACM TOG 32(4) (to appear, 2013)Google Scholar
  101. 101.
    Velten, A., Willwacher, T., Gupta, O., Veeraraghavan, A., Bawendi, M., Raskar, R.: Recovering Three-Dimensional Shape around a Corner using Ultrafast Time-of-Flight Imaging. Nat. Comm. 3, 745 (2012)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Ilya Reshetouski
    • 1
  • Ivo Ihrke
    • 1
  1. 1.Inria Bordeaux Sud-OuestFrance

Personalised recommendations