Advertisement

The Visual Computer

, Volume 27, Issue 12, pp 1099–1114 | Cite as

Reconstruction of high contrast images for dynamic scenes

  • Shanmuganathan Raman
  • Subhasis Chaudhuri
Original Article

Abstract

High Dynamic Range (HDR) imaging requires one to composite multiple, differently exposed images of a scene in the irradiance domain and perform tone mapping of the generated HDR image for displaying on Low Dynamic Range (LDR) devices. In the case of dynamic scenes, standard techniques may introduce artifacts called ghosts if the scene changes are not accounted for. In this paper, we consider the blind HDR problem for dynamic scenes. We develop a novel bottom-up segmentation algorithm through superpixel grouping which enables us to detect scene changes. We then employ a piecewise patch-based compositing methodology in the gradient domain to directly generate the ghost-free LDR image of the dynamic scene. Being a blind method, the primary advantage of our approach is that we do not assume any knowledge of camera response function and exposure settings while preserving the contrast even in the non-stationary regions of the scene. We compare the results of our approach for both static and dynamic scenes with that of the state-of-the-art techniques.

Keywords

High dynamic range imaging De-ghosting Low dynamic range image generation Computational photography 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Adobe Photoshop CS5 User Guide. Adobe Systems Incorporated, San Jose (2010) Google Scholar
  2. 2.
    Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., Cohen, M.: Interactive digital photomontage. ACM Trans. Graph. 23, 294–302 (2004) CrossRefGoogle Scholar
  3. 3.
    Agrawal, A., Raskar, R.: What is the range of surface reconstructions from a gradient field. In: ECCV, pp. 578–591. Springer, Graz (2006) Google Scholar
  4. 4.
    Ashikhmin, M.: A tone mapping algorithm for high contrast images. In: Proceedings of the 13th Eurographics Workshop on Rendering, pp. 145–156. Eurographics Association, Pisa (2002) Google Scholar
  5. 5.
    Aydin, T.O., Mantiuk, R., Myszkowski, K., Seidel, H.P.: Dynamic range independent image quality assessment. ACM Trans. Graph. 27, 69:1–69:10 (2008) CrossRefGoogle Scholar
  6. 6.
    Banterle, F., Ledda, P., Debattista, K., Chalmers, A.: Inverse tone mapping. In: GRAPHITE ’06: Proceedings of the 4th International Conference on Computer Graphics and Interactive Techniques in Australasia and Southeast Asia, pp. 349–356. ACM, Kuala Lumpur (2006) CrossRefGoogle Scholar
  7. 7.
    Bogart, R., Kainz, F., Hess, D.: The openexr file format. In: SIGGRAPH Technical Sketch, San Diego, USA (2003) Google Scholar
  8. 8.
    Borenstein, E., Ullman, S.: Class-specific, top-down segmentation. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) Computer Vision ECCV 2002. Lecture Notes in Computer Science, vol. 2351, pp. 639–641. Springer, Berlin (2002) Google Scholar
  9. 9.
    Borenstein, E., Ullman, S.: Learning to segment. In: Computer Vision-ECCV, pp. 315–328. Springer, Prague (2004) CrossRefGoogle Scholar
  10. 10.
    Brinkmann, R.: The Art and Science of Digital Compositing. Morgan Kaufmann, San Mateo (1999) Google Scholar
  11. 11.
    Debevec, P.E., Malik, J.: Recovering high dynamic range radiance maps from photographs. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’97, pp. 369–378. ACM, Los Angeles (1997) CrossRefGoogle Scholar
  12. 12.
    Drago, F., Myszkowski, K., Annen, T., Chiba, N.: Adaptive logarithmic mapping for displaying high contrast scenes. Comput. Graph. Forum 22(3), 419–426 (2003) CrossRefGoogle Scholar
  13. 13.
    Durand, F., Dorsey, J.: Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans. Graph. 21, 257–266 (2002) Google Scholar
  14. 14.
    Fattal, R., Lischinski, D., Werman, M.: Gradient domain high dynamic range compression. ACM Trans. Graph. 21, 249–256 (2002) CrossRefGoogle Scholar
  15. 15.
    Fulkerson, B., Vedaldi, A., Soatto, S.: Class segmentation and object localization with superpixel neighborhoods. In: Computer Vision, 2009 IEEE 12th International Conference on, pp. 670–677. IEEE, Kyoto (2010) Google Scholar
  16. 16.
    Gallo, O., Gelfand, N., Chen, W., Tico, M., Pulli, K.: Artifact-free high dynamic range imaging. In: IEEE International Conference on Computational Photography (ICCP), San Francisco, CA, USA (2009) Google Scholar
  17. 17.
    Goshtasby, A.: Fusion of multi-exposure images. Image Vis. Comput. 23(6), 611–618 (2005) CrossRefGoogle Scholar
  18. 18.
    Granados, M., Ajdin, B., Wand, M., Theobalt, C., Seidel, H., Lensch, H.: Optimal HDR reconstruction with linear digital cameras. In: Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pp. 215–222. IEEE, San Francisco (2010) CrossRefGoogle Scholar
  19. 19.
    Grossberg, M., Nayar, S.: Determining the camera response from images: what is knowable? IEEE Trans. Pattern Anal. Mach. Intell. 25(11), 1455–1467 (2003) CrossRefGoogle Scholar
  20. 20.
    Guo, D., Cheng, Y., Zhuo, S., Sim, T.: Correcting over-exposure in photographs. In: Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, San Francisco, CA, USA, pp. 515–521 (2010). doi: 10.1109/CVPR.2010.5540170 CrossRefGoogle Scholar
  21. 21.
    Hoiem, D., Efros, A.A., Hebert, M.: Automatic photo pop-up. ACM Trans. Graph. 24, 577–584 (2005) CrossRefGoogle Scholar
  22. 22.
    Jacobs, K., Loscos, C., Ward, G.: Automatic high-dynamic range image generation for dynamic scenes. IEEE Comput. Graph. Appl. 28(2), 84–93 (2008) CrossRefGoogle Scholar
  23. 23.
    Khan, E., Akyuz, A., Reinhard, E.: Ghost removal in high dynamic range images. In: Image Processing, 2006 IEEE International Conference on, Atlanta, GA, USA, pp. 2005–2008 (2006) Google Scholar
  24. 24.
    Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 30, 228–242 (2008) CrossRefGoogle Scholar
  25. 25.
    Levinshtein, A., Stere, A., Kutulakos, K., Fleet, D., Dickinson, S., Siddiqi, K.: Turbopixels: Fast superpixels using geometric flows. IEEE Trans. Pattern Anal. Mach. Intell. 31(12), 2290–2297 (2009) CrossRefGoogle Scholar
  26. 26.
    Lin, S., Gu, J., Yamazaki, S., Shum, H.Y.: Radiometric calibration from a single image. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR’04, pp. 938–945. IEEE Computer Society, Washington (2004) CrossRefGoogle Scholar
  27. 27.
    Lischinski, D., Farbman, Z., Uyttendaele, M., Szeliski, R.: Interactive local adjustment of tonal values. ACM Trans. Graph. 25, 646–653 (2006) CrossRefGoogle Scholar
  28. 28.
    Mann, S.: Comparametric equations with practical applications in quantigraphic image processing. IEEE Trans. Image Process. 9(8), 1389–1406 (2000) MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Mann, S., Picard, R.W.: On being undigital with digital cameras: Extending dynamic range by combining exposed pictures. In: Proc. of IS & T 48th Annual Conference, pp. 422–428 (1995) Google Scholar
  30. 30.
    Mantiuk, R., Seidel, H.: Modeling a generic tone-mapping operator. Comput. Graph. Forum 27(2), 699–708 (2008) CrossRefGoogle Scholar
  31. 31.
    Mantiuk, R., Myszkowski, K., Seidel, H.P.: A perceptual framework for contrast processing of high dynamic range images. ACM Trans. Appl. Percept. 3, 286–308 (2006) CrossRefGoogle Scholar
  32. 32.
    Mantiuk, R., Krawczyk, G., Mantiuk, R., Seidel, H.P.: High dynamic range imaging pipeline: Perception-motivated representation of visual content. In: Rogowitz, B.E., Pappas, T.N., Daly, S.J. (eds.) Human Vision and Electronic Imaging XII. Proceedings of SPIE. SPIE, San Jose (2007) Google Scholar
  33. 33.
    Masood, S., Zhu, J., Tappen, M.: Automatic correction of saturated regions in photographs using cross-channel correlation. Comput. Graph. Forum 28(7), 1861–1869 (2009) CrossRefGoogle Scholar
  34. 34.
    Mertens, T., Kautz, J., Reeth, F.V.: Exposure fusion: A simple and practical alternative to high dynamic range photography. Comput. Graph. Forum 28(1), 161–171 (2009) CrossRefGoogle Scholar
  35. 35.
    Mitsunaga, T., Nayar, S.: Radiometric self calibration. In: Computer Vision and Pattern Recognition, 1999. CVPR 1999. Proceedings of the 1999 IEEE Computer Society Conference on, Ft. Collins, CO, USA, p. 1374 (1999) Google Scholar
  36. 36.
    Mori, G.: Guiding model search using segmentation. In: Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on, Beijing, China, vol. 2, pp. 1417–1423 (2005) Google Scholar
  37. 37.
    Mori, G., Ren, X., Efros, A., Malik, J.: Recovering human body configurations: Combining segmentation and recognition. In: Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, Washington, DC, USA, vol. 2, pp. II-326–II-333 (2004) Google Scholar
  38. 38.
    Pattanaik, S.N., Tumblin, J., Yee, H., Greenberg, D.P.: Time-dependent visual adaptation for fast realistic image display. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’00, pp. 47–54. ACM, New York (2000) CrossRefGoogle Scholar
  39. 39.
    Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. 22, 313–318 (2003) CrossRefGoogle Scholar
  40. 40.
    Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12(7), 629–639 (1990) CrossRefGoogle Scholar
  41. 41.
    Porter, T., Duff, T.: Compositing digital images. SIGGRAPH Comput. Graph. 18, 253–259 (1984) CrossRefGoogle Scholar
  42. 42.
    Raman, S., Chaudhuri, S.: A matte-less, variational approach to automatic scene compositing. In: Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on, Rio de Janeiro, Brazil, pp. 1–6 (2007) Google Scholar
  43. 43.
    Raman, S., Chaudhuri, S.: Bilateral filter based compositing for variable exposure photography. In: EUROGRAPHICS Short Papers, Munich, Germany (2009) Google Scholar
  44. 44.
    Raman, S., Chaudhuri, S.: Bottom-up segmentation for ghost-free reconstruction of a dynamic scene from multi-exposure images. In: Proceedings of the Seventh Indian Conference on Computer Vision, Graphics and Image Processing, ICVGIP ’10, pp. 56–63. ACM, Chennai (2010) CrossRefGoogle Scholar
  45. 45.
    Reinhard, E., Heidrich, W., Pattanaik, S., Debevec, P., Ward, G., Myszkowski, K.: High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting. Morgan Kaufmann Series in Computer Graphics. Elsevier, Amsterdam (2010) Google Scholar
  46. 46.
    Reinhard, E., Devlin, K.: Dynamic range reduction inspired by photoreceptor physiology. IEEE Trans. Vis. Comput. Graph. 11(1), 13–24 (2005) CrossRefGoogle Scholar
  47. 47.
    Reinhard, E., Stark, M., Shirley, P., Ferwerda, J.: Photographic tone reproduction for digital images. ACM Trans. Graph. 21, 267–276 (2002) CrossRefGoogle Scholar
  48. 48.
    Ren, X., Malik, J.: Learning a classification model for segmentation. In: Computer Vision, 2003. Proceedings. Ninth IEEE International Conference on, Nice, France, vol. 1, pp. 10–17 (2003) CrossRefGoogle Scholar
  49. 49.
    Rother, C., Kolmogorov, V., Blake, A.: “Grabcut”: Interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23, 309–314 (2004) CrossRefGoogle Scholar
  50. 50.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000) CrossRefGoogle Scholar
  51. 51.
    Spheron: Spherocam HDR. http://www.spheron.com/
  52. 52.
    Szeliski, R.: Image alignment and stitching: A tutorial. Found. Trends Comput. Graph. Vis. 2(1) (2008) Google Scholar
  53. 53.
    Uyttendaele, M., Eden, A., Skeliski, R.: Eliminating ghosting and exposure artifacts in image mosaics. In: Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, Kauai, HI, USA, vol. 2, pp. II-509–II-516 (2001) CrossRefGoogle Scholar
  54. 54.
    Vedaldi, A., Soatto, S.: Quick shift and kernel methods for mode seeking. In: Proceedings of the European Conference on Computer Vision (ECCV), Marseille, France (2008) Google Scholar
  55. 55.
    Ward, G.: Fast robust image registration for compositing high dynamic range photographcs from hand-held exposures. J. Graph. Tools 8(2), 17–30 (2003) Google Scholar
  56. 56.
    Ward, G.J.: The radiance lighting simulation and rendering system. In: Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’94, pp. 459–472. ACM, New York (1994) CrossRefGoogle Scholar
  57. 57.
    Zhang, W., Cham, W.K.: Gradient-directed composition of multi-exposure images. In: Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, San Francisco, CA, USA, pp. 530–536 (2010) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  1. 1.Department of Electrical EngineeringIndian Institute of Technology, BombayPowaiIndia

Personalised recommendations