Abstract
Photographing scenes with high dynamic range (HDR) poses great challenges to consumer cameras with their limited sensor bit depth. To address this, Zhao et al. recently proposed a novel sensor concept – the modulo camera – which captures the least significant bits of the recorded scene instead of going into saturation. Similar to conventional pipelines, HDR images can be reconstructed from multiple exposures, but significantly fewer images are needed than with a typical saturating sensor. While the concept is appealing, we show that the original reconstruction approach assumes noise-free measurements and quickly breaks down otherwise. To address this, we propose a novel reconstruction algorithm that is robust to image noise and produces significantly fewer artifacts. We theoretically analyze correctness as well as limitations, and show that our approach significantly outperforms the baseline on real data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Böhm, M., Blecher, F., Eckhardt, A., Schneider, B., Benthien, S., Keller, H., Lulé, T., Rieve, P., Sommer, M., Lind, R.C., Humm, L., Daniels, M., Wu, N., Yen, H., Efron, U.: High dynamic range image sensors in thin film on ASIC technology for automotive applications. In: Ricken, D.E., Gessner, W. (eds.) Advanced Microsystems for Automotive Applications, pp. 157–172. Springer, Berlin (1998). (Jan)
Chavez, S., Xiang, Q.S., An, L.: Understanding phase maps in MRI: a new cutline phase unwrapping method. IEEE Trans. Med. Imaging 21, 966–977 (2002)
Cordts, M., Omran, M., Ramos, S., Scharwächter, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., Schiele, B.: The Cityscapes dataset for semantic urban scene understanding. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016
Debevec, P.E., Malik, J.: Recovering high dynamic range radiance maps from photographs. In: ACM Transactions Graphics (Proceedings of ACM SIGGRAPH), pp. 369–378, August 1997
Foi, A., Trimeche, M., Katkovnik, V., Egiazarian, K.: Practical poissonian-Gaussian noise modeling and fitting for single-image raw-data. IEEE Trans. Image Process. 17(10), 1737–1754 (2008)
Fuji Photo Film Co., Ltd: 4th-generation Super CCD (2003). http://www.fujifilmusa.com/shared/bin/4thGenSUPERCCDBrochure.pdf
Gallo, O., Gelfand, N., Chen, W., Tico, M., Pulli, K.: Artifact-free high dynamic range imaging. IEEE International Conference on Computer Photography (ICCP), April 2009
Goldstein, R.M., Zebker, H.A., Werner, C.L.: Satellite radar interferometry: two-dimensional phase unwrapping. Radio Sci. 23(4), 713–720 (1988)
Granados, M., Ajdin, B., Wand, M., Theobalt, C., Seidel, H.P., Lensch, H.P.A.: Optimal HDR reconstruction with linear digital cameras. In: IEEE Conference on Computer Vision and Pattern Recogion (CVPR), pp. 215–222, June 2010
Grossberg, M.D., Nayar, S.K.: High dynamic range from multiple images: which exposures to combine?. In: ICCV Workshop on Color and Photometric Methods in Computer Vision (CPMCV), October 2003
Heo, Y.S., Lee, K.M., Lee, S.U., Moon, Y., Cha, J.: Ghost-free high dynamic range imaging. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010. LNCS, vol. 6495, pp. 486–500. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-19282-1_39
Hoefflinger, B. (ed.): High-Dynamic-Range (HDR) Vision. Springer, Heidelberg (2007)
Jacobs, K., Loscos, C., Ward, G.: Automatic high-dynamic range image generation for dynamic scenes. IEEE Comput. Graph. Appl. 28(2), 84–93 (2008)
Loose, M., Meier, K., Schemmel, J.: A self-calibrating single-chip CMOS camera with logarithmic response. IEEE J. Solid-State Circuits 36(4), 586–596 (2001)
Mäkitalo, M., Foi, A.: Noise parameter mismatch in variance stabilization, with an application to Poisson-Gaussian noise estimation. IEEE Trans. Image Process. 23(12), 5348–5359 (2014)
Mann, S., Picard, R.W.: On being ‘undigital’ with digital cameras: extending dynamic range by combining differently exposed pictures. In: Proceedings of Imaging Science and Technology, pp. 442–448, May 1995
Mantiuk, R.K., Myszkowski, K., Seidel, H.P.: High dynamic range imaging. In: Wiley Encyclopedia of Electrical and Electronics Engineering. Wiley (2015)
Nayar, S.K., Mitsunaga, T.: High dynamic range imaging: spatially varying pixel exposures. In: IEEE Conference on Computer Vision and Pattern Recogition (CVPR), June 2000
Robertson, M.A., Borman, S., Stevenson, R.L.: Estimation-theoretic approach to dynamic range enhancement using multiple exposures. Electron. Imaging 12(2), 219–228 (2003)
Stumpfel, J., Tchou, C., Jones, A., Hawkins, T., Wenger, A., Debevec, P.: Direct HDR capture of the sun and sky. In: Proceedings of the 3rd International Conference on Computer Graphics, Virtual Reality, Visualisation and Interact, in Africa. pp. 145–149. AFRIGRAPH 2004, November 2004
Tumblin, J., Agrawal, A., Raskar, R.: Why I want a gradient camera. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 103–110, June 2014
Zhao, H., Shi, B., Fernandez-Cull, C., Yeung, S.K., Raskar, R.: Unbounded high dynamic range photography using a modulo camera. In: International Conference on Computational Photography (ICCP), April 2015
Acknowledgments
The research leading to these results has received funding from the European Research Council under the European Union’s Seventh Framework Programme (FP/2007–2013)/ERC Grant agreement No. 307942.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Lang, F., Plötz, T., Roth, S. (2017). Robust Multi-image HDR Reconstruction for the Modulo Camera. In: Roth, V., Vetter, T. (eds) Pattern Recognition. GCPR 2017. Lecture Notes in Computer Science(), vol 10496. Springer, Cham. https://doi.org/10.1007/978-3-319-66709-6_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-66709-6_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-66708-9
Online ISBN: 978-3-319-66709-6
eBook Packages: Computer ScienceComputer Science (R0)