Machine Vision and Applications

, Volume 27, Issue 8, pp 1325–1337 | Cite as

Stereo-based bokeh effects for photography

Special Issue Paper
  • 254 Downloads

Abstract

Bokeh, a sought-after photo rendering style of out-of-focus blur, typically aims at an esthetic quality which is not available to low-end consumer-grade cameras due to the lens design. We present a bokeh simulation method using stereo-vision techniques. We refine a depth map obtained by stereo matching, also using some minor user interaction. Overexposed regions are recovered according to depth information. A depth-aware bokeh effect is then applied with user-adjustable apertures sizes or shapes. We also simulate swirly bokeh, also known as cat-eye effect. Our method mainly aims at the visual quality of the bokeh effect rather than (so far) at time efficiency. Experiments show that our results are natural looking and that they can be comparable to bokeh effects achieved with expensive real-world bokeh-capable camera systems.

Keywords

Bokeh Stereo vision Computational photography 

References

  1. 1.
    Abbott, J., Morse, B.: Interactive depth-aware effects for stereo image editing. In: Proceedings of the 3D vision, pp. 263–270 (2013)Google Scholar
  2. 2.
    Demers, J.: Depth of field: a survey of techniques. GPU Gems 1(375), U390 (2004)Google Scholar
  3. 3.
    He, S.: How to customize heart bokeh (aperture) for the camera? DIY tutorial. www.youtube.com/watch?v=he_RUMxo-H8 (2013)
  4. 4.
    Hirschmüller, H.: Accurate and efficient stereo processing by semi-global matching and mutual information. Proc. Comput. Vis. Pattern Recogn. 2, 807–814 (2005)Google Scholar
  5. 5.
    Kopf, J., Cohen, M.F., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. ACM Trans. Graph. 26, 96 (2007)CrossRefGoogle Scholar
  6. 6.
    Klette, R.: Concise Computer Vision: An Introduction into Theory and Algorithms. Springer, London (2014)CrossRefMATHGoogle Scholar
  7. 7.
    Klette, R., Rosenfeld, A.: Digital Geometry: Geometric Methods for Digital Picture Analysis. Morgan Kaufmann, San Francisco (2004)MATHGoogle Scholar
  8. 8.
    Kraus, M., Strengert, M.: Depth-of-field rendering by pyramidal image processing. Comput. Graph. Forum 26, 645–654 (2007)CrossRefGoogle Scholar
  9. 9.
    Lee, S., Eisemann, E., Seidel, H.-P.: Depth-of-field rendering with multiview synthesis. ACM Trans. Graph. 28, 134 (2009)Google Scholar
  10. 10.
    Lee, S., Eisemann, E., Seidel, H.-P.: Real-time lens blur effects and focus control. ACM Trans. Graph. 29, 65 (2010)Google Scholar
  11. 11.
    Lee, S., Jounghyun Kim, G., Choi, S.: Real-time depth-of-field rendering using anisotropically filtered mipmap interpolation. IEEE Trans. Vis. Comput. Graph. 15, 453–464 (2009)CrossRefGoogle Scholar
  12. 12.
    Lee, S., Kim, G.J., Choi, S.: Real-time depth-of-field rendering using point splatting on per-pixel layers. Comput. Graph. Forum 27, 1955–1962 (2008)CrossRefGoogle Scholar
  13. 13.
    Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 30, 228–242 (2008)CrossRefGoogle Scholar
  14. 14.
    Liu, J., Gong, X., Liu, J.: Guided inpainting and filtering for Kinect depth maps. In: Proceedings of International Conference on Pattern Recognition, pp. 2055–2058 (2012)Google Scholar
  15. 15.
    Liu, D., Nicolescu, R., Klette, R.: Bokeh effects based on stereo vision. In: Proceedings of Computer Analysis of Images and Patterns, pp. 198–210 (2015)Google Scholar
  16. 16.
    Liu, S., Lai, P., Tian, D., Gomila, C., Chen, C.W.: Joint trilateral filtering for depth map compression. In: Proceedings of Visual Communications and Image Processing, No. 77440F (2010)Google Scholar
  17. 17.
    Matsuo, T., Fukushima, N., Ishibashi, Y.: Weighted joint bilateral filter with slope depth compensation filter for depth map refinement. In: Proceedings of International Conference Computer Vision Theory and Applications, pp. 300–309 (2013)Google Scholar
  18. 18.
    Mowery, A.: Create the famed Petzval swirly bokeh with your lens for cheap. www.diyphotography.net/create-famed-petzval-swirly-bokeh-lens-cheap/ (2015)
  19. 19.
    Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K.: Digital photography with flash and no-flash image pairs. ACM Trans. Graph. 23, 664–672 (2004)CrossRefGoogle Scholar
  20. 20.
    Rockwell, K.: Nikon Lens Bokeh Comparison. www.kenrockwell.com/nikon/bokeh-comparison.htm (2006)
  21. 21.
    Rockwell, K.: Bokeh. www.kenrockwell.com/tech/bokeh.htm (2008)
  22. 22.
    Wang, Q., Yu, Z., Rasmussen, C., Yu, J.: Stereo vision-based depth of field rendering on a mobile device. J. Electron. Imaging 23, 023009 (2014)CrossRefGoogle Scholar
  23. 23.
    Wu, J., Zheng, C., Hu, X., Wang, Y., Zhang, L.: Realistic rendering of bokeh effect based on optical aberrations. Vis. Comput. 26, 555–563 (2010)CrossRefGoogle Scholar
  24. 24.
    Xue, W., Zhang, X., Sheng, B., Ma, L.: Image-based depth-of-field rendering with non-local means filtering. In: Proceedings of International Conference Multimedia Expo Workshops, pp. 1–6 (2013)Google Scholar
  25. 25.
    Yu, X., Wang, R., Yu, J.: Real-time depth of field rendering via dynamic light field generation and filtering. Comput. Graph. Forum 29, 2099–2107 (2010)CrossRefGoogle Scholar
  26. 26.
    Yu, Z., Yu, X., Thorpe, C., Grauer-Gray, S., Li, F., Yu, J.: Racking focus and tracking focus on live video streams: a stereo solution. Vis. Comput. 30, 45–58 (2014)CrossRefGoogle Scholar
  27. 27.
    Zhang, W., Cham, W.-K.: Single image focus editing. In: Proceedings of International Conference Computer Vision Workshops, pp. 1947–1954 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.The University of AucklandAucklandNew Zealand
  2. 2.Auckland University of TechnologyAucklandNew Zealand

Personalised recommendations