Advertisement

Stereoscopic oil paintings from RGBD images

  • Fay HuangEmail author
  • Bo-Ru Huang
Article
  • 5 Downloads

Abstract

Stroke-based rendering is one of the major approaches for creating synthetic paintings, but with only a minor attention so far to stereo painting synthesis. In this article, a fully automatic stereoscopic oil-painting synthesis algorithm is proposed, which takes a photograph and a depth map as input, and generates a pair of oil-painting style, stereo-viewable paintings. Common drawbacks of existing stroke-based rendering results are impressions of repetition and flatness due to the regularity of the used 2D stroke patterns. To reduce these impressions, the proposed approach introduces the concepts of a defocused image, a complexity map, a point map, and a direction map. Those maps serve as important references for decision making and thus, are the foundation for the entire painting simulation process. The key feature of making the developed stroke-based algorithm different from others is that it generates a unique 3D brushstroke according to the characteristics of a local image region. This has greatly reduced the undesirable machine-like appearance in the resulting image. Moreover, a comfortable stereo-viewing experience is assured by the proposed stereo painting and hole-filling strategies. Experimental results show that the proposed algorithm is applicable to a wide variety of image subjects and different depth distributions.

Keywords

Stroke-based rendering Painterly rendering Stereo painting synthesis 

Notes

Acknowledgements

Special thanks to Prof. Reinhard Klette for valuable suggestions and critical comments. Thanks also to Dr. Dongwei Liu for support regarding the depth-map generation of images in Figs. 3 (pavilion) and 9 (mountain trail).

Compliance with Ethical Standards

Conflict of interests

This study was funded by the Ministry of Science and Technology, Taiwan (MOST 104-2221-E-197-020-MY2). Fay Huang is a member of Chinese Image Processing and Pattern Recognition Society (IPPR, Taiwan). She worked as a postdoctoral fellow at Institute of Information Science, Academic Sinica, Taiwan, from 2003 to 2004. She was also a consultant of Smart System Institute, Institute for Information Industry, Taiwan, in 2017. Bo-Ru Huang declares that he has no conflict of interest.

References

  1. 1.
    AKVIS ArtWork. [Software]. Available from akvis.com/en/artwork/
  2. 2.
    Bae S, Durand F (2007) Defocus magnification. Comput Graph Forum 26(3):571–579CrossRefGoogle Scholar
  3. 3.
    Benedetti L, Winnemöller H, Corsini M, Scopigno R (2014) Painting with Bob: assisted creativity for novices. In: Annual ACM symposium on user interface software technology, pp 419–428Google Scholar
  4. 4.
    Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6):679–698CrossRefGoogle Scholar
  5. 5.
    Chang W-H, Cheng M-C, Kuo C-M, Huang G-D (2015) Feature-oriented artistic styles transfer based on effective texture synthesis. J. Inf Hiding Multimed Signal Process 6(1):29–46Google Scholar
  6. 6.
    Chen X, Jin X, Zhao Q, Wu H (2012) Artistic illumination transfer for portraits. Comput Graph Forum 31(4):1425–1434CrossRefGoogle Scholar
  7. 7.
    Deep Art Mobile App. [Software]. Available from deeparteffects.com/
  8. 8.
    DEPTHY. [Online software]. Available at depthy.me
  9. 9.
    FotoSketcher. [Software]. Available from fotosketcher.com/
  10. 10.
    Gatys LA, Ecker AS, Bethge M (2015) A neural algorithm of artistic style.: computing research repository (CoRR). arXiv:1508.06576
  11. 11.
    Gooch B, Coombe G, Shirley P (2002) Artistic vision: painterly rendering using computer vision techniques. In: International symposium on non-photorealistic animation rendering, pp 83–90Google Scholar
  12. 12.
    Guo C-E, Zhu S-C, Wu YN (2007) Primal sketch: integrating structure and texture. Comput Vis Image Underst 106(1):5–19CrossRefGoogle Scholar
  13. 13.
    Hertzmann A (1998) Painterly rendering with curved brush strokes of multiple sizes. In: International conference on computer graphics interactive techniques, pp 453–460Google Scholar
  14. 14.
    Howard I, Rogers B (1995) Binocular vision and stereopsis Oxford psychology series, vol 29. Oxford University Press, New YorkGoogle Scholar
  15. 15.
    Kang HW, Chui CK, Chakraborty UK (2006) A unified scheme for adaptive stroke-based rendering. Vis Comput 22(9):814–824CrossRefGoogle Scholar
  16. 16.
    Kim Y, Winnemöller H, Lee S (2014) WYSIWYG stereo painting with usability enhancements. IEEE Trans Vis Comput Graph 20(7):957–969CrossRefGoogle Scholar
  17. 17.
    Lee KJ, Kim DH, Yun ID, Lee SU (2007) Three-dimensional oil painting reconstruction with stroke-based rendering. Vis Comput 23(9):873–880CrossRefGoogle Scholar
  18. 18.
    Lee H, Seo S, Ryoo S, Yoon K (2011) Directional texture transfer with edge enhancement. Comput Graph 35(1):81–91CrossRefGoogle Scholar
  19. 19.
    Litwinowicz P (1997) Processing images and video for an impressionist effect. In: International conference on computer graphics interactive techniques, pp 407–414Google Scholar
  20. 20.
    Liu D (2016) Computer vision meets arts: stereo computational photography. Ph.D. dissertation, Dept. Comp. Sci., The Univ. of Auckland, New ZealandGoogle Scholar
  21. 21.
    Meier B J (1996) Painterly rendering for animation. In: International conference on computer graphics interactive techniques, pp 477–484Google Scholar
  22. 22.
    Northam L, Asente P, Kaplan CS (2012) Consistent stylization and painterly rendering of stereoscopic 3D images. In: Proceedings of the symposium on non-photorealistic animation rendering, pp 47–56Google Scholar
  23. 23.
    Shiraishi M, Yamaguchi Y (2000) An algorithm for automatic painterly rendering based on local source image approximation. In: International symposium on non-photorealistic animation rendering, pp 53–58Google Scholar
  24. 24.
    Stavrakis E (2008) Stereoscopic non-photorealistic rendering . Ph.D. dissertation, Vienna Univ. of Tech., AustriaGoogle Scholar
  25. 25.
    Wang B, Wang W, Yang H, Sun J (2004) Efficient example-based painting and synthesis of 2D directional texture. IEEE Trans Vis Comput Graph 10(3):266–277CrossRefGoogle Scholar
  26. 26.
    Ware C, Gobrecht C, Paton M (1998) Dynamic adjustment of stereo display parameters. IEEE Trans Syst Man Cybern 28:56–65CrossRefGoogle Scholar
  27. 27.
    Zeng K, Zhao M, Xiong C, Zhu S-C (2009) From image parsing to painterly rendering. ACM Trans Graph 29(1):2:1–2:11CrossRefGoogle Scholar
  28. 28.
    Zhang W, Cham W-K (2012) Single-image refocusing and defocusing. IEEE Trans Image Process 21(2):873–882MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer Science and Information EngineeringNational Ilan UniversityYilanTaiwan

Personalised recommendations