Stereoscopic oil paintings from RGBD images

  • Fay HuangEmail author
  • Bo-Ru Huang


Stroke-based rendering is one of the major approaches for creating synthetic paintings, but with only a minor attention so far to stereo painting synthesis. In this article, a fully automatic stereoscopic oil-painting synthesis algorithm is proposed, which takes a photograph and a depth map as input, and generates a pair of oil-painting style, stereo-viewable paintings. Common drawbacks of existing stroke-based rendering results are impressions of repetition and flatness due to the regularity of the used 2D stroke patterns. To reduce these impressions, the proposed approach introduces the concepts of a defocused image, a complexity map, a point map, and a direction map. Those maps serve as important references for decision making and thus, are the foundation for the entire painting simulation process. The key feature of making the developed stroke-based algorithm different from others is that it generates a unique 3D brushstroke according to the characteristics of a local image region. This has greatly reduced the undesirable machine-like appearance in the resulting image. Moreover, a comfortable stereo-viewing experience is assured by the proposed stereo painting and hole-filling strategies. Experimental results show that the proposed algorithm is applicable to a wide variety of image subjects and different depth distributions.


Stroke-based rendering Painterly rendering Stereo painting synthesis 



Special thanks to Prof. Reinhard Klette for valuable suggestions and critical comments. Thanks also to Dr. Dongwei Liu for support regarding the depth-map generation of images in Figs. 3 (pavilion) and 9 (mountain trail).

Compliance with Ethical Standards

Conflict of interests

This study was funded by the Ministry of Science and Technology, Taiwan (MOST 104-2221-E-197-020-MY2). Fay Huang is a member of Chinese Image Processing and Pattern Recognition Society (IPPR, Taiwan). She worked as a postdoctoral fellow at Institute of Information Science, Academic Sinica, Taiwan, from 2003 to 2004. She was also a consultant of Smart System Institute, Institute for Information Industry, Taiwan, in 2017. Bo-Ru Huang declares that he has no conflict of interest.


  1. 1.
    AKVIS ArtWork. [Software]. Available from
  2. 2.
    Bae S, Durand F (2007) Defocus magnification. Comput Graph Forum 26(3):571–579CrossRefGoogle Scholar
  3. 3.
    Benedetti L, Winnemöller H, Corsini M, Scopigno R (2014) Painting with Bob: assisted creativity for novices. In: Annual ACM symposium on user interface software technology, pp 419–428Google Scholar
  4. 4.
    Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6):679–698CrossRefGoogle Scholar
  5. 5.
    Chang W-H, Cheng M-C, Kuo C-M, Huang G-D (2015) Feature-oriented artistic styles transfer based on effective texture synthesis. J. Inf Hiding Multimed Signal Process 6(1):29–46Google Scholar
  6. 6.
    Chen X, Jin X, Zhao Q, Wu H (2012) Artistic illumination transfer for portraits. Comput Graph Forum 31(4):1425–1434CrossRefGoogle Scholar
  7. 7.
    Deep Art Mobile App. [Software]. Available from
  8. 8.
    DEPTHY. [Online software]. Available at
  9. 9.
    FotoSketcher. [Software]. Available from
  10. 10.
    Gatys LA, Ecker AS, Bethge M (2015) A neural algorithm of artistic style.: computing research repository (CoRR). arXiv:1508.06576
  11. 11.
    Gooch B, Coombe G, Shirley P (2002) Artistic vision: painterly rendering using computer vision techniques. In: International symposium on non-photorealistic animation rendering, pp 83–90Google Scholar
  12. 12.
    Guo C-E, Zhu S-C, Wu YN (2007) Primal sketch: integrating structure and texture. Comput Vis Image Underst 106(1):5–19CrossRefGoogle Scholar
  13. 13.
    Hertzmann A (1998) Painterly rendering with curved brush strokes of multiple sizes. In: International conference on computer graphics interactive techniques, pp 453–460Google Scholar
  14. 14.
    Howard I, Rogers B (1995) Binocular vision and stereopsis Oxford psychology series, vol 29. Oxford University Press, New YorkGoogle Scholar
  15. 15.
    Kang HW, Chui CK, Chakraborty UK (2006) A unified scheme for adaptive stroke-based rendering. Vis Comput 22(9):814–824CrossRefGoogle Scholar
  16. 16.
    Kim Y, Winnemöller H, Lee S (2014) WYSIWYG stereo painting with usability enhancements. IEEE Trans Vis Comput Graph 20(7):957–969CrossRefGoogle Scholar
  17. 17.
    Lee KJ, Kim DH, Yun ID, Lee SU (2007) Three-dimensional oil painting reconstruction with stroke-based rendering. Vis Comput 23(9):873–880CrossRefGoogle Scholar
  18. 18.
    Lee H, Seo S, Ryoo S, Yoon K (2011) Directional texture transfer with edge enhancement. Comput Graph 35(1):81–91CrossRefGoogle Scholar
  19. 19.
    Litwinowicz P (1997) Processing images and video for an impressionist effect. In: International conference on computer graphics interactive techniques, pp 407–414Google Scholar
  20. 20.
    Liu D (2016) Computer vision meets arts: stereo computational photography. Ph.D. dissertation, Dept. Comp. Sci., The Univ. of Auckland, New ZealandGoogle Scholar
  21. 21.
    Meier B J (1996) Painterly rendering for animation. In: International conference on computer graphics interactive techniques, pp 477–484Google Scholar
  22. 22.
    Northam L, Asente P, Kaplan CS (2012) Consistent stylization and painterly rendering of stereoscopic 3D images. In: Proceedings of the symposium on non-photorealistic animation rendering, pp 47–56Google Scholar
  23. 23.
    Shiraishi M, Yamaguchi Y (2000) An algorithm for automatic painterly rendering based on local source image approximation. In: International symposium on non-photorealistic animation rendering, pp 53–58Google Scholar
  24. 24.
    Stavrakis E (2008) Stereoscopic non-photorealistic rendering . Ph.D. dissertation, Vienna Univ. of Tech., AustriaGoogle Scholar
  25. 25.
    Wang B, Wang W, Yang H, Sun J (2004) Efficient example-based painting and synthesis of 2D directional texture. IEEE Trans Vis Comput Graph 10(3):266–277CrossRefGoogle Scholar
  26. 26.
    Ware C, Gobrecht C, Paton M (1998) Dynamic adjustment of stereo display parameters. IEEE Trans Syst Man Cybern 28:56–65CrossRefGoogle Scholar
  27. 27.
    Zeng K, Zhao M, Xiong C, Zhu S-C (2009) From image parsing to painterly rendering. ACM Trans Graph 29(1):2:1–2:11CrossRefGoogle Scholar
  28. 28.
    Zhang W, Cham W-K (2012) Single-image refocusing and defocusing. IEEE Trans Image Process 21(2):873–882MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer Science and Information EngineeringNational Ilan UniversityYilanTaiwan

Personalised recommendations