Abstract
Artistic video stylization, which is widely used in multimedia entertainment, transforms a given video into different artistic styles. Most of the existing video stylization algorithms can simulate single or limited video artistic styles. Although some algorithms can achieve multi-style video processing, these algorithms are complex and difficult to implement. To solve this problem, we propose a multi-styled video stylization algorithm based on texture advection, where different artistic styles are synthesized and transferred from user-specified texture samples of desired styles. We use the direction field-guided texture synthesis to compute the texture layer that represents the artistic style. Painterly directional video styles are simulated competently by the orientation changes in the synthesized anisotropic textures. There appeared local distorted region of the texture layer during texture advection under the optical flow field. To address this issue, we propose the texture inpaint to synthesize the limited distorted region and make the stylized video temporally coherent. We also accelerate the video stylization by using the CUDA parallel computing framework that parallelly computes the morphological operations used for video abstraction. Finally, we produce stylized videos of multiple artistic styles with satisfactory experimental results, including the styles of oil painting, watercolor painting and stylized lines drawing.
摘要
创新点
本文采用基于变化方向场的纹理合成算法得到不同风格的纹理层, 通过传输不同的样本纹理获取不同的艺术风格。本文进一步采取局部纹理修补技术减少基于光流场传输的纹理层出现的拉伸走样现象, 使得不同帧之间的纹理层平滑衔接, 保证视频艺术化后的时域连续性。根据输入样本纹理的风格, 本文模拟了油画、水彩画、点画以及风格化线条等多种视频艺术化效果, 均取得了令人满意的效果。
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Hays J, Essa I. Image and video based painterly animation. In: Proceedings of 3rd International Symposium on Non-Photorealistic Animation and Rendering. New York: ACM Press, 2004. 113–120
O’Donovan P, Hertzmann A. AniPaint: interactive painterly animation from video. IEEE Trans Vis Comput Graph, 2012, 18: 475–487
Bousseau A, Neyret F, Thollot J, et al. Video watercolorization using bidirectional texture advection. ACM Trans Graph, 2007, 26: 104
Cao C, Chen S, Zhang W, et al. Automatic motion-guided video stylization and personalization. In: Proceedings of 19th ACM International Conference on Multimedia. New York: ACM Press, 2011. 1041–1044
Litwinowicz P. Processing images and video for an impressionist effect. In: Proceedings of 24th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM Press/Addison-Wesley Publishing Co., 1997. 407–414
Hertzmann A. Painterly rendering with curved brush strokes of multiple sizes. In: Proceedings of 25th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM Press, 1998. 453–460
Neyret F. Advected textures. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. Eurographics Association, 2003. 147–153
Hertzmann A, Jacobs C E, Oliver N, et al. Image analogies. In: Proceedings of 28th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM Press, 2001. 327–340
Comaniciu D, Meer P. Mean shift: a robust approach toward feature space analysis. IEEE Trans Patt Anal Mach Intell, 2002, 24: 603–619
Collomosse J P, Rowntree D, Hall P M. Stroke surfaces: temporally coherent artistic animations from video. IEEE Trans Vis Comput Graph, 2003, 11: 540–549
Wang J, Thiesson B, Xu Y, et al. Image and video segmentation by anisotropic kernel mean shift. In: Proceedings of 8th European Conference on Computer Vision, Prague, 2004. 238–249
Wang J, Xu Y, Shum H Y, et al. Video tooning. ACM Trans Graph, 2004, 23: 574–583
Lin L, Zeng K, Lv H, et al. Painterly animation using video semantics and feature correspondence. In: Proceedings of 8th International Symposium on Non-Photorealistic Animation and Rendering. New York: ACM Press, 2010. 73–80
Guo Y W, Yu J H, Xu X D, et al. Example based painting generation. J Zhejiang Univ Sci A, 2006, 7: 1152–1159
Kyprianidis J E, Kang H. Image and video abstraction by coherence-enhancing filtering. In: Proceedings of Computer Graphics Forum. Blackwell Publishing Ltd., 2011, 30: 593–602
Wang B, Wang W, Yang H, et al. Efficient example-based painting and synthesis of 2d directional texture. IEEE Trans Vis Comput Graph, 2004, 10: 266–277
Efros A A, Freeman W T. Image quilting for texture synthesis and transfer. In: Proceedings of 28th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM Press, 2001. 341–346
Kang H, Lee S, Chui C K. Flow-based image abstraction. IEEE Trans Vis Comput Graph, 2009, 15: 62–76
Liu C. Beyond pixels: exploring new representations and applications for motion analysis. Dissertation for the Doctoral Degree. Massachusetts Institute of Technology, 2009
Bruhn A, Weickert J, Schnorr C. Lucas/Kanade meets Horn/Schunck: combining local and global optical flow methods. Int J Comput Vis, 2005, 61: 211–231
Brox T, Bruhn A, Papenberg N, et al. High accuracy optical flow estimation based on a theory for warping. In: Proceedings of 8th European Conference on Computer Vision, Prague, 2004. 25–36
Criminisi A, Perez P, Toyama K. Region filling and object removal by exemplar-based image inpainting. IEEE Trans Image Process, 2004, 13: 1200–1212
Bousseau A, Kaplan M, Thollot J, et al. Interactive watercolor rendering with temporal coherence and abstraction. In: Proceedings of 4th International Symposium on Non-Photorealistic Animation and Rendering. New York: ACM Press, 2006. 141–149
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Tang, Y., Zhang, Y., Shi, X. et al. Multi-style video stylization based on texture advection. Sci. China Inf. Sci. 58, 1–13 (2015). https://doi.org/10.1007/s11432-014-5255-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11432-014-5255-9