Abstract
This survey presents a comprehensive study on non-photorealistic rendering (NPR). NPR technique renders 2D input image into abstracted and artistic stylized images. NPR mainly dwells on image processing, computer vision, and visualizing the graphics processing techniques. The survey highlights the evolution of IA–AR system and the subsequent classification of NPR techniques. The survey also has cognized the various works done on stroke-based rendering, color image analogy, region-based rendering, image filtering abstraction, and stylization. The inference drawn from the survey is the computer system using a stylus to fully automatic structure-preserving image abstraction and stylization got evolved from a traditional method of human interaction. From the survey carried out on the most significant papers from 1963 to 2017, we felt the need for setting up of benchmark guidelines, the data set with varies subjective matters, and quality assessment techniques with various statistical essences. From the survey information pertaining to benchmark image characteristics, properties and their constraints with the contextual feature in an image have been identified. Finally, survey work listed out the NPR application in various fields of image processing and highlighted empirical challenges and hampers in NPR domain. This survey work has empowered us to proceed in the right direction and enthusiasm to bring forth the problem statement and carry out research work.
Similar content being viewed by others
References
Johnson, T.E.: Sketchpad III: a computer program for drawing in three dimensions. In: Proceedings of the May 21–23, 1963, Spring Joint Computer Conference (AFIPS ‘63(Spring)). ACM, New York (1963). http://dx.doi.org/10.1145/1461551.1461592
Appel, A.: The notion of quantitative invisibility and the machine rendering of solids. In: Rosenthal, S. (ed.) Proceedings of the 1967 22nd National Conference (ACM ‘67), pp. 387–393. ACM, New York (1967). http://dx.doi.org/10.1145/800196.806007
Baecker, R.M.: Picture-driven animation. In: Proceedings of the May 14–16, 1969, Spring Joint Computer Conference (AFIPS ‘69 (Spring)), pp. 273–288. ACM, New York, NY (1969). https://doi.org/10.1145/1476793.1476838
Perlin, K.: An image synthesizer. In: Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques 1985 (SIGGRAPH ‘85) (1985). http://dx.doi.org/10.1145/325334.325247
Kyprianidis, J.E., Collomosse, J., Wang,T., Isenberg,T.: State of the “art”: a taxonomy of artistic stylization techniques for images and video. In: IEEE Transactions on Visualization and Computer Graphics 19, 5 (May 2013), pp. 866–885 (2013). http://dx.doi.org/10.1109/TVCG.2012.160
Haeberli, P.: Paint by numbers: abstract image representations. In: Proceedings of the 17th Annual Conference on Computer Graphics and Interactive Techniques SIGGRAPH 1990, pp. 207–214. ACM, New York (1990). http://dx.doi.org/10.1145/97879.97902
DeCarlo, D., Santella, A.: Stylization and abstraction of photographs. In: Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ‘02), pp. 769–776. ACM, New York, NY (2002). https://doi.org/10.1145/566570.566650
Winnemöller, H., Olsen, S.C., Gooch, B.: Real-time video abstraction. In: ACM-Transaction Graphics. 25, 3 (July 2006), pp. 1221–1226 (2006). https://doi.org/10.1145/1141911.1142018
Kang, H., Lee, S., Chui, C.K.: Flow-based image abstraction. In: IEEE Transactions on Visualization and Computer Graphics 15, 1 (January 2009), pp. 62–76 (2009). http://dx.doi.org/10.1109/TVCG.2008.81
Anders, A., Marc, A.: . Approximating bounded, non-orientable surfaces from points. In: Proceedings of the Shape Modeling International 2004(SMI ‘04), pp. 243–252. IEEE Computer Society, Washington, DC (2004). https://doi.org/10.1109/SMI.2004.14
Hegde, S., Gatzidis, C., Tian, F.: Painterly rendering techniques: a state-of-the-art review of current approaches. Comput. Anim. Virtual Worlds 24(1), 43–64 (2013). https://doi.org/10.1002/cav.1435
Kaplan, M., Gooch, B., Cohen, E.: Interactive artistic rendering. In: Proceedings of the 1st International Symposium on Non-photorealistic Animation and Rendering (NPAR ‘00), pp. 67–74. ACM, New York (2000). https://doi.org/10.1145/340916.340925
Salisbury, M.P., Anderson, S.E., Barzel, R., Salesin, D.H. Interactive pen-and-ink illustration. In: Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ‘94), pp. 101-108. ACM, New York (1994). https://doi.org/10.1145/192161.192185
Hertzmann, A.: Painterly rendering with curved brush strokes of multiple sizes. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ‘98), pp. 453–460. ACM, New York (1998). http://dx.doi.org/10.1145/280814.280951
Salisbury, M.P., Wong, M.T., Hughes, J.F., Salesin, D.H.: Orientable textures for image-based pen-and-ink illustration. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ‘97), pp. 401-406. ACM Press/Addison-Wesley Publishing Co., New York (1997). https://doi.org/10.1145/258734.258890
Gooch, B., Reinhard, E., Gooch, A.: Human facial illustrations: creation and psychophysical evaluation. ACM Trans. Graph. 23, 1 (January 2004), pp. 27–44 (2004). http://dx.doi.org/10.1145/966131.966133
Grubert, J., Carpendale, S., Isenberg, T.: Interactive stroke-based npr using hand postures on large displays. Technical Report 2007–2008, pp. 279–282, 35, Department of Computer Science, University of Calgary, the short paper at Eurographics December 2008 (2008)
O’Donovan, P., Hertzmann, A.: AniPaint: interactive painterly animation from video. In: IEEE Transactions on Visualization and Computer Graphics 18, 3 (March 2012), pp. 475–487 (2012). http://dx.doi.org/10.1109/TVCG.2011.51
Santella, A., DeCarlo, D.: Abstracted painterly renderings using eye-tracking data. In: Proceedings of the 2nd International Symposium on Non-photorealistic Animation and Rendering (NPAR ‘02). ACM, New York (2002). http://dx.doi.org/10.1145/508530.508544
Kang, H.W., He, W., Chui, C.K., Chakraborty, U.K.: Interactive Sketch Generation. Vis. Comput 21, 821–830 (2005). https://doi.org/10.1007/s00371-005-0328-9
Zhao, M., Zhu, S.-C.: Abstract painting with interactive control of perceptual entropy. In: ACM Trans. Appl. Percept. 10, 1, Article 5 (March 2013) (2013). http://dx.doi.org/10.1145/2422105.2422110
Semmo, A., Limberger, D., Kyprianidis, J.E., Döllner, J.: Image stylization by interactive oil paint filtering. Comput. graph.55, (April 2016), pp. 157–171 (2016). https://doi.org/10.1016/j.cag.2015.12.001
Bousseau, A., Kaplan, M., Thollot, J., Sillion, F.X.: Interactive watercolor rendering with temporal coherence and abstraction. In: Proceedings of the 4th International Symposium on Non-photorealistic Animation and Rendering (NPAR ‘06), pp. 141–149. ACM, New York (2006). https://doi.org/10.1145/1124728.1124751
Lu, J., Sander, P.V., Finkelstein, A.: Interactive Painterly Stylization of Images, Videos and 3D Animations. In: Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D ‘10), pp. 127–134. ACM, New York (2010). http://dx.doi.org/10.1145/1730804.173082
Zhang, E., Hays, J., Turk, G.: . Interactive tensor field design and visualization on surfaces. In: IEEE Transactions on Visualization and Computer Graphics 13, 1 (January 2007), pp. 94–107 (2007). http://dx.doi.org/10.1109/TVCG.2007.16
Haggerty, P.: Almost automatic computer painting. IEEE Comput. Graph. Appl. 11, 11–12 (1991). https://doi.org/10.1109/mcg.1991.10051
Shiraishi, M., Yamaguchi, Y.: An algorithm for automatic painterly rendering based on local source image approximation. In: Proceedings of the 1st International Symposium on Non-photorealistic Animation and Rendering (NPAR ‘00), pp. 53–58. ACM, New York (2000). https://doi.org/10.1145/340916.340923
Collomosse, J.P., Hall, P.M.: Painterly rendering using image salience. In: Proceedings of the 20th UK Conference on Eurographics (EGUK ‘02), pp. 122–132. IEEE Computer Society, Washington, DC (2002)
Shugrina, M., Betke, M., Collomosse, J.: Empathic painting: interactive stylization through observed emotional state. In: Proceedings of the 4th International Symposium on Non-photorealistic Animation and Rendering (NPAR ‘06), pp. 87–96. ACM, New York (2006). https://doi.org/10.1145/1124728.1124744
Szirányi ,T., Tóth, Z.: Optimization of paintbrush rendering of images by dynamic mcmc methods. In: Figueiredo, M., Zerubia, J., Jain, A.K. (eds.) Proceedings of the Third International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR ‘01), pp. 201–215. Springer-Verlag, Berlin, Heidelberg (2001)
Wei, L.-Y., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ‘00), pp. 479-488. ACM Press/Addison-Wesley Publishing Co., New York (2000). http://dx.doi.org/10.1145/344779.345009
Ashikhmin, M.: Fast texture transfer. IEEE Comput. Graph. Appl. 23, 38–43 (2003). https://doi.org/10.1109/MCG.2003.1210863
Hertzmann, A.: Paint by relaxation. In: Ho-Shing Ip, H., Magnenat-Thalmann, N., Lau, R.W.H., Chua, T.S. (eds.) Computer Graphics International 2001 (CGI ‘01), pp. 47–54. IEEE Computer Society, Washington, DC, USA (2001)
Wen, F., Luan, Q., Liang, L., Xu, Y.-Q., Shum, H.-Y.: Color sketch generation. In: Proceedings of the 4th International Symposium on Non-photorealistic Animation and Rendering (NPAR ‘06), pp. 47–54. ACM, New York (2006). https://doi.org/10.1145/1124728.1124737
Setlur, V. Wilkinson, S.: Automatic stained glass rendering. In: Nishita. T., Peng, Q., Seidel, H.-P. (eds.) Proceedings of the 24th International Conference on Advances in Computer Graphics (CGI’06), pp. 682–691. Springer-Verlag, Berlin, Heidelberg (2006). http://dx.doi.org/10.1007/11784203_66
Litwinowicz, P.: Processing images and video for an impressionist effect. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ‘97), pp. 407–414. ACM Press/Addison-Wesley Publishing Co., New York (1997). https://doi.org/10.1145/258734.258893
Tresset, P., Leymarie, F.F.: Generative portrait sketching. In: Proc VSMM 2005, pp. 739–748, Hal Twaites (2005)
Colton, S., Valstar, M. F., Pantic, M.: Emotionally aware automated portrait painting. In: Proceedings of the 3rd International Conference on Digital Interactive Media in Entertainment and Arts (DIMEA ‘08), pp. 304–311. ACM, New York (2008). https://doi.org/10.1145/1413634.1413690
Mandryk. R.L., Mould. D., Li, H.: Evaluation of emotional response to non-photorealistic images. In: Spencer, S.N. (ed.) Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Non-photorealistic Animation and Rendering (NPAR ‘11), pp. 7–16. ACM, New York (2011). https://doi.org/10.1145/2024676.2024678
Collomosse, J.P., Hall, P.M.: Genetic paint: a search for salient paintings. In: Rothlauf, F., Branke, J., Cagnoni, S., Corne, D.W., Drechsler, R. (eds.) Proceedings of the 3rd European Conference on Applications of Evolutionary Computing (EC’05), pp. 437–447. Springer-Verlag, Berlin, Heidelberg (2005). http://dx.doi.org/10.1007/978-3-540-32003-6_44
Collomosse, J.P., Rowntree, D., Hall, P.M.: Stroke surfaces: temporally coherent artistic animations from video. In: IEEE Transactions on Visualization and Computer Graphics, vol 11, 5 (September 2005), pp. 540–549 (2005). http://dx.doi.org/10.1109/TVCG.2005.85
Ostromoukhov, V. Hersch, R.D.: Artistic screening. In: Mair, S.G., Cook, R. (eds.) Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ‘95), pp. 219–228. ACM, New York (1995). https://doi.org/10.1145/218380.218445
Ostromoukhov, V., Emmel, P., Rudaz, N., Amidror, I., Hersch, R.D.: Multi-level colour halftoning algorithms. Intl. symposium on advanced imaging and network technologies. Berlin, Oct 1996. SPIE 2949, 332–340 (1996)
Ostromoukhov, V. Hersch, R.D.: Multi-color and artistic dithering. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ‘99), pp. 425–432. ACM Press/Addison-Wesley Publishing Co., New York (1999). http://dx.doi.org/10.1145/311535.311605
Ostromoukhov, V.: Digital facial engraving. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ‘99), pp. 17–424. ACM Press/Addison-Wesley Publishing Co., New York (1999). http://dx.doi.org/10.1145/311535.311604
Ulichney, R.A.: Review of halftoning techniques. In: Proc. SPIE 3963, Color Imaging: Device-Independent Color, Color Hardcopy, and Graphic Arts V (1999). https://doi.org/10.1117/12.373419.
Streit, L., Buchanan, J.: Importance driven halftoning. Comput. Graph. Forum 173, 207–217 (1998). https://doi.org/10.1111/1467-8659.00268
Ostromoukhov, V., Hersch, R.D., Amidror, I.: Rotated dispersed dither: a new technique for digital halftoning. In: Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ‘94), pp. 123–130. ACM, New York (1994). https://doi.org/10.1145/192161.192188
Verevka, O., Buchanan, J.W.: Halftoning with image-based dither screens. In: Proceedings of the 1999 Conference on Graphics Interface ‘99, pp. 167–174. Morgan Kaufmann Publishers Inc., San Francisco (1999)
Hausner A.: Pointillist halftoning. In: Proc. CGIM, pp. 134–139. ACTA Press, Calgary (2005)
Streit, L., Veryovka, O., Buchanan, J.: Non-photorealistic rendering using an adaptive halftoning technique. In: Proceedings of the Western Computer Graphics Symposium, pp. 37–45. Banff (1999)
Li, Hua, Mould, David: Contrast-aware halftoning. Comput. Graph. Forum. 29(2), 273–280 (2010). https://doi.org/10.1111/j.1467-8659.2009.01596.x
Ahmed, A.G.M., Deussen, O.: Amplitude modulated line-based halftoning. In: Proceedings of the Conference on Computer Graphics & Visual Computing (CGVC ‘16), pp. 41–43. Eurographics Association, Goslar (2016). https://doi.org/10.2312/cgvc.20161294
Kim, D., Son, M., Lee, Y., Kang, H., Lee, S.: Feature-guided image stippling. In: Proceedings of the Nineteenth Eurographics Conference on Rendering (EGSR ‘08), pp. 1209–1216. Eurographics Association, Aire-la-Ville, (2008). http://dx.doi.org/10.1111/j.1467-8659.2008.01259.x
Li, H., Mould, D.: Structure-preserving stippling by priority-based error diffusion. In: Proceedings of Graphics Interface 2011 (GI ‘11), pp. 127–134. Canadian Human-Computer Communications Society, School of Computer Science, University of Waterloo, Waterloo (2011)
Pang, W.-M., Qu,Y., Wong, T.-T., Cohen-Or, D., Heng, P.-A.: Structure-aware Halftoning. In: ACM-SIGGRAPH-2008 papers (SIGGRAPH ‘08). ACM, New York (2008). https://doi.org/10.1145/1399504.1360688
Son, M., Lee, Y., Kang, H., Lee, S.: Structure grid for directional stippling. Graph. Models 73, 74–87 (2011). https://doi.org/10.1016/j.gmod.2010.12.001
Hausner, A.: Simulating decorative mosaics. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ‘01), pp. 573–580. ACM, New York (2001). https://doi.org/10.1145/383259.383327
Elber, G., Wolberg, G.: Rendering traditional mosaics. Vis. Comput. 19(1), 67–78 (2003)
Kim, J., Pellacini, F.: Jigsaw image mosaics. In: Proceedings of the 29th Annual Conference on Computer-Graphics and Interactive Techniques (SIGGRAPH ‘02), pp. 657–664. ACM, New York (2002). https://doi.org/10.1145/566570.566633
Di Blasi, G., Gallo, G., Petralia, M.: Fast techniques for mosaic rendering. In: Neumann, L., Sbert, M., Gooch, B., Purgathofer, W. (eds.) Proceedings of the First Eurographics Conference on Computational Aesthetics in Graphics, Visualization and Imaging (Computational Aesthetics’05), pp. 29–39. Eurographics Association, Aire-la-Ville (2005). http://dx.doi.org/10.2312/COMPAESTH/COMPAESTH05/029-039
Dobashi, Y., Haga, T., Johan, H., Nishita, T.: A method for creating mosaic images using Voronoi diagrams. Eurographics Association 2002—short presentation, pp. 341–348. (2002). https://doi.org/10.2312/egs.20021036
Faustino, G.M., Henrique de Figueiredo, L.: Simple adaptive mosaic effects. In: Proceedings of the XVIII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI ‘05), pp. 315–322. IEEE Computer Society, Washington, DC (2005). https://doi.org/10.1109/SIBGRAPI.2005.46
Mould, D.: A stained glass image filter. In: Proceedings of the 14th Eurographics Workshop on Rendering (EGRW ‘03), pp. 20–25. Eurographics Association, Aire-la-Ville (2003)
Orchard, J., Kaplan, C.S.: Cut-out image mosaics. In: Proceedings of the 6th International Symposium on Non-photorealistic Animation and Rendering (NPAR ‘08), pp/ 79–87. ACM, New York (2008). http://dx.doi.org/10.1145/1377980.1377997
Choi, Y.S., Koo, B.K., Lee, J.H.: Template based image mosaics. In: Ma, L., Rauterberg, M., Nakatsu, R. (eds.). Entertainment computing—ICEC 2007. ICEC 2007. Lecture Notes in Computer Science, vol 475–478. Springer, Berlin, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74873-1_64
Liu Y., Veksler, O.: Animated classic mosaics from video. In: Bebis G. et al. (eds) Advances in Visual Computing. ISVC 2009. Lecture Notes in Computer Science, vol 5876, pp/ 1085–1096. Springer, Berlin, Heidelberg (2009). http://dx.doi.org/10.1007/978-3-642-10520-3_104
Hu, W., Chen, Z., Pan, H., Yu, Y., Grinspun, E., Wang, W.: Surface mosaic synthesis with irregular tiles. IEEE Trans. Vis. Comput. Graph. 22, 1302–1313 (2016). https://doi.org/10.1109/TVCG.2015.2498620
Hertzmann, A., Jacobs, C.E., Oliver, N., Curless, B., Salesin, D.H.: Image analogies. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ‘01), p. 327–340. ACM, New York (2001). https://doi.org/10.1145/383259.383295
Curtis, C.J., Anderson, S.E., Seims, J.E., Fleischer, K.W., Salesin, D.H.: Computer-generated watercolor, pp. 421–430. ACM Press/Addison-Wesley Publishing Co., New York (1997). https://doi.org/10.1145/258734.258896
Faridul, H.S., Pouli, T., Chamaret, C., Stauder, J., Tremeau, A., Reinhard, E.: Survey of color mapping and its applications. The Eurographics Association ISBN: 1017-4656 (2014). https://doi.org/10.2312/egst.20141035
Reinhard, E., Ashikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graph. Appl. 21, 34–41 (2001). https://doi.org/10.1109/38.946629
Daniel, Ruderman L., Cronin, Thomas W., Chiao, Chuan-Chin: Statistics of cone responses to natural images: implications for visual coding. J. Opt. Soc. Am. A 15(8), 2036–2045 (1998). https://doi.org/10.1364/josaa.15.002036
Sýkora, D., Buriánek. J., Žára, J.: Unsupervised colorization of black-and-white cartoons. In: Spencer, S.N. (ed.) Proceedings of the 3rd International Symposium on Non-photorealistic Animation and Rendering (NPAR ‘04), pp. 121–127. ACM, New York (2004). http://dx.doi.org/10.1145/987657.987677
Neumann, L. Neumann, A.: Color style transfer techniques using hue, lightness and saturation histogram matching. In: Proceedings of the First Eurographics Conference on Computational Aesthetics in Graphics, Visualization and Imaging (2005). http://dx.doi.org/10.2312/COMPAESTH/COMPAESTH05/111-122
Li, Y., Lizhuang, M., Di, W.: Fast colorization using edge and gradient constrains. In: Proceedings of the 15th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, pp 309–316 (2007)
Greenfield, G.R., House, D.H.: Image recolouring induced by palette colour associations. J Winter School Comput. Graph. CAD Syst. 11(1), 3–7 (2003)
Chang, Y., Saito, S., Nakajima, M.: Example-based color transformation of image and video using basic color categories. IEEE Trans Image Process (2007). https://doi.org/10.1109/tip.2006.888347
Wang, C.-M., Wang, R.-J.: Image-based color ink diffusion rendering. IEEE Trans. Vis. Comput. Graph. 13(2), 235–246 (2007). https://doi.org/10.1109/TVCG.2007.41
Bae, S., Paris, S., Durand, F.: Two-scale tone management for photographic look. In: SIGGRAPH’06: ACM SIGGRAPH 2006 Papers, pp. 637–645. ACM, New York (2006). https://doi.org/10.1145/1179352.1141935
Lischinski, D., Farbman, Z., Uyttendaele, M., Szeliski, R.: Interactive local adjustment of tonal values. ACM trans. Graph. 25(3), 646–653 (2006). https://doi.org/10.1145/1179352.1141936
Pu, Y., Su, Y., Wei, X., Zhao, Y., Xu, D., Qian, W.: Stylized rendering of Yunnan heavy color painting. In: 2008 Second International Symposium on Intelligent Information Technology Application, Shanghai, pp. 397–401 (2008). https://doi.org/10.1109/IITA.2008.357
Su, Z., Deng, D., Yang, X., Luo, X.: Color transfer based on multiscale gradient-aware decomposition and color distribution mapping. In: Proceedings of the 20th ACM International Conference on Multimedia (MM ‘12), pp. 753–756. ACM, New York (2012). https://doi.org/10.1145/2393347.2396304
Yang, H., Kwon, Y., Min, K.: A Texture-based approach for hatching color photographs. In: Proceedings of the 6th International Conference on Advances in Visual Computing, Vol Part I (ISVC’10), pp. 86–95. Springer-Verlag, Berlin, Heidelberg (2010)
Wang, B., Yu, Y., Wong, T.-T., Chen, C., Xu, Y.-Q.: Data-driven image color theme enhancement. ACM Trans. Graph. 29, 10 (2010). https://doi.org/10.1145/1882261.1866172
Jeschke, S., Cline, D., Wonka, P.: Estimating color and texture parameters for vector graphics (2011). https://doi.org/10.1111/j.1467-8659.2011.01877.x
Pouli, T., Reinhard, E.: Progressive color transfer for images of arbitrary dynamic range. Comput. Graph. 35(1), 67–80 (2011). https://doi.org/10.1016/j.cag.2010.11.003
Kim, H.R., Yoo, M.-J., Kang, H., Lee, I.K.: Perceptually-based color assignment. Comput. Graph. Forum 33, 309–318 (2014). https://doi.org/10.1111/cgf.12499
Lin, S., Ritchie, D., Fisher, M., Hanrahan, P.: Probabilistic color-by-numbers: suggesting pattern colorizations using factor graphs. ACM Tans. 32(37), 1–37 (2013). https://doi.org/10.1145/2461912.2461988
Zhao, F., Su, Z., Liang, X., Lai, Y.K., Rosin, P.L.: Example-based image colorization using locality consistent sparse representation. IEEE Trans. Image Process. 26, 11 (2017). https://doi.org/10.1109/tip.2017.2732239
Okura, F., Vanhoey, K., Bousseau, A., Efros, A.A., Drettakis, G.: Unifying color and texture transfer for predictive appearance manipulation. In: Proceedings of the 26th Eurographics Symposium on Rendering (EGSR ‘15), pp. 53–63. Eurographics Association, Aire-la-Ville (2015). http://dx.doi.org/10.1111/cgf.12678
Alvarez, L., Gousseau, Y., Morel, J.-M., Salgado, A.: Exploring the space of abstract textures by principles and random sampling. J Math Imaging Vis. 53, 332 (2015). https://doi.org/10.1007/s10851-015-0582-z
Hristova, H., Le Meur, O., Cozot, R., Bouatouch. K.: Style-aware robust color transfer. In: Proceedings of the Workshop on Computational Aesthetics (CAE ‘15), pp. 67–77. Eurographics Association, Goslar, Germany (2015)
Risser, E., Wilmot, P., Barnes, C.: Stable and controllable neural texture synthesis and style transfer using histogram losses. CoRR 1701.08893. abs/1701.08893 (2017 )
Hensman, P., Aizawa, K.: cGAN-based manga colorization using a single training image. In: 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), pp 72–77, Kyoto (2017). https://doi.org/10.1109/icdar.2017.295
Liao, J., Yao, Y., Yuan, L., Hua, G., Kang, S.B.: Visual attribute transfer through deep image analogy. ACM Trans. Graph… 36(4), 15 (2017). https://doi.org/10.1145/3072959.3073683
Elad, M., Milanfar, P.: Style transfer via texture synthesis. Trans. Image Proc. 26, 2338–2351 (2017). https://doi.org/10.1109/TIP.2017.2678168
Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style. CoRR (2015). https://doi.org/10.1167/16.12.326
Yi, Z., Li, Y., Ji, S., Gong, M.: Artistic stylization of face photos based on a single exemplar. Vis Comput 33, 1443–1452 (2017). https://doi.org/10.1007/s00371-016-1290-4
Zhao, H., Rosin, P.L., Lai, Y.-K.: Automatic semantic style transfer using deep convolution neural networks and soft masks. CoRR (2017)
Liu, X.C., Cheng, M.M., Lai, Y.K., Rosin, P.L.: Depth-aware neural style transfer. In: Spencer, S.N. (ed.). Proceedings of the Symposium on Non-photorealistic Animation and Rendering (NPAR ‘17). ACM, New York (2017). https://doi.org/10.1145/3092919.3092924
Agarwala, A.: SnakeToonz: a semi-automatic approach to creating cel animation from video. In: Proceedings of the 2nd International Symposium on Non-photorealistic Animation and Rendering (NPAR ‘02). ACM, New York (2002). http://dx.doi.org/10.1145/508530.508554
Agarwala, A., Hertzmann, A., Salesin, D.H., Seitz, S.M.: Keyframe-based tracking for rotoscoping and animation. ACM Trans Graph 23, 584–591 (2004). https://doi.org/10.1145/1015706.1015764
Bousseau, A., Kaplan, M., Thollot, J., Sillion, F.X.: Interactive watercolor rendering with temporal coherence and abstraction, pp. 141–149. ACM, New York (2006). https://doi.org/10.1145/1124728.1124751
Bousseau, A., Neyret, F., Thollot, J., Salesin, D.: Video watercolorization using bidirectional texture advection. ACM Trans. Graph. (2007). https://doi.org/10.1145/1276377.1276507
Wang, T., Collomosse, J., Slatter, D., Cheatle, P., Greig, D.: Video stylization for digital ambient displays of home movies. ACM Computers & Graphics, New York (2010). http://dx.doi.org/10.1145/1809939.1809955
Kagaya, Mizuki, Brendel, William, Deng, Qingqing, Kesterson, Todd, Todorovic, Sinisa, Neill, Patrick J., Zhang, Eugene: Video painting with space-time-varying style parameters. IEEE Trans. Vis. Comput. Graph. 17(1), 74–87 (2011)
Salisbury, M., Anderson, C., Lischinski, D., Salesin, D.H.: Scale-dependent reproduction of pen-and-ink illustrations, pp. 461–468. ACM, New York (1996). http://dx.doi.org/10.1145/237170.237286
Song, Y.-Z., Rosin, P.L., Hall, P.M., Collomosse, J.: Arty shapes. In: Brown, P., Cunningham, D. W., Interrante, V., McCormack, J. (eds.) Proceedings of the Fourth Eurographics Conference on Computational Aesthetics in Graphics, Visualization and Imaging (Computational Aesthetics’08), pp. 65–72. Eurographics Association, Aire-la-Ville (2008). http://dx.doi.org/10.2312/COMPAESTH/COMPAESTH08/065-072
Bangham, A., Gibson, S.E., Richard, H.: The art of scale-space, vol 1 (2003). https://doi.org/10.5244/c.17.58
Mould, D., Grant, K.: Stylized black and white images from photographs. In: Proceedings of the 6th International Symposium on Non-photorealistic Animation and Rendering (NPAR ‘08), pp.49–58. ACM, New York (2008). http://dx.doi.org/10.1145/1377980.1377991
Zeng, K., Zhao, M., Xiong, C., Zhu, S.-C.: From image parsing to painterly rendering. ACM Trans. Graph. 29, 11 (2009). https://doi.org/10.1145/1640443.1640445
Reinhard, E., Ashikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graph. Appl. 21, 34–41 (2001). https://doi.org/10.1109/38.946629
Zhao, M., Zhu, S.-C.: Sisley the abstract painter. In: Proceedings of the 8th International Symposium on Non-photorealistic Animation and Rendering (NPAR ‘10), pp. 99–107. ACM, New York (2010). https://doi.org/10.1145/1809939.1809951
Osher, S., Rudin, L.I.: Feature-oriented image enhancement using shock filters. SIAM J. Numer. Anal. 27, 919–940 (1990). https://doi.org/10.1137/0727053
Doellner, J., Nienhaus, M.: Edge-enhancement-an algorithm for real-time non-photorealistic rendering. J. WSCG 11(1), 346–353 (2003). (ISSN 1213-6972 WSCG’2003)
Kyprianidis, J.E., Kang, H., Döllner, J.: Image and video abstraction by anisotropic kuwahara filtering. Comput. Graph. Forum 28(7), 1955–1963 (2009). https://doi.org/10.1111/j1467-8659.2009.01574.x
Raskar, R., Ilie, A., Yu, J.: Image fusion for context enhancement and video surrealism. In: Spencer, S.N. (ed.). Proceedings of the 3rd International Symposium on Non-photorealistic Animation and Rendering (NPAR ‘04), pp. 85–152. ACM, New York (2004). http://dx.doi.org/10.1145/987657.987671
Redmond, N., Dingliana, J.: Adaptive abstraction of 3D scenes in real-time. Eurographics (2007). https://doi.org/10.2312/egs.20071038
Mould, D.: Texture-preserving abstraction. In: Proceedings of the Symposium on Non-photorealistic Animation and Rendering (NPAR ‘12), pp. 75–82. Eurographics Association, Goslar (2012)
Kyprianidis, J.E.: Image and video abstraction by multi-scale anisotropic kuwahara filtering. In: Spencer, S.N. (ed.) Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Non-photorealistic Animation and Rendering (NPAR ‘11), pp. 55–64. ACM, New York (2011). https://doi.org/10.1145/2024676.2024686
Kyprianidis, J.E., Semmo, A., Kang H., Döllner J.: Anisotropic kuwahara filtering with polynomial weighting functions (2010). https://doi.org/10.2312/localchapterevents/tpcg/tppcg10/025-030
Chun, S., Jung, K., Kim, J.: Oil painting rendering through virtual light effect and regional analysis. In: Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry (VRCAI ‘11), pp. 419–422. ACM, New York (2011). http://dx.doi.org/10.1145/2087756.2087833
Kolliopoulos, A., Wang, J.M., Hertzmann, A.: Segmentation-based 3D artistic rendering. In: Akenine-Möller, T., Heidrich, W. (eds.) Proceedings of the 17th Eurographics Conference on Rendering Techniques (EGSR ‘06), pp. 361–370. Eurographics Association, Aire-la-Ville (2006). http://dx.doi.org/10.2312/EGWR/EGSR06/361-370
Kyprianidis, J.E., Kang, H.: Image and Video abstraction by coherence-enhancing filtering. Comput. Graph. Forum. 30, 593–602 (2011). https://doi.org/10.1111/j.1467-8659.2011.01882.x
Williams, S.G.D., Green, R.: Mr. painting robot. Siggraph, Computer Graphics Proceedings, pp. 83–89 (2009)
Kang, Henry, Lee, Seungyong: Shape-simplifying image abstraction. Comput. Graph. Forum. 27, 1773–1780 (2008). https://doi.org/10.1111/j.1467-8659.2008.01322.x
Papari, G., Petkov, N., Campisi, P.: Artistic edge and corner enhancing smoothing. Trans. Image Proc. 16(10), 2449–2462 (2007). https://doi.org/10.1109/TIP.2007.903912
Winnemöller, H.: XDoG: advanced image stylization with extended difference-of-gaussians. In: Spencer, S.N. (ed.) Proceedings of the ACM Siggraph/Eurographics Symposium on Non-photorealistic Animation and Rendering (NPAR ‘11), pp. 147–156. ACM, New York (2011), https://doi.org/10.1145/2024676.2024700
Nagendra Swamy H.S., Pavan Kumar M.P.: An integrated filter based approach for image abstraction and stylization. In: Swamy, P., Guru, D. (eds.) Multimedia Processing, Communication and Computing Applications. Lecture Notes in Electrical Engineering, vol 213. Springer, New Delhi (2013)
Kang, H., Lee, S., Chui, C.K.: Coherent line drawing. In: Proceedings of the 5th International Symposium on Non-photorealistic Animation and Rendering (NPAR ‘07), pp. 43–50. ACM, New York (2007). https://doi.org/10.1145/1274871.1274878
Spicker, M., Kratt, J., Arellano, D., Deussen, O.: Depth-aware coherent line drawings. In: SIGGRAPH Asia 2015 Technical Briefs (SA ‘15). ACM, New York (2015). https://doi.org/10.1145/2820903.2820909
Nagendra Swamy H.S., Pavan Kumar, M.P.: Line drawing for conveying shapes in HDR images. Int. J. Innov. Eng. Technol. (IJIET) 2(2), 353–362 (2013)
Mould, D.: Image and video abstraction using cumulative range geodesic filtering. Comput. Graph. 37, 413–430 (2013). https://doi.org/10.1016/j.cag.2013.03.002
De Arruda, F.A.P.V., de Queiroz, J.E.R., Gomes, H.M.: Non-photorealistic neural-sketching. J. Braz. Comput. Soc. 18, 237 (2012). https://doi.org/10.1007/s13173-012-0061-y
Tsai, H.C., Lee, Y.H., Lee, R.R., Chu, H.K.: User-guided line abstraction using coherence and structure analysis. Comput. Vis. Media 3(2), 177–188 (2017). https://doi.org/10.1007/s41095-016-0076-y
Zhang, S., Li, X., Hu, S., Martin, R.R.: Online video stream abstraction and stylization. IEEE Trans. Multimedia 13(6), 1286–1294 (2011). https://doi.org/10.1109/tmm.2011.2165052
Son, M., Lee, Y., Kang, H., Lee, S.: Art-photographic detail enhancement. Comput. Graph. Forum. 33(2), 391–400 (2014). https://doi.org/10.1111/cgf.12298
Cho, H., Lee, H., Kang, H., Lee, S.: Bilateral texture filtering. ACM Trans Graph (TOG) 33(4), 128 (2014). https://doi.org/10.1145/2601097.2601188
Shahcheraghi, Z., See, J., Halin, A.A.: Image abstraction using anisotropic diffusion symmetric nearest neighbor filter. In Ooi, W., Snoek, C., Tan, H., Ho, C.-K., Huet, B., Ngo, C.-W. (eds.) Proceedings of the 15th Pacific-Rim Conference on Advances in Multimedia Information Processing—PCM 2014—Volume 8879, Vol. 8879, pp. 343–352. Springer-Verlag, New York (2014). https://doi.org/10.1007/978-3-319-13168-9_38
Li, J., Mould, D.: Image warping for a painterly effect. In: Proceedings of the Workshop on Computational Aesthetics (CAE ‘15), pp. 131–140. Eurographics Association, Goslar (2015)
Jeon, J., Lee, H., Kang, H., Lee, S.: Scale-aware structure-preserving texture filtering. Comput. Graph. Forum 35, 77–86 (2016). https://doi.org/10.1111/cgf.13005
Lee, H., Jeon, J., Kim, J., Lee, S.: Structure-texture decomposition of images with interval gradient. Comput. Graph. Forum 36, 262–274 (2017). https://doi.org/10.1111/cgf.12875
Jing, S.; Liu, Y., Xu, K.: A second order variation based bilateral filter for image stylization and texture removal. In: 2016 International Conference on Virtual Reality and Visualization (ICVRV), Hangzhou, pp. 242–249 (2016). https://doi.org/10.1109/icvrv.2016.47
Deshpande, A., Raman, S.: Adaptive artistic stylization of images. In: Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP ‘16). ACM, New York (2016). https://doi.org/10.1145/3009977.3009985
Sadreazami, H., Asif, A., Mohammadi, A.: Iterative graph-based filtering for image abstraction and stylization. IEEE Trans. Circ. Syst. II Express Briefs 65(2), 251–255 (2018). https://doi.org/10.1109/tcsii2017.2669866
Azami, R., Mould, D.: Detail and color enhancement in photo stylization. In: Spencer, S.N. (ed.) Proceedings of the Symposium on Computational Aesthetics (CAE ‘17). ACM, New York (2017). https://doi.org/10.1145/3092912.3092917
Shakeri, H., Nixon, M., DiPaola, S.: Saliency-based artistic abstraction with deep learning and regression trees. J. Imaging Sci. Technol. 61(6), 060402-1–060402-9 (2017)
Wu, Yi-Ta, Yeh, Jeng-Sheng, Wu, Fu-Che, Chuang, Yung-Yu.: Tangent-based binary image abstraction. J. Imaging. 3, 16 (2017). https://doi.org/10.3390/jimaging3020016
Mould, D., Rosin, P.L.: A benchmark image set for evaluating stylization. In: Proceedings of the Joint Symposium on Computational Aesthetics and Sketch Based Interfaces and Modeling and Non-photorealistic Animation and Rendering (Expresive ‘16), pp. 11–20. Eurographics Association, Aire-la-Ville (2016)
Mould, D., Rosin, P.L.: Developing and applying a benchmark for evaluating image stylization. Comput. Graph. 67, 58–76 (2017). https://doi.org/10.1016/j.cag.2017.05.025
Rosin, P.L., Mould, D., Berger, I., Collomosse, J., Lai, Y.K., Li, C., Li, H., Shamir, A., Wand, M., Wang, T., Winnemöller, H.: Benchmarking non-photorealistic rendering of portraits. In: Spencer, S.N. (ed.) Proceedings of the Symposium on Non-photorealistic Animation and Rendering (NPAR ‘17). ACM, New York (2017). https://doi.org/10.1145/3092919.3092921
Hasler, D., Suesstrunk, S.E.: Measuring colorfulness in natural images. Proc. SPIE Int. Soc. Opt. Eng. 5007, 87–95 (2003). https://doi.org/10.1117/12.477378
Machado, P., Cardoso, A.: Computing aethetics. In: Moreira de Oliveira, F. (ed.) Proceedings of the 14th Brazilian Symposium on Artificial Intelligence: Advances in Artificial Intelligence (SBIA ‘98), pp. 219–228. Springer-Verlag, London (1998)
Matković, K., Neumann, L., Neumann, A., Psik, T., Purgathofer, W.: Global contrast factor—a new approach to image contrast. In: Neumann, L., Sbert, M., Gooch, B., Purgathofer, W. (eds.) Proceedings of the First Eurographics Conference on Computational Aesthetics in Graphics, Visualization and Imaging (Computational Aesthetics’05), pp. 159–167. Eurographics Association, Aire-la-Ville (2005). http://dx.doi.org/10.2312/COMPAESTH/COMPAESTH05/159-167
Bahrami, K., Kot, A.C.: A fast approach for no-reference image sharpness assessment based on maximum local variation. IEEE Signal Proc. Lett. 21(6), 751–755 (2014). https://doi.org/10.1109/lsp.2014.2314487
Smith, S.M., Brady, J.M.: Susan—a new approach to low level image processing. Int. J. Comput. Vis. 23(1), 45–78 (1997). https://doi.org/10.1023/A:1007963824710
Immerkær, J.: Fast noise variance estimation. Comput. Vis. Image Underst. 64, 300–302 (1996). https://doi.org/10.1006/cviu.1996.0060
Yusra, A., Al-Najjar, Y., Soong, D.C.: Comparison of image quality assessment: PSNR, HVS, SSIM, UIQI. Int. J. Sci. Eng. Res. 3(8), 1 (2012). (ISSN 2229-5518)
Walpole, R.E., Myers, R.H., Myers, S.L., Ye, K.: Probability and statistics for engineers and scientists, 9th edn. Prentice Hall, New York (2010)
Tietjen, C., Isenberg, T., Preim, B.: Combining silhouettes, shading, and volume rendering for surgery education and planning. In: Proc. of Euro-Vis, pp. 303–310. Eurographics Association (2005). https://doi.org/10.2312/vissym/eurovis05/303-310
Besançon, L., Semmo, A., Biau, D., Frachet, B., Pineau, V., Sariali, E.H., Taouachi, R., Isenberg, T., Dragicevic, P.: Reducing affective responses to surgical images through color manipulation and stylization. In: Proceedings of the Joint Symposium on Computational Aesthetics and Sketch-Based Interfaces and Modeling and Non-photorealistic Animation and Rendering (Expressive ‘18). ACM, New York (2018). https://doi.org/10.1145/3229147.3229158
Yang, Q.: Recursive bilateral filtering. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds) Computer Vision—ECCV 2012. ECCV 2012. Lecture Notes in Computer Science, vol 7572. Springer, Berlin (2012)
Orzan, A., Bousseau, A., Barla, P., Thollot, J.: Structure-preserving manipulation of photographs. In: Proceedings of the 5th International Symposium on Non-photorealistic Animation and Rendering (NPAR ‘07), pp. 130–110. ACM, New York (2007). https://doi.org/10.1145/1274871.1274888
Secord A.: Weighted voronoi stippling. In: proceedings of the 2nd International Symposium on Non-photorealistic Animation and Rendering, pp. 37–43. NPAR’02. ACM, New York (2002). http://dx.doi.org/10.1145/508530.508537
Christoudias, C., Georgescu, B., Meer, P.: Synergism in low level vision. Proc. 16th Int’l. Conf.Pattern Recogn. 4, 150–155 (2002). https://doi.org/10.1109/icpr.2002.1047421
Orzan, A., Bousseau, A., Winnemöller, H., Barla, P., Thollot, J., Salesin, D.: Diffusion curves: a vector representation for smooth-shaded images. ACM Trans. Graph. 27, 56 (2008). https://doi.org/10.1145/2483852.2483873
Gatys, L.A., Ecker, A.S., Bethge, M.: Texture synthesis using convolution neural networks. In: Cortes, C., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Proceedings of the 28th International Conference on Neural Information Processing Systems—Volume 1 (NIPS’15), vol. 1, pp. 262–270. MIT Press, Cambridge (2015)
Ulyanov, D., Lebedev, V., Vedaldi, A., Lempitsky, V.: Texture networks: feed-forward synthesis of textures and stylized images. CoRR (2016). arXiv:1603.03417
Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. CoRR (2017). arXiv:1703.06868
Dumoulin, V., Shlens, J., Kudlur, M.: A learned representation for artistic style. In: International Conference on Learning Representations (ICLR-2017). arxiv.org/abs/1610.07629
van Noord, N., Postma, E.: A learned representation of artist-specific colourisation. In: IEEE International Conference on Computer Vision Workshops (ICCVW), Venice, pp. 2907–2915 (2017). https://doi.org/10.1109/ICCVW.2017.343
Wang, X., Oxholm, G., Zhang, D., Wang, Y.-F.: Multimodal transfer: a hierarchical deep convolution neural network for fast artistic style transfer. Proc. IEEE Confer. Comput. Vis. Pattern Recogn. 2017, 7178–7186 (2017)
Luan, F., Paris, S., Shechtman, E., Bala, K.: Deep photo style transfer. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, (2017). arXiv:1703.07511
Winnemoeller, H., Kyprianidis, J.E., Olsen, S.C.: XDoG: an eXtended difference-of-gaussians compendium including advanced image stylization. Comput. Graph. 36, 740–753 (2012). https://doi.org/10.1016/j.cag.2012.03.004
Acknowledgements
During our work involving setting up of benchmark data set, many noteworthy images were scrutinized. We the authors offer our heartfelt thanks for the following authors and resource persons for their courtesy in publishing the images pertaining to their work. (1) Dr Jan Eric Kypriandis and associates. (2) Dr Henry Kang and associates. (3) Ar.Yeshaswini Rondamath, Asst.Professor, BMSSA, Bangalore. (4) Prof. Vishwas CGM, Asst. Professor. JNNCE, Shivamogga. (5) Dr Nirmala Shivananda. Prof and Head, Dept. of CSE. JNNCE Shivamogga. Our thanks are due to Dr David Mould for the valuable suggestion in assessing the image property.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Kumar, M.P.P., Poornima, B., Nagendraswamy, H.S. et al. A comprehensive survey on non-photorealistic rendering and benchmark developments for image abstraction and stylization. Iran J Comput Sci 2, 131–165 (2019). https://doi.org/10.1007/s42044-019-00034-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42044-019-00034-1