Advertisement

Artistic Stylization by Nonlinear Filtering

  • Jan Eric Kyprianidis
Chapter
Part of the Computational Imaging and Vision book series (CIVI, volume 42)

Abstract

Image processing techniques that perform local filtering operations provide an interesting alternative to other classical techniques, such as stroke-based rendering or segmentation-based approaches. In this chapter, several popular approaches developed in the previous years are reviewed. Among these are approaches based on the bilateral filter, the difference of Gaussians filter, and the Kuwahara filter, as well as approaches that combine diffusion with shock filtering. In addition, a brief introduction to approaches based on morphological filtering and techniques working in the gradient domain is given. Besides discussing isotropic approaches, a focus is placed on anisotropic generalizations that take the local structure into account. These typically create a strong artistic look by enhancing and exaggerating directional image features.

Keywords

Partial Differential Equation Bilateral Filter Canny Edge Detector Gradient Domain Line Integral Convolution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Agrawal, A., Raskar, R.: Gradient domain manipulation techniques in vision and graphics. In: ICCV Course (2007) Google Scholar
  2. 2.
    Alvarez, L., Mazorra, L.: Signal and image restoration using shock filters and anisotropic diffusion. SIAM J. Numer. Anal. 31(2), 590–605 (1994). doi: 10.1137/0731032 MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Aubert, G., Kornprobst, P.: Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations. Springer, Berlin (2006) zbMATHGoogle Scholar
  4. 4.
    Aurich, V., Weule, J.: Non-linear Gaussian filters performing edge preserving diffusion. In: Proc. DAGM-Symposium, pp. 538–545 (1995) Google Scholar
  5. 5.
    Barash, D., Comaniciu, D.: A common framework for nonlinear diffusion, adaptive smoothing, bilateral filtering and mean shift. Image Vis. Comput. 22(1), 73–81 (2004) CrossRefGoogle Scholar
  6. 6.
    Bhat, P., Zitnick, C.L., Cohen, M.F., Curless, B.: GradientShop: a gradient-domain optimization framework for image and video filtering. ACM Trans. Graph. 29(2), 10 (2010). doi: 10.1145/1731047.1731048 CrossRefGoogle Scholar
  7. 7.
    Bousseau, A., Kaplan, M., Thollot, J., Sillion, F.X.: Interactive watercolor rendering with temporal coherence and abstraction. In: Proc. NPAR, pp. 141–149 (2006). doi: 10.1145/1124728.1124751 Google Scholar
  8. 8.
    Bousseau, A., Neyret, F., Thollot, J., Salesin, D.: Video watercolorization using bidirectional texture advection. ACM Trans. Graph. 26(3), 104 (2007). doi: 10.1145/1276377.1276507 CrossRefGoogle Scholar
  9. 9.
    Brox, T., Boomgaard, R., Lauze, F., Weijer, J., Weickert, J., Mrázek, P., Kornprobst, P.: Adaptive structure tensors and their applications. In: Visualization and Processing of Tensor Fields, pp. 17–47. Springer, Berlin (2006). doi: 10.1007/3-540-31272-2_2 CrossRefGoogle Scholar
  10. 10.
    Cabral, B., Leedom, L.C.: Imaging vector fields using line integral convolution. In: Proc. SIGGRAPH, pp. 263–270 (1993). doi: 10.1145/166117.166151 Google Scholar
  11. 11.
    Canny, J.F.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8, 769–798 (1986). doi: 10.1109/TPAMI.1986.4767851 Google Scholar
  12. 12.
    Chen, J., Paris, S., Durand, F.: Real-time edge-aware image processing with the bilateral grid. ACM Trans. Graph. 26(3), 103 (2007). doi: 10.1145/1276377.1276506 CrossRefGoogle Scholar
  13. 13.
    Criminisi, A., Sharp, T., Rother, C., Pérez, P.: Geodesic image and video editing. ACM Trans. Graph. 29(5), 134 (2010). doi: 10.1145/1857907.1857910 CrossRefGoogle Scholar
  14. 14.
    Didas, S., Weickert, J.: Combining curvature motion and edge-preserving denoising. In: Proc. SSVM 2007. LNCS, vol. 4485, pp. 568–579. Springer, Berlin (2007). doi: 10.1007/978-3-540-72823-8 Google Scholar
  15. 15.
    Fabbri, R., Costa, L.D.F., Torelli, J.C., Bruno, O.M.: 2D Euclidean distance transform algorithms. ACM Comput. Surv. 40(1), 2 (2008). doi: 10.1145/1322432.1322434 CrossRefGoogle Scholar
  16. 16.
    Fischer, J., Bartz, D., Straber, W.: Stylized augmented reality for improved immersion. In: Proc. VR, pp. 195–202 (2005). doi: 10.1109/VR.2005.1492774 Google Scholar
  17. 17.
    Gastal, E.S.L., Oliveira, M.M.: Domain transform for edge-aware image and video processing. ACM Trans. Graph. 30(4), 69 (2011). doi: 10.1145/2010324.1964964 CrossRefGoogle Scholar
  18. 18.
    Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn. Prentice Hall, New York (2006) Google Scholar
  19. 19.
    Gooch, B., Reinhard, E., Gooch, A.: Human facial illustrations: Creation and psychophysical evaluation. ACM Trans. Graph. 23(1), 27–44 (2004). doi: 10.1145/966131.966133 CrossRefGoogle Scholar
  20. 20.
    Grayson, M.A.: The heat equation shrinks embedded plane curves to round points. J. Differ. Geom. 26(2), 285–314 (1987) MathSciNetzbMATHGoogle Scholar
  21. 21.
    Guichard, F., Morel, J.M.: A note on two classical enhancement filters and their associated PDE’s. Int. J. Comput. Vis. 52(2), 153–160 (2003). doi: 10.1023/A:1022904124348 CrossRefGoogle Scholar
  22. 22.
    Haralick, R.M.: Digital step edges from zero crossing of second directional derivatives. IEEE Trans. Pattern Anal. Mach. Intell. 6(1), 58–68 (1984). doi: 10.1109/TPAMI.1984.4767475 CrossRefGoogle Scholar
  23. 23.
    Haralick, R.M., Sternberg, S.R., Zhuang, X.: Image analysis using mathematical morphology. IEEE Trans. Pattern Anal. Mach. Intell. 9(4), 532–550 (1987). doi: 10.1109/TPAMI.1987.4767941 CrossRefGoogle Scholar
  24. 24.
    Kang, H., Lee, S.: Shape-simplifying image abstraction. Comput. Graph. Forum 27(7), 1773–1780 (2008). doi: 10.1111/j.1467-8659.2008.01322.x CrossRefGoogle Scholar
  25. 25.
    Kang, H., Lee, S., Chui, C.K.: Coherent line drawing. In: Proc. NPAR, pp. 43–50 (2007). doi: 10.1145/1274871.1274878 CrossRefGoogle Scholar
  26. 26.
    Kang, H., Lee, S., Chui, C.K.: Flow-based image abstraction. IEEE Trans. Vis. Comput. Graph. 15(1), 62–76 (2009). doi: 10.1109/TVCG.2008.81 CrossRefGoogle Scholar
  27. 27.
    Kim, D., Son, M., Lee, Y., Kang, H., Lee, S.: Feature-guided image stippling. Comput. Graph. Forum 27(4), 1209–1216 (2008). doi: 10.1111/j.1467-8659.2008.01259.x CrossRefGoogle Scholar
  28. 28.
    Kramer, H.P., Bruckner, J.B.: Iterations of a non-linear transformation for enhancement of digital images. Pattern Recognit. 7(1–2), 53–58 (1975) MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Kuwahara, M., Hachimura, K., Ehiu, S., Kinoshita, M.: Processing of ri-angiocardiographic images. In: Digital Processing of Biomedical Images, pp. 187–203. Plenum, New York (1976) CrossRefGoogle Scholar
  30. 30.
    Kyprianidis, J.E.: Image and video abstraction by multi-scale anisotropic Kuwahara filtering. In: Proc. NPAR, pp. 55–64 (2011). doi: 10.1145/2024676.2024686 Google Scholar
  31. 31.
    Kyprianidis, J.E., Döllner, J.: Image abstraction by structure adaptive filtering. In: Proc. EG UK TPCG, pp. 51–58 (2008). doi: 10.2312/LocalChapterEvents/TPCG/TPCG08/051-058 Google Scholar
  32. 32.
    Kyprianidis, J.E., Döllner, J.: Real-time image abstraction by directed filtering. In: ShaderX7, pp. 285–302. Charles River Media, London (2009) Google Scholar
  33. 33.
    Kyprianidis, J.E., Kang, H.: Image and video abstraction by coherence-enhancing filtering. Comput. Graph. Forum 30(2), 593–602 (2011). doi: 10.1111/j.1467-8659.2011.01882.x CrossRefGoogle Scholar
  34. 34.
    Kyprianidis, J.E., Kang, H., Döllner, J.: Image and video abstraction by anisotropic Kuwahara filtering. Comput. Graph. Forum 28(7), 1955–1963 (2009). doi: 10.1111/j.1467-8659.2009.01574.x CrossRefGoogle Scholar
  35. 35.
    Kyprianidis, J.E., Kang, H., Döllner, J.: Anisotropic Kuwahara filtering on the GPU. In: GPUPro, pp. 247–264. AK Peters, Wellesley (2010) Google Scholar
  36. 36.
    Kyprianidis, J.E., Semmo, A., Kang, H., Döllner, J.: Anisotropic Kuwahara filtering with polynomial weighting functions. In: Proc. EG UK TPCG, pp. 25–30 (2010) Google Scholar
  37. 37.
    Lee, H., Seo, S., Ryoo, S., Yoon, K.: Directional texture transfer. In: Proc. NPAR, pp. 43–50 (2010). doi: 10.1145/1809939.1809945 Google Scholar
  38. 38.
    Maragos, P., Schafer, R.: Morphological filters—Part I: Their set-theoretic analysis and relations to linear shift-invariant filters. IEEE Trans. Acoust. Speech Signal Process. 35(8), 1153–1169 (1987). doi: 10.1109/TASSP.1987.1165259 MathSciNetCrossRefGoogle Scholar
  39. 39.
    Maragos, P., Schafer, R.: Morphological filters—Part II: Their relations to median, order-statistic, and stack filters. IEEE Trans. Acoust. Speech Signal Process. 35(8), 1170–1184 (1987). doi: 10.1109/TASSP.1987.1165254 MathSciNetCrossRefGoogle Scholar
  40. 40.
    Marr, D., Hildreth, R.C.: Theory of edge detection. Proc. R. Soc. Lond. B, Biol. Sci. 207, 187–217 (1980) CrossRefGoogle Scholar
  41. 41.
    Orzan, A., Bousseau, A., Barla, P., Thollot, J.: Structure-preserving manipulation of photographs. In: Proc. NPAR, pp. 103–110 (2007) CrossRefGoogle Scholar
  42. 42.
    Osher, S., Rudin, L.I.: Feature-oriented image enhancement using shock filters. SIAM J. Numer. Anal. 27(4), 919–940 (1990). doi: 10.1137/0727053 zbMATHCrossRefGoogle Scholar
  43. 43.
    Papari, G., Petkov, N.: Continuous glass patterns for painterly rendering. IEEE Trans. Image Process. 18(3), 652–664 (2009). doi: 10.1109/TIP.2008.2009800 MathSciNetCrossRefGoogle Scholar
  44. 44.
    Papari, G., Petkov, N., Campisi, P.: Artistic edge and corner enhancing smoothing. IEEE Trans. Image Process. 16(10), 2449–2462 (2007). doi: 10.1109/TIP.2007.903912 MathSciNetCrossRefGoogle Scholar
  45. 45.
    Paris, S., Kornprobst, P., Tumblin, J., Durand, F.: Bilateral filtering: theory and applications. Found. Trends Comput. Graph. Vis. 4(1), 7–73 (2009). doi: 10.1561/0600000020 Google Scholar
  46. 46.
    Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12(7), 629–639 (1990). doi: 10.1109/34.56205 CrossRefGoogle Scholar
  47. 47.
    Pham, T.Q., van Vliet, L.J.: Separable bilateral filtering for fast video preprocessing. In: Proc. ICME, pp. 454–457 (2005). doi: 10.1109/ICME.2005.1521458 Google Scholar
  48. 48.
    Porikli, F.: Constant time O(1) bilateral filtering. In: Proc. CVPR, pp. 1–8 (2008). doi: 10.1109/CVPR.2008.4587843 Google Scholar
  49. 49.
    Pratt, W.K.: Digital Image Processing, 3rd edn. Wiley, New York (2001). doi: 10.1002/0471221325 CrossRefGoogle Scholar
  50. 50.
    Son, M., Lee, Y., Kang, H., Lee, S.: Structure grid for directional stippling. Graph. Models 73(3), 74–87 (2011). doi: 10.1016/j.gmod.2010.12.001 CrossRefGoogle Scholar
  51. 51.
    Sýkora, D., Buriánek, J., Žára, J.: Colorization of black-and-white cartoons. Image Vis. Comput. 23(9), 767–782 (2005). doi: 10.1016/j.imavis.2005.05.010 CrossRefGoogle Scholar
  52. 52.
    Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proc. ICCV, pp. 839–846 (1998). doi: 10.1109/ICCV.1998.710815 Google Scholar
  53. 53.
    Torre, V., Poggio, T.A.: On edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(2), 147–163 (1986). doi: 10.1109/TPAMI.1986.4767769 CrossRefGoogle Scholar
  54. 54.
    van den Boomgaard, R.: Decomposition of the Kuwahara–Nagao operator in terms of linear smoothing and morphological sharpening. In: Proc. ISMM, pp. 283–292. CSIRO, Collingwood (2002) Google Scholar
  55. 55.
    Weickert, J.: Anisotropic Diffusion in Image Processing. Teubner, Leipzig (1998) zbMATHGoogle Scholar
  56. 56.
    Weickert, J.: Coherence-enhancing diffusion of colour images. Image Vis. Comput. 17(3), 201–212 (1999) CrossRefGoogle Scholar
  57. 57.
    Weickert, J.: Coherence-enhancing shock filters. In: DAGM-Symposium, pp. 1–8. Springer, Berlin (2003). doi: 10.1007/978-3-540-45243-0_1 Google Scholar
  58. 58.
    Wikipedia: Expressionism—Wikipedia, The Free Encyclopedia (2012) Google Scholar
  59. 59.
    Winnemöller, H.: XDoG: Advanced image stylization with eXtended difference-of-Gaussians. In: Proc. NPAR, pp. 147–155 (2011). doi: 10.1145/2024676.2024700 Google Scholar
  60. 60.
    Winnemöller, H., Olsen, S.C., Gooch, B.: Real-time video abstraction. In: Proc. SIGGRAPH, pp. 1221–1226 (2006). doi: 10.1145/1141911.1142018 Google Scholar
  61. 61.
    Winnemöller, H., Kyprianidis, J.E., Olsen, S.C.: XDoG: an extended difference-of-Gaussians compendium including advanced image stylization. Comput. Graph. 36(6), 740–753 (2012). doi: 10.1016/j.cag.2012.03.004 CrossRefGoogle Scholar
  62. 62.
    Wyszecki, G., Stiles, W.S.: Color Science: Concepts and Methods, Quantitative Data and Formulae. Wiley-Interscience, New York (1982) Google Scholar
  63. 63.
    Young, R.A.: The Gaussian derivative model for spatial vision: I. Retinal mechanisms. Spat. Vis. 2(4), 273–293 (1987). doi: 10.1163/156856887X00222 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  1. 1.Hasso-Plattner-InstitutUniversity of PotsdamPotsdamGermany

Personalised recommendations