Advertisement

DanbooRegion: An Illustration Region Dataset

Conference paper
  • 433 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12358)

Abstract

Region is a fundamental element of various cartoon animation techniques and artistic painting applications. Achieving satisfactory region is essential to the success of these techniques. Motivated to assist diversiform region-based cartoon applications, we invite artists to annotate regions for in-the-wild cartoon images with several application-oriented goals: (1) To assist image-based cartoon rendering, relighting, and cartoon intrinsic decomposition literature, artists identify object outlines and eliminate lighting-and-shadow boundaries. (2) To assist cartoon inking tools, cartoon structure extraction applications, and cartoon texture processing techniques, artists clean-up texture or deformation patterns and emphasize cartoon structural boundary lines. (3) To assist region-based cartoon digitalization, clip-art vectorization, and animation tracking applications, artists inpaint and reconstruct broken or blurred regions in cartoon images. Given the typicality of these involved applications, this dataset is also likely to be used in other cartoon techniques. We detail the challenges in achieving this dataset and present a human-in-the-loop workflow namely Feasibility-based Assignment Recommendation (FAR) to enable large-scale annotating. The FAR tends to reduce artist trails-and-errors and encourage their enthusiasm during annotating. Finally, we present a dataset that contains a large number of artistic region compositions paired with corresponding cartoon illustrations. We also invite multiple professional artists to assure the quality of each annotation.

Keywords

Artistic creation Fine art Cartoon Region processing 

Supplementary material

504454_1_En_9_MOESM1_ESM.zip (16.9 mb)
Supplementary material 1 (zip 17353 KB)

References

  1. 1.
    Arbeláez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011)CrossRefGoogle Scholar
  2. 2.
    Bell, S., Bala, K., Snavely, N.: Intrinsic images in the wild. ACM Trans. Graph. (SIGGRAPH) 33(4), 1–12 (2014)CrossRefGoogle Scholar
  3. 3.
    Bell, S., Upchurch, P., Snavely, N., Bala, K.: OpenSurfaces: a richly annotated catalog of surface appearance. ACM Trans. Graph. (SIGGRAPH) 32(4), 1–17 (2013)CrossRefGoogle Scholar
  4. 4.
    Bessmeltsev, M., Solomon, J.: Vectorization of line drawings via polyvector fields. ACM Trans. Graph. 38(1), 1–12 (2019).  https://doi.org/10.1145/3202661CrossRefGoogle Scholar
  5. 5.
    Bi, S., Han, X., Yu, Y.: An L1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition. ACM Trans. Graph. 34(4) (2015).  https://doi.org/10.1145/2766946. https://doi.org/10.1145/2766946
  6. 6.
    Branson, S., Van Horn, G., Perona, P.: Lean crowdsourcing: combining humans and machines in an online system. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017Google Scholar
  7. 7.
    caydett: Cel shading tutorial by caydett (2018). https://www.deviantart.com/caydett/art/Cel-Shading-Tutorial-270935090
  8. 8.
    Chattopadhay, A., Sarkar, A., Howlader, P., Balasubramanian, V.N.: GradCAM: generalized gradient-based visual explanations for deep convolutional networks. In: WACV. IEEE, March 2018.  https://doi.org/10.1109/wacv.2018.00097
  9. 9.
    Chen, Q., Koltun, V.: Photographic image synthesis with cascaded refinement networks. In: ICCV (2017)Google Scholar
  10. 10.
    Chen, X., Girshick, R., He, K., Dollar, P.: TensorMask: a foundation for dense object segmentation. In Arxiv (2019)Google Scholar
  11. 11.
    Cho, H., Lee, H., Kang, H., Lee, S.: Bilateral texture filtering. ACM Trans. Graph. 33(4), 1–8 (2014).  https://doi.org/10.1145/2601097.2601188CrossRefGoogle Scholar
  12. 12.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002).  https://doi.org/10.1109/34.1000236CrossRefGoogle Scholar
  13. 13.
    Cour, T., Benezit, F., Shi, J.: Spectral segmentation with multiscale graph decomposition. In: CVPR. IEEE (2005).  https://doi.org/10.1109/cvpr.2005.332
  14. 14.
    Dalstein, B., Ronfard, R., van de Panne, M.: Vector graphics animation with time-varying topology. ACM Trans. Graph. 34(4) (2015).  https://doi.org/10.1145/2766913
  15. 15.
    DanbooruCommunity: Danbooru 2017: a large-scale crowdsourced and tagged anime illustration dataset (2018)Google Scholar
  16. 16.
    Dvorožňák, M., Nejad, S.S., Jamriška, O., Jacobson, A., Kavan, L., Sýkora, D.: Seamless reconstruction of part-based high-relief models from hand-drawn images. In: Proceedings of International Symposium on Sketch-Based Interfaces and Modeling (2018)Google Scholar
  17. 17.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. IJCV 59, 167–181 (2004)CrossRefGoogle Scholar
  18. 18.
    Fourey, S., Tschumperle, D., Revoy, D.: A fast and efficient semi-guided algorithm for flat coloring line-arts. In: EUROGRAPHICS (2018)Google Scholar
  19. 19.
    Gao, C., Liu, Q., Xu, Q., Wang, L., Liu, J., Zou, C.: SketchyCOCO: image generation from freehand scene sketches. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020Google Scholar
  20. 20.
    Gao, S., et al.: A data-synthesis-driven method for detecting and extracting vague cognitive regions. Int. J. Geograph. Inf. Sci., 1–27 (2017).  https://doi.org/10.1080/13658816.2016.1273357
  21. 21.
    Garces, E., Agarwala, A., Gutierrez, D., Hertzmann, A.: A similarity measure for illustration style. ACM Trans. Graph. (SIGGRAPH 2014) 33(4), 1–9 (2014)CrossRefGoogle Scholar
  22. 22.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, June 2016.  https://doi.org/10.1109/cvpr.2016.90
  23. 23.
    Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, July 2017.  https://doi.org/10.1109/cvpr.2017.243
  24. 24.
    Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: CVPR (2017)Google Scholar
  25. 25.
    Kender, J.R., Smith, E.M.: Shape from Darkness: Deriving Surface Information from Dynamic Shadows, chap. 3, pp. 378–385. Jones and Bartlett Publishers Inc, USA (1992)Google Scholar
  26. 26.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. Computer Science (2014)Google Scholar
  27. 27.
    Lalonde, J.F., Hoiem, D., Efros, A.A., Rother, C., Winn, J., Criminisi, A.: Photo clip art. ACM Trans. Graph. (TOG) 26(3), 3 (2007).  https://doi.org/10.1145/1276377.1276381CrossRefGoogle Scholar
  28. 28.
    Li, C., Liu, X., Wong, T.T.: Deep extraction of manga structural lines. ACM Trans. Graph. 36(4), 1–12 (2017)CrossRefGoogle Scholar
  29. 29.
    Liu, C., Rosales, E., Sheffer, A.: Strokeaggregator: consolidating raw sketches into artist-intended curve drawings. ACM Trans. Graph. 37, 1–15 (2018)Google Scholar
  30. 30.
    Liu, X., Wong, T.T., Heng, P.A.: Closure-aware sketch simplification. ACM Trans. Graph. 34(6), 168:1–168:10 (2015)Google Scholar
  31. 31.
    MicahBuzan: Cel shading tutorial (2020). https://www.micahbuzan.com/cel-shading-tutorial/
  32. 32.
    Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33715-4_54CrossRefGoogle Scholar
  33. 33.
    Neubert, P., Protzel, P.: Compact watershed and preemptive SLIC: on improving trade-offs of superpixel segmentation algorithms. In: ICPR (2014)Google Scholar
  34. 34.
    Ren, H., Li, J., Gao, N.: Two-stage sketch colorization with color parsing. IEEE Access 8, 44599–44610 (2020)CrossRefGoogle Scholar
  35. 35.
    Rivière, M., Okabe, M.: Extraction of a cartoon topology. In: ACM SIGGRAPH 2014 Posters on - SIGGRAPH 2014. ACM Press (2014).  https://doi.org/10.1145/2614217.2614260
  36. 36.
    Ronchi, M.R., Perona, P.: Describing common human visual actions in images. In: Proceedings of the British Machine Vision Conference (BMVC), pp. 52.1–52.12. BMVA Press, September 2015.  https://doi.org/10.5244/C.29.52. https://dx.doi.org/10.5244/C.29.52
  37. 37.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  38. 38.
    Sangkloy, P., Lu, J., Fang, C., Yu, F., Hays, J.: Scribbler: controlling deep image synthesis with sketch and color. In: CVPR (2017)Google Scholar
  39. 39.
    Sharon, E., Galun, M., Sharon, D., Basri, R., Brandt, A.: Hierarchy and adaptivity in segmenting visual scenes. Nature 442(7104), 810–813 (2006).  https://doi.org/10.1038/nature04977CrossRefGoogle Scholar
  40. 40.
    Shugrina, M., et al.: Creative flow+ dataset. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019Google Scholar
  41. 41.
    Simo-Serra, E., Iizuka, S., Ishikawa, H.: Mastering sketching: adversarial augmentation for structured prediction. ACM Trans. Graph. 37(1), 1–13 (2018)CrossRefGoogle Scholar
  42. 42.
    Simo-Serra, E., Iizuka, S., Ishikawa, H.: Real-time data-driven interactive rough sketch inking. ACM Trans. Graph. 37, 1–14 (2018)Google Scholar
  43. 43.
    Simo-Serra, E., Iizuka, S., Sasaki, K., Ishikawa, H.: Learning to simplify: fully convolutional networks for rough sketch cleanup. ACM Trans. Graph. 35(4), 1–11 (2016)CrossRefGoogle Scholar
  44. 44.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: TPAMI (2014)Google Scholar
  45. 45.
    Sýkora, D., Buriánek, J., Žára, J.: Sketching cartoons by example. In: Proceedings of Eurographics Workshop on Sketch-Based Interfaces and Modeling, pp. 27–34 (2005)Google Scholar
  46. 46.
    Sykora, D., Dingliana, J., Collins, S.: LazyBrush: flexible painting tool for hand-drawn cartoons. Comput. Graph. Forum 28(2), 599–608 (2009)CrossRefGoogle Scholar
  47. 47.
    Sýkora, D., et al.: Ink-and-ray: bas-relief meshes for adding global illumination effects to hand-drawn characters. ACM Trans. Graph. 33(2), 16 (2014)CrossRefGoogle Scholar
  48. 48.
    TaiZan: Paintschainer tanpopo. PreferredNetwork (2016)Google Scholar
  49. 49.
    Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271). Narosa Publishing House (1998).  https://doi.org/10.1109/iccv.1998.710815
  50. 50.
    Uijlings, J.R.R., van de Sande, K.E.A., Gevers, T., Smeulders, A.W.M.: Selective search for object recognition. IJCV 104, 154–171 (2013)CrossRefGoogle Scholar
  51. 51.
    Wang, T.C., Liu, M.Y., Zhu, J.Y., Tao, A., Kautz, J., Catanzaro, B.: High-resolution image synthesis and semantic manipulation with conditional GANs. In: CVPR (2018)Google Scholar
  52. 52.
    Wilber, M.J., Fang, C., Jin, H., Hertzmann, A., Collomosse, J., Belongie, S.: Bam! the behance artistic media dataset for recognition beyond photography. In: The IEEE International Conference on Computer Vision (ICCV), October 2017Google Scholar
  53. 53.
    Xie, S., Tu, Z.: Holistically-nested edge detection. In: CVPR (2015)Google Scholar
  54. 54.
    Xu, L., Yan, Q., Xia, Y., Jia, J.: Structure extraction from texture via relative total variation. ACM Trans. Graph. 31(6), 1 (2012).  https://doi.org/10.1145/2366145.2366158CrossRefGoogle Scholar
  55. 55.
    Xu, N., Price, B., Cohen, S., Huang, T.: Deep image matting. In: CVPR. IEEE, July 2017.  https://doi.org/10.1109/cvpr.2017.41
  56. 56.
    Zhang, L., Li, C., Wong, T.T., Ji, Y., Liu, C.: Two-stage sketch colorization. ACM Trans. Graph. 37, 1–14 (2018)CrossRefGoogle Scholar
  57. 57.
    Zhang, S.H., Chen, T., Zhang, Y.F., Hu, S.M., Martin, R.R.: Vectorizing cartoon animations. TVCG 15, 618–629 (2009)Google Scholar
  58. 58.
    Zhang, T.Y., Suen, C.Y.: A fast parallel algorithm for thinning digital patterns. Commun. ACM 27, 236–239 (1984)CrossRefGoogle Scholar
  59. 59.
    Zhu, H., Liu, X., Wong, T.T., Heng, P.A.: Globally optimal toon tracking. ACM Trans. Graph. 35(4), 75:1–75:10 (2016)CrossRefGoogle Scholar
  60. 60.
    Zou, C., Mo, H., Gao, C., Du, R., Fu, H.: Language-based colorization of scene sketches. ACM Trans. Graph. (Proceedings of ACM SIGGRAPH Asia 2019) 38(6), 233:1–233:16 (2019)Google Scholar
  61. 61.
    Zou, C., et al.: SketchyScene: richly-annotated scene sketches. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11219, pp. 438–454. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01267-0_26CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Style2Paints ResearchSuzhouChina
  2. 2.Soochow UniversitySuzhouChina

Personalised recommendations