Detecting Recurring Deformable Objects: An Approximate Graph Matching Method for Detecting Characters in Comics Books

  • Hoang Nam Ho
  • Christophe Rigaud
  • Jean-Christophe Burie
  • Jean-Marc Ogier
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8746)


Graphs are popular data structures used to model pair wise relations between elements from a given collection. In image processing, adjacency graphs are often used to represent the relations between segmented regions. The comparison of such graphs has been largely studied but graph matching strategies are essential to find, efficiently, similar patterns. In this paper, we propose a method to detect the recurring characters in comics books. We would like to draw attention of the reader. In this paper, the term “character” means the protagonists of the story. In our approach, each panel is represented with an attributed adjacency graph. Then, an inexact graph matching strategy is applied to find recurring structures among this set of graphs. The main idea is that the same character will be represented by similar subgraphs in the different panels where it appears. The two-step matching process consists in a node matching step and an edge validation step. Experiments show that our approach is able to detect recurring structures in the graph and consequently the recurrent characters in a comics book. The originality of our approach is that no prior object model is required the characters. The algorithm detects, automatically, all recurring structures corresponding to the main characters of the story.


Comics Character detection Attributed adjacency graph Approximate graph matching Spatial relation 



This work was supported by the European Regional Development Fund, the region Poitou-Charentes (France), the General Council of Charente Maritime (France) and the town of La Rochelle (France).


  1. 1.
    Commission Internationale de l’Eclairage, Colorimetry, CIE 15.2 (1986)Google Scholar
  2. 2.
    Arai, K., Tolle, H.: Automatic e-comic content adaptation. Int. J. Ubiquitous Comput. (IJUC) 1(1), 1–11 (2010)Google Scholar
  3. 3.
    Bunke, H., Riesen, K.: Recent advances in graph-based pattern recognition with applications in document analysis. Pattern Recogn. 44(5), 1057–1067 (2011)CrossRefzbMATHGoogle Scholar
  4. 4.
    Bunke, H.: Error-tolerant graph matching: a formal framework and algorithms. In: Amin, A., Pudil, P., Dori, D. (eds.) SPR 1998 and SSPR 1998. LNCS, vol. 1451, pp. 1–14. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  5. 5.
    Cyb: Cosmozone, vol. 1. Studio Cyborga (2009)Google Scholar
  6. 6.
    Fischer, A., Suen, C.Y., Frinken, V., Riesen, K., Bunke, H.: A fast matching algorithm for graph-based handwriting recognition. In: Kropatsch, W.G., Artner, N.M., Haxhimusa, Y., Jiang, X. (eds.) GbRPR 2013. LNCS, vol. 7877, pp. 194–203. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  7. 7.
    Chan, C.H., Leung, H., Komura, T.: Automatic panel extraction of color comic images. In: Ip, H.H.-S., Au, O.C., Leung, H., Sun, M.-T., Ma, W.-Y., Hu, S.-M. (eds.) PCM 2007. LNCS, vol. 4810, pp. 775–784. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  8. 8.
    Ho, A.K.N., Burie, J.C., Ogier, J.M.: Panel and speech balloon extraction from comic books. In: DAS 2012, Tenth IAPR International Workshop on Document Analysis Systems, Gold Coast, Australia (2012)Google Scholar
  9. 9.
    Hu, M.K.: Visual pattern recognition by moment invariants. IRE Trans. Inf. Theor. 8(2), 179–187 (1962)CrossRefzbMATHGoogle Scholar
  10. 10.
  11. 11.
    Inokuchi, A., Washio, T., Motoda, H.: An apriori-based algorithm for mining frequent substructures from graph data. In: Zighed, D.A., Komorowski, J., Żytkow, J.M. (eds.) PKDD 2000. LNCS (LNAI), vol. 1910, pp. 13–23. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  12. 12.
    Ishii, D., Watanabe, H.: A study on frame position detection of digitized comics images. In: Proceedings of Workshop on Picture Coding and Image Processing, PCSJ2010/IMPS2010, Nagoya, Japan, December 2010, pp. 124–125 (2010)Google Scholar
  13. 13.
    Kuramochi, M., Karypis, G.: Frequent subgraph discovery. In: Proceedings of the 2001 IEEE International Conference on Data Mining, pp. 313–320 (2001)Google Scholar
  14. 14.
    Kuramochi, M., Karypis, G.: Finding frequent patterns in a large sparse graph. Data Min. Knowl. Discov. 11, 243–271 (2005)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Lopresti, D., Wilfong, G.: A fast technique for comparing graph representations with applications to performance evaluation. Int. J. Doc. Anal. Recogn. 6(4), 219–229 (2003)CrossRefGoogle Scholar
  16. 16.
    Luo, B., Wilson, R.C., Hancock, E.R.: Spectral embedding of graphs. Pattern Recogn. 36(10), 2210–2230 (2003)CrossRefGoogle Scholar
  17. 17.
    Luqman, M.M., Ramel, J.Y., Llads, J., Brouard, T.: Fuzzy multilevel graph embedding. Pattern Recogn. 46(2), 551–565 (2013)CrossRefzbMATHGoogle Scholar
  18. 18.
    Messmer, B.T., Bunke, H.: A new algorithm for error-tolerant subgraph isomorphism detection. IEEE Trans. Pattern Anal. Mach. Intell. 20(5), 493–504 (1998)CrossRefGoogle Scholar
  19. 19.
    Rigaud, C., Karatzas, D., Van de Weijer, J., Burie, J.C., Ogier, J.M.: Automatic text localisation in scanned comic books. In: Proceedings of the 8th International Conference on Computer Vision Theory and Applications (VISAPP), pp. 814–819 (2013)Google Scholar
  20. 20.
    Rigaud, C., Tsopze, N., Burie, J.-C., Ogier, J.-M.: Robust frame and text extraction from comic books. In: Kwon, Y.-B., Ogier, J.-M. (eds.) GREC 2011. LNCS, vol. 7423, pp. 129–138. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  21. 21.
    Shapiro, L., Haralick, R.: Structural descriptions and inexact matching. IEEE Trans. Pattern Anal. Mach. Intell. 3(5), 504–519 (1981)CrossRefGoogle Scholar
  22. 22.
    Shervashidze, N., Vishwanathan, S.V.N., Petri, T., Mehlhorn, K., Borgwardt, K.M.: Efficient graphlet kernels for large graph comparison. In: Twelfth International Conference on Artificial Intelligence and Statistics, pp. 488–495 (2009)Google Scholar
  23. 23.
    Sidere, N., Héroux, P., Ramel, J.Y.: Embedding labeled graphs into occurence matrix. In: Proceedings of the IAPR Workshop on Graphics Recognition, GREC 2009, La Rochelle, France, pp. 44–50 (2009)Google Scholar
  24. 24.
    Sun, W., Kise, K.: Similar manga retrieval using visual vocabulary based on regions of interest. In: Proceedings of the 11th International Conference on Document Analysis and Recognition (ICDAR2011), October 2011, pp. 1075–1079 (2011)Google Scholar
  25. 25.
    Tanaka, T., Shoji, K., Toyama, F., Miyamichi, J.: Layout analysis of tree-structured scene frames in comic images. In: Proceedings of International Joint Conference on Artificial Intelligence, IJCAI-07, Hyderabad, India, January 2007, pp. 2885–2890 (2007)Google Scholar
  26. 26.
    Tsai, W.H., Fu, K.S.: Error-correcting isomorphisms of attributed relational graphs for pattern analysis. IEEE Trans. Syst. Man Cybern. 9(12), 757–768 (1979)CrossRefzbMATHGoogle Scholar
  27. 27.
    Yan, X., Han, J.: Span: graph-based substructure pattern mining. In: Proceedings of the 2002 IEEE International Conference on Data Mining, pp. 721–725 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Hoang Nam Ho
    • 1
  • Christophe Rigaud
    • 1
  • Jean-Christophe Burie
    • 1
  • Jean-Marc Ogier
    • 1
  1. 1.L3i LaboratoryUniversity of La RochelleLa Rochelle Cedex 1France

Personalised recommendations