Estimating Comic Content from the Book Cover Information Using Fine-Tuned VGG Model for Comic Search

  • Byeongseon ParkEmail author
  • Mitsunori Matsushita
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11296)


The purpose of this research is to realize retrieval of comic based on content information. Resources of the contents information of existing comics were only the comics itself and review. However, these pieces of information have drawbacks that they can not sufficiently extract information necessary for searching, and that they contain a lot of unnecessary information. In order to solve this problem, we proposed to use the book cover of comics as a resource to grasp the contents of comics. In the proposed method, we estimate the age and cultural background of comics expressed by clothes and belongings written on the cover of comics from the reasoning model which performed fine-tuning from the VGG-16 model. Also, we associated comics with each other based on the obtained semantic vectors and tags. As a result of the experiment, the accuracy of the model was 0.693, and the reproducibility of the tag to the correct data was 0.918. Furthermore, we observed unity in the comics related by the obtained information.


Content estimation Transfer learning Comic computing 



The authors would like to thank S. Inoue, Y. Baba and Y. Higuchi for assistance with the data collection.


  1. 1.
    Park, B., Okamoto, K., Yamashita, R., Matsushita, M.: Designing a comic exploration system using a hierarchical topic classification of reviews. Inf. Eng. Express Int. Instit. Appl. Inform. 3(2), 45–57 (2017)Google Scholar
  2. 2.
    Rigaud, C., Gurin, C., Karatzas, D., Burie, J.C., Ogier, J.M.: Knowledge-driven understanding of images in comic books. IJDAR 18(3), 199–221 (2015)CrossRefGoogle Scholar
  3. 3.
    Tanaka, T., Shoji, K., Toyama, F., Miyamichi, J.: Layout analysis of tree-structured scene frames in comic images. In: Proceedings of the 20th International Joint Conference on Artificial Intelligence, IJCAI, Hyderabad, pp. 2885–2890 (2007)Google Scholar
  4. 4.
    Arai, K., Tolle H.: Method for automatic E-comic scene frame extraction for reading comic on mobile devices. In: Seventh International Conference on Information Technology, pp. 370–375. IEEE, Las Vegas (2010)Google Scholar
  5. 5.
    Chu, W-T., Li, W-W.: Manga FaceNet: face detection in manga based on deep neural network. In: 17th Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval, ICMR, Bucharest, pp. 412–415 (2017)Google Scholar
  6. 6.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE, pp. 2278–2324. IEEE (1998)Google Scholar
  7. 7.
    Blei, D.M., Andrew, Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)zbMATHGoogle Scholar
  8. 8.
    Blei, D.M.: Probabilistic topic models. Comun. ACM 55(4), 77–84 (2012)CrossRefGoogle Scholar
  9. 9.
    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE, Miami (2009)Google Scholar
  10. 10.
    Lin, M., Chen, Q., Yan, S.: Network in network. In: International Conference on Learning Representations, San Diego, (2014)Google Scholar
  11. 11.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations, San Diego (2015)Google Scholar
  12. 12.
    Saito, M., Matsui, Y.: Illustration2Vec: a semantic vector representation of illustrations. In: SIGGRAPH ASIA 2015 Technical Briefs, No. 5 (2015)Google Scholar
  13. 13.
    Islam, A., Inkpen, D.: Semantic text similarity using corpus-based word similarity and string similarity. ACM Trans. Knowl. Discov. Data 2(2), 10 (2008)CrossRefGoogle Scholar
  14. 14.
    Cha, S.H.: Comprehensive survey on distance/similarity measures between probability density functions. Int. J. Math. Models Methods Appl. Sci. 1(4), 300–307 (2007)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Graduate School of InformaticsKansai UniversityTakatsuki-shi, OsakaJapan

Personalised recommendations