Advertisement

Finding Person Relations in Image Data of News Collections in the Internet Archive

  • Eric Müller-Budack
  • Kader Pustu-Iren
  • Sebastian Diering
  • Ralph Ewerth
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11057)

Abstract

The amount of multimedia content in the World Wide Web is rapidly growing and contains valuable information for many applications in different domains. The Internet Archive initiative has gathered billions of time-versioned web pages since the mid-nineties. However, the huge amount of data is rarely labeled with appropriate metadata and automatic approaches are required to enable semantic search. Normally, the textual content of the Internet Archive is used to extract entities and their possible relations across domains such as politics and entertainment, whereas image and video content is usually disregarded. In this paper, we introduce a system for person recognition in image content of web news stored in the Internet Archive. Thus, the system complements entity recognition in text and allows researchers and analysts to track media coverage and relations of persons more precisely. Based on a deep learning face recognition approach, we suggest a system that detects persons of interest and gathers sample material, which is subsequently used to identify them in the image data of the Internet Archive. We evaluate the performance of the face recognition system on an appropriate standard benchmark dataset and demonstrate the feasibility of the approach with two use cases.

Keywords

Deep learning Face recognition Internet Archive Big data application 

Notes

Acknowledgement

This work is financially supported by the German Research Foundation (DFG: Deutsche Forschungsgemeinschaft, project number: EW 134/4-1). The work was partially funded by the European Commission for the ERC Advanced Grant ALEXANDRIA (No. 339233, Wolfgang Nejdl).

References

  1. 1.
    Abadi, M., et al.: Tensorflow: large-scale machine learning on heterogeneous distributed systems. CoRR abs/1603.04467 (2016)Google Scholar
  2. 2.
    Best-Rowden, L., Jain, A.K.: Longitudinal study of automatic face recognition. Trans. Pattern Anal. Mach. Intell. 40, 148–162 (2018)CrossRefGoogle Scholar
  3. 3.
    Brambilla, M., Ceri, S., Della Valle, E., Volonterio, R., Acero Salazar, F.X.: Extracting emerging knowledge from social media. In: International Conference on World Wide Web, pp. 795–804. IW3C2 (2017)Google Scholar
  4. 4.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Conference on Computer Vision and Pattern Recognition, pp. 886–893. IEEE (2005)Google Scholar
  5. 5.
    Ding, C., Tao, D.: Trunk-branch ensemble convolutional neural networks for video-based face recognition. Trans. Pattern Anal. Mach. Intell. 40, 1002–1014 (2017)CrossRefGoogle Scholar
  6. 6.
    Gangemi, A., Presutti, V., Reforgiato Recupero, D., Nuzzolese, A.G., Draicchio, F., Mongiovì, M.: Semantic web machine reading with FRED. Semant. Web 8(6), 873–893 (2017)CrossRefGoogle Scholar
  7. 7.
    Guo, Y., Zhang, L., Hu, Y., He, X., Gao, J.: MS-Celeb-1M: a dataset and benchmark for large-scale face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 87–102. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46487-9_6CrossRefGoogle Scholar
  8. 8.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Conference on Computer Vision and Pattern Recognition, pp. 770–778. IEEE (2016)Google Scholar
  9. 9.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report 07–49, University of Massachusetts, Amherst (2007)Google Scholar
  10. 10.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105. NIPS (2012)Google Scholar
  11. 11.
    Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: Sphereface: deep hypersphere embedding for face recognition. In: Conference on Computer Vision and Pattern Recognition, vol. 1. IEEE (2017)Google Scholar
  12. 12.
    Masi, I., et al.: Learning pose-aware models for pose-invariant face recognition in the wild. Trans. Pattern Anal. Mach. Intell. (2018)Google Scholar
  13. 13.
    Masi, I., Hassner, T., Tran, A.T., Medioni, G.: Rapid synthesis of massive face sets for improved face recognition. In: International Conference on Automatic Face & Gesture Recognition, pp. 604–611. IEEE (2017)Google Scholar
  14. 14.
    Masi, I., Rawls, S., Medioni, G., Natarajan, P.: Pose-aware face recognition in the wild. In: Conference on Computer Vision and Pattern Recognition, pp. 4838–4846. IEEE (2016)Google Scholar
  15. 15.
    Masi, I., Tran, A.T., Hassner, T., Leksut, J.T., Medioni, G.: Do we really need to collect millions of faces for effective face recognition? In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 579–596. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46454-1_35CrossRefGoogle Scholar
  16. 16.
    Moro, A., Raganato, A., Navigli, R.: Entity linking meets word sense disambiguation: a unified approach. Trans. Assoc. Comput. Linguist. 2, 231–244 (2014)Google Scholar
  17. 17.
    Navigli, R., Ponzetto, S.P.: BabelNet: the automatic construction, evaluation and application of a wide-coverage multilingual semantic network. Artif. Intell. 193, 217–250 (2012)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: Conference on Computer Vision and Pattern Recognition, pp. 815–823. IEEE (2015)Google Scholar
  19. 19.
    Sun, Y., Wang, X., Tang, X.: Deep learning face representation from predicting 10,000 classes. In: Conference on Computer Vision and Pattern Recognition, pp. 1891–1898. IEEE (2014)Google Scholar
  20. 20.
    Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: DeepFace: closing the gap to human-level performance in face verification. In: Conference on Computer Vision and Pattern Recognition, pp. 1701–1708. IEEE (2014)Google Scholar
  21. 21.
    Van Erp, M., Rizzo, G., Troncy, R.: Learning with the web: Spotting named entities on the intersection of NERD and machine learning. In: Workshop on Making Sense of Microposts, pp. 27–30 (2013)Google Scholar
  22. 22.
    Wen, Y., Li, Z., Qiao, Y.: Latent factor guided convolutional neural networks for age-invariant face recognition. In: Conference on Computer Vision and Pattern Recognition, pp. 4893–4901. IEEE (2016)Google Scholar
  23. 23.
    Yang, S., Luo, P., Loy, C.C., Tang, X.: From facial parts responses to face detection: a deep learning approach. In: International Conference on Computer Vision, pp. 3676–3684. IEEE (2015)Google Scholar
  24. 24.
    Yi, D., Lei, Z., Liao, S., Li, S.Z.: Learning face representation from scratch. CoRR abs/1411.7923 (2014)Google Scholar
  25. 25.
    Yin, X., Yu, X., Sohn, K., Liu, X., Chandraker, M.: Towards large-pose face frontalization in the wild. CoRR abs/1704.06244 (2017)Google Scholar
  26. 26.
    Zhu, X., Lei, Z., Liu, X., Shi, H., Li, S.Z.: Face alignment across large poses: a 3D solution. In: Conference on Computer Vision and Pattern Recognition, pp. 146–155. IEEE (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Leibniz Information Centre for Science and Technology (TIB)HannoverGermany
  2. 2.L3S Research CenterLeibniz Universität HannoverHannoverGermany

Personalised recommendations