Advertisement

A First Summarization System of a Video in a Target Language

  • Kamel SmaïliEmail author
  • Dominique Fohr
  • Carlos-Emiliano González-Gallardo
  • Michał Grega
  • Lucjan Janowski
  • Denis Jouvet
  • Artur Komorowski
  • Arian Koźbiał
  • David Langlois
  • Mikołaj Leszczuk
  • Odile Mella
  • Mohamed A. Menacer
  • Amaia Mendez
  • Elvys Linhares Pontes
  • Eric SanJuan
  • Damian Świst
  • Juan-Manuel Torres-Moreno
  • Begona Garcia-Zapirain
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 833)

Abstract

In this paper, we present the first results of the project AMIS (Access Multilingual Information opinionS) funded by Chist-Era. The main goal of this project is to understand the content of a video in a foreign language. In this work, we consider the understanding process, such as the aptitude to capture the most important ideas contained in a media expressed in a foreign language. In other words, the understanding will be approached by the global meaning of the content of a support and not by the meaning of each fragment of a video.

Several stumbling points remain before reaching the fixed goal. They concern the following aspects: Video summarization, Speech recognition, Machine translation and Speech segmentation. All these issues will be discussed and the methods used to develop each of these components will be presented. A first implementation is achieved and each component of this system is evaluated on a representative test data. We propose also a protocol for a global subjective evaluation of AMIS.

Keywords

Video summarization Speech recognition Machine translation Text boundary segmentation Text summarization Sentence compression 

Notes

Acknowledgment

We would like to acknowledge the support of Chist-Era for funding this work through the AMIS (Access Multilingual Information opinionS) project. Research work funded by the National Science Center, Poland, conferred on the basis of the decision number DEC-2015/16/Z/ST7/00559.

References

  1. 1.
    Baran, R., Zeja, A.: The IMCOP system for data enrichment and content discovery and delivery. In: 2015 International Conference on Computational Science and Computational Intelligence (CSCI), pp. 143–146, December 2015.  https://doi.org/10.1109/CSCI.2015.137
  2. 2.
    Bell, P., Lai, C., Llewellyn, C., Birch, A., Sinclair, M.: A system for automatic broadcast news summarisation, geolocation and translation. In: INTERSPEECH, pp. 730–731 (2015)Google Scholar
  3. 3.
    Choukri, K., Nikkhou, M., Paulsson, N.: Network of data centres (NetDc): BNSC-an Arabic broadcast news speech corpus. In: LREC (2004)Google Scholar
  4. 4.
    Christensen, H., Kolluru, B., Gotoh, Y., Renals, S.: From text summarisation to style-specific summarisation for broadcast news. In: European Conference on Information Retrieval, pp. 223–237. Springer (2004)Google Scholar
  5. 5.
    Furui, S., Kikuchi, T., Shinnaka, Y., Hori, C.: Speech-to-text and speech-to-speech summarization of spontaneous speech. IEEE Trans. Speech Audio Process. 12(4), 401–408 (2004)CrossRefGoogle Scholar
  6. 6.
    Gales, M.J.: Maximum likelihood linear transformations for hmm-based speech recognition. Comput. Speech Lang. 12(2), 75–98 (1998)CrossRefGoogle Scholar
  7. 7.
    González-Gallardo, C.E., Torres-Moreno, J.M.: Sentence boundary detection for French with subword-level information vectors and convolutional neural networks. arXiv preprint arXiv:1802.04559 (2018)
  8. 8.
    Gygli, M., Grabner, H., Gool, L.V.: Video summarization by learning submodular mixtures of objectives. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3090–3098, June 2015.  https://doi.org/10.1109/CVPR.2015.7298928
  9. 9.
    Huang, M., Mahajan, A.B., Dementhon, D.F.: Automatic performance evaluation for video summarization. Technical reportGoogle Scholar
  10. 10.
    Jouvet, D., Langlois, D., Menacer, M.A., Fohr, D., Mella, O., Smaïli, K.: Adaptation of speech recognition vocabularies for improved transcription of YouTube videos. In: Proceedings of the ICNLSSP Conference (2017)Google Scholar
  11. 11.
    Leszczuk, M., Grega, M., Koźbiał, A., Gliwski, J., Wasieczko, K., Smaïli, K.: Video summarization framework for newscasts and reports - work in progress. In: Dziech, A., Czyżewski, A. (eds.) Multimedia Communications, Services and Security, pp. 86–97. Springer International Publishing, Cham (2017)CrossRefGoogle Scholar
  12. 12.
    Linhares Pontes, E., Huet, S., Linhares, A.C., Torres-Moreno, J.M.: Multi-sentence compression with word vertex-labeled graphs and integer linear programming. In: Proceedings of TextGraphs-12: The Workshop on Graph-based Methods for Natural Language Processing. Association for Computational Linguistics (2018)Google Scholar
  13. 13.
    Liu, Y., Chawla, N.V., Harper, M.P., Shriberg, E., Stolcke, A.: A study in machine learning from imbalanced data for sentence boundary detection in speech. Comput. Speech Lang. 20(4), 468–494 (2006)CrossRefGoogle Scholar
  14. 14.
    Maegaard, B., Choukri, K., Jørgensen, L.D., Krauwer, S.: NEMLAR: Arabic language resources and tools. In: Arabic Language Resources and Tools Conference, pp. 42–54 (2004)Google Scholar
  15. 15.
    Menacer, M.A., Langlois, D., Mella, O., Fohr, D., Jouvet, D., Smaïli, K.: Is statistical machine translation approach dead? In: ICNLSSP 2017 - International Conference on Natural Language, Signal and Speech Processing, pp. 1–5. ISGA, Casablanca, December 2017. https://hal.inria.fr/hal-01660016
  16. 16.
    Menacer, M.A., Mella, O., Fohr, D., Jouvet, D., Langlois, D., Smaïli, K.: Development of the Arabic loria automatic speech recognition system (ALASR) and its evaluation for Algerian dialect. In: ACLing 2017 - 3rd International Conference on Arabic Computational Linguistics, Dubai, United Arab Emirates, pp. 1–8, November 2017. https://hal.archives-ouvertes.fr/hal-01583842
  17. 17.
    Mohri, M., Pereira, F., Riley, M.: Speech recognition with weighted finite-state transducers. In: Springer Handbook of Speech Processing, pp. 559–584. Springer (2008)Google Scholar
  18. 18.
    Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., Hannemann, M., Motlicek, P., Qian, Y., Schwarz, P., Silovsky, J., Stemmer, G., Vesely, K.: The kaldi speech recognition toolkit. In: IEEE 2011 Workshop on Automatic Speech Recognition and Understanding. IEEE Signal Processing Society, December 2011. IEEE Catalog No.: CFP11SRW-USBGoogle Scholar
  19. 19.
    Quemy, A., Jamrog, K., Janiszewski, M.: Unsupervised video semantic partitioning using IBM watson and topic modelling. In: Proceedings of the Workshops of the EDBT/ICDT 2018 Joint Conference (EDBT/ICDT 2018), pp. 44–49, March 2018Google Scholar
  20. 20.
    Sharghi, A., Laurel, J.S., Gong, B.: Query-focused video summarization: dataset, evaluation, and a memory network based approach. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017, pp. 2127–2136. IEEE Computer Society (2017).  https://doi.org/10.1109/CVPR.2017.229
  21. 21.
    Stolcke, A.: Entropy-based pruning of backoff language models. arXiv preprint cs/0006025 (2000)Google Scholar
  22. 22.
    Torres-Moreno, J.M.: Artex is another text summarizer. arXiv preprint arXiv:1210.3312 (2012)
  23. 23.
    Torres-Moreno, J.M.: Automatic Text Summarization. Wiley, London (2014)CrossRefGoogle Scholar
  24. 24.
    Veselỳ, K., Ghoshal, A., Burget, L., Povey, D.: Sequence-discriminative training of deep neural networks. In: Interspeech 2013 (2013)Google Scholar
  25. 25.
    Zhang, J.J., Fung, P.: Active learning with semi-automatic annotation for extractive speech summarization. ACM Trans. Speech Lang. Process. (TSLP) 8(4), 6 (2012)Google Scholar
  26. 26.
    Ziemski, M., Junczys-Dowmunt, M., Pouliquen, B.: The united nations parallel corpus v1. 0. In: LREC (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Kamel Smaïli
    • 1
    Email author
  • Dominique Fohr
    • 1
  • Carlos-Emiliano González-Gallardo
    • 2
  • Michał Grega
    • 3
  • Lucjan Janowski
    • 3
  • Denis Jouvet
    • 1
  • Artur Komorowski
    • 3
  • Arian Koźbiał
    • 3
  • David Langlois
    • 1
  • Mikołaj Leszczuk
    • 3
  • Odile Mella
    • 1
  • Mohamed A. Menacer
    • 1
  • Amaia Mendez
    • 4
  • Elvys Linhares Pontes
    • 2
  • Eric SanJuan
    • 2
  • Damian Świst
    • 3
  • Juan-Manuel Torres-Moreno
    • 2
    • 5
  • Begona Garcia-Zapirain
    • 4
  1. 1.Loria University of LorraineNancyFrance
  2. 2.LIA Université d’Avignon et des Pays de VaucluseAvignonFrance
  3. 3.AGH University of Science and Technology KrakówKrakówPoland
  4. 4.University of DEUSTO BilbaoBilbaoSpain
  5. 5.Ecole Polytechnique de MontréalMontrealCanada

Personalised recommendations