Advertisement

Cue-Phrase-Based Text Segmentation and Optimal Segment Concatenation for the NTCIR-14 QA Lab-PoliInfo Task

  • Katsumi KanasakiEmail author
  • Jiawei YongEmail author
  • Shintaro Kawamura
  • Shoichi Naitoh
  • Kiyohiko Shinomiya
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11966)

Abstract

The segmentation subtask of the NTCIR-14 QA Lab-PoliInfo task is finding a segment of text in assembly minutes that corresponds to a summary sentence. We divided the segmentation subtask into two steps, segmentation and search. Cue phrases were effectively used to detect segment boundaries. We compared five methods for detecting segment boundaries: a rule-based method, three supervised learning methods, and a novel semi-supervised learning method. The supervised models were trained using minutes data (in Japanese) we had segmented. In the search step, contiguous segments were concatenated to form larger segments, and the segment that maximized the value of a formula was selected as the answer. We compared the proposed formula with the conventional BM25 formula. We achieved the highest F-measure during the NTCIR-14 formal run despite our method’s simplicity.

Keywords

Text segmentation Semi-supervised method Hierarchical attention network Okapi BM25 

References

  1. 1.
    Agirre, E., Diab, M., Cer, D., Gonzalez-Agirre, A.: Semeval-2012 task 6: a pilot on semantic textual similarity. In: Proceedings of the First Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the Main Conference and the Shared Task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation, pp. 385–393. Association for Computational Linguistics (2012)Google Scholar
  2. 2.
    Badjatiya, P., Kurisinkel, L.J., Gupta, M., Varma, V.: Attention-based neural text segmentation. In: Pasi, G., Piwowarski, B., Azzopardi, L., Hanbury, A. (eds.) ECIR 2018. LNCS, vol. 10772, pp. 180–193. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-76941-7_14CrossRefGoogle Scholar
  3. 3.
    Beeferman, D., Berger, A., Lafferty, J.: Statistical models for text segmentation. Mach. Learn. 34(1–3), 177–210 (1999)CrossRefGoogle Scholar
  4. 4.
    Choi, F.Y.Y.: Advances in domain independent linear text segmentation. In: Proceedings of the 1st North American Chapter of the Association for Computational Linguistics Conference, pp. 26–33 (2000)Google Scholar
  5. 5.
    Eisenstein, J.: Hierarchical text segmentation from multi-scale lexical cohesion. In: Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 353–361. Association for Computational Linguistics (2009)Google Scholar
  6. 6.
    Eisenstein, J., Barzilay, R.: Bayesian unsupervised topic segmentation. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 334–343. Association for Computational Linguistics (2008)Google Scholar
  7. 7.
    Hearst, M.A.: Texttiling: segmenting text into multi-paragraph subtopic passages. Comput. linguist. 23(1), 33–64 (1997)Google Scholar
  8. 8.
    Kimura, T., Tagami, R., Katsuyama, H., Sugimoto, S., Miyamori, H.: KSU systems at the NTCIR-14 QA Lab-PoliInfo task. In: Proceedings of the 14th NTCIR Conference on Evaluation of Information Access Technologies, pp. 251–267 (2019)Google Scholar
  9. 9.
    Kimura, Y., et al.: Overview of the NTCIR-14 QA Lab-PoliInfo task. In: Proceedings of the 14th NTCIR Conference on Evaluation of Information Access Technologies, pp. 121–140 (2019)Google Scholar
  10. 10.
    Koshorek, O., Cohen, A., Mor, N., Rotman, M., Berant, J.: Text segmentation as a supervised learning task. In: NAACL-HLT (2018)Google Scholar
  11. 11.
    Lavrenko, V., Croft, W.B.: Relevance based language models. In: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2001, pp. 120–127. ACM, New York (2001)Google Scholar
  12. 12.
    Riloff, E., Jones, R., et al.: Learning dictionaries for information extraction by multi-level bootstrapping. In: AAAI/IAAI, pp. 474–479 (1999)Google Scholar
  13. 13.
    Robertson, S., Walker, S., Jones, S., Hancock-Beaulieu, M., Gatford, M.: Okapi at TREC-3. In: Proceedings of the Third Text REtrieval Conference (1994)Google Scholar
  14. 14.
    Suzuki, M., Matsuda, K., Sekine, S., Okazaki, N., Inui, K.: Neural joint learning for classifying Wikipedia articles into fine-grained named entity types. In: Proceedings of the 30th Pacific Asia Conference on Language, Information and Computation: Posters, pp. 535–544 (2016)Google Scholar
  15. 15.
    Terazawa, K., Shirato, D., Akiba, T., Masuyama, S.: AKBL at NTCIR-14 QA Lab-PoliInfo task. In: Proceedings of the 14th NTCIR Conference on Evaluation of Information Access Technologies, pp. 190–197 (2019)Google Scholar
  16. 16.
    Yang, P., Fang, H., Lin, J.: Anserini: enabling the use of Lucene for information retrieval research. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1253–1256. ACM (2017)Google Scholar
  17. 17.
    Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1480–1489 (2016)Google Scholar
  18. 18.
    Yokote, K., Iwayama, M.: NAMI question answering system at QA Lab-PoliInfo. In: Proceedings of the 14th NTCIR Conference on Evaluation of Information Access Technologies, pp. 278–288 (2019)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Ricoh Company, Ltd.EbinaJapan

Personalised recommendations