Advertisement

Improving Document Similarity Calculation Using Cosine-Similarity Graphs

  • Yasunao TakanoEmail author
  • Yusuke Iijima
  • Kou Kobayashi
  • Hiroshi Sakuta
  • Hiroki Sakaji
  • Masaki Kohana
  • Akio Kobayashi
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 926)

Abstract

Data mining information using various indices and determining candidates that can be judged as having the same tendency based on similarity between documents is common. The accuracy of similarity largely depends on a sufficient amount of data and requires advanced analysis using natural language processing. In this paper, we present an approach for filtering based on the cosine similarity graph and clustering between candidates. For filtering candidates, we focus on the inflection point of the graphs when plotting by sorting similarities in descending order. By clustering among higher similarities, we aim at filtering candidates that cannot be removed without analyzing advanced natural language processing. The proposed method was applied to movie reviews and sightseeing location reviews written in Japanese. Although this study is a work in progress, it shows that candidates can be recommended without having to manually apply natural language processing, such as preparing stopwords for each category.

Keywords

Document similarity Cosine similarity Content-based filtering 

References

  1. 1.
    Adomavicius, G., Tuzhilin, A.: Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17(6), 734–749 (2005)CrossRefGoogle Scholar
  2. 2.
    Le, Q., Mikolov, T.: Distributed representations of sentences and documents. In: Proceedings of the 31st International Conference on International Conference on Machine Learning, ICML 2014, vol. 32, pp. II–1188–II–1196 (2014)Google Scholar
  3. 3.
    MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics, pp. 281–297 (1967)Google Scholar
  4. 4.
    Töscher, A., Jahrer, M., Legenstein, R.: Improved neighborhood-based algorithms for large-scale recommender systems. In: Proceedings of the 2nd KDD Workshop on Large-Scale Recommender Systems and the Netflix Prize Competition, NETFLIX 2008, pp. 4:1–4:6 (2008)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Yasunao Takano
    • 1
    Email author
  • Yusuke Iijima
    • 1
  • Kou Kobayashi
    • 1
  • Hiroshi Sakuta
    • 1
  • Hiroki Sakaji
    • 2
  • Masaki Kohana
    • 3
  • Akio Kobayashi
    • 4
  1. 1.Aoyama Gakuin UniversityChuo-ku, Sagamihara-shiJapan
  2. 2.The University of TokyoBunkyo-kuJapan
  3. 3.Ibaraki UniversityHitachiJapan
  4. 4.RIKEN AIP CenterChuo-kuJapan

Personalised recommendations