Skip to main content
SpringerLink
Log in

Constructing a Turkish Corpus for Paraphrase Identification and Semantic Similarity

  • Conference paper
  • First Online:
Computational Linguistics and Intelligent Text Processing (CICLing 2016)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9623))

  • 1393 Accesses

Abstract

The Paraphrase identification (PI) task has practical importance for work in Natural Language Processing (NLP) because of the problem of linguistic variation. Accurate methods should help improve performance of key NLP applications. Paraphrase corpora are important resources in developing and evaluating PI methods. This paper describes the construction of a paraphrase corpus for Turkish. The corpus comprises pairs of sentences with semantic similarity scores based on human judgments, permitting experimentation with both PI and semantic similarity. We believe this is the first such corpus for Turkish. The data collection and scoring methodology is described and initial PI experiments with the corpus are reported. Our approach to PI is novel in using ‘knowledge lean’ methods (i.e. no use of manually constructed knowledge bases or processing tools that rely on these). We have previously achieved excellent results using such techniques on the Microsoft Research Paraphrase Corpus, and close to state-of-the-art performance on the Twitter Paraphrase Corpus.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    TuPC can be downloaded from: http://aslieyecioglu.com/data/.

References

  1. Agirre, E., et al.: Semeval-2012 task 6: a pilot on semantic textual similarity. In: Proceedings of the 6th International Workshop on Semantic Evaluation, in conjunction with the First Joint Conference on Lexical and Computational Semantics, pp. 385–393 (2012)

    Google Scholar 

  2. Bannard, C., Callison-Burch, C.: Paraphrasing with bilingual parallel corpora. In: Proceedings of the 43th Annual Meeting on Association for Computational Linguistics, pp. 597–604 (2005)

    Google Scholar 

  3. Bird, S., et al.: Natural Language Processing with Python. O’Reilly Media Inc., Newton (2009)

    MATH  Google Scholar 

  4. Blacoe, W., Lapata, M.: A comparison of vector-based representations for semantic composition. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL 2012), pp. 546–556 (2012)

    Google Scholar 

  5. Callison-Burch, C. et al.: Improved statistical machine translation using paraphrases. In: Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics (HLT-NAACL 2006), pp. 17–24 (2006)

    Google Scholar 

  6. Chang, C., Lin, C.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 1–27 (2011)

    Article  Google Scholar 

  7. Chen, D.L., Dolan, W.B.: Collecting highly parallel data for paraphrase evaluation. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (HLT 2011), pp. 190–200 (2011)

    Google Scholar 

  8. Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20(1), 37–46 (1960)

    Article  MathSciNet  Google Scholar 

  9. Das, D., Smith, N.A.: Paraphrase identification as probabilistic quasi-synchronous recognition. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: ACL-IJCNLP 2009, pp. 468–476 (2009)

    Google Scholar 

  10. Demir, S., et al.: Turkish Paraphrase Corpus. In: Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC 2012), pp. 4087–4091 (2012)

    Google Scholar 

  11. Dolan, B., et al.: Unsupervised construction of large paraphrase corpora. In: Proceedings of the 20th international conference on Computational Linguistics - COLING 2004, NJ, USA, p. 350–es (2004)

    Google Scholar 

  12. Duclaye, F., et al.: Using the web as a linguistic resource for learning reformulations automatically. In: Proceedings of the Third International Conference on Language Resources and Evaluation (LREC 2002), Las Palmas, Canary Islands, Spain, pp. 390–396 (2002)

    Google Scholar 

  13. Eyecioglu, A., Keller, B.: ASOBEK : Twitter Paraphrase Identification with Simple Overlap Features and SVMs. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), Denver, Colorado, pp. 64–69 (2015)

    Google Scholar 

  14. Fellbaum, C.: WordNet: An Electronic Lexical Database. MIT Press, Cambridge (1998)

    MATH  Google Scholar 

  15. Fernando, S., Stevenson, M.: A semantic similarity approach to paraphrase detection. In: Proceedings of the 11th Annual Research Colloquium of the UK Special Interest Group for Computational Linguistics, pp. 45–52 (2008)

    Google Scholar 

  16. Finch, A. et al.: Using Machine Translation Evaluation Techniques to Determine Sentence-level Semantic Equivalence. In: Proceedings of the Third International Workshop on Paraphrasing (IWP 2005), pp. 17–24 (2005)

    Google Scholar 

  17. Fleiss, J.L.: Measuring nominal scale agreement among many raters. Psychol. Bull. 76, 378–382 (1971)

    Article  Google Scholar 

  18. Ganitkevitch, J., et al.: PPDB : the paraphrase database. In: Proceedings of NAACL-HLT, Atlanta, Gerogia, pp. 758–764 (2013)

    Google Scholar 

  19. Gwet, K.L.: Handbook of Inter-rater Reliability. Advanced Analytics, Gaithersburg (2012)

    Google Scholar 

  20. Hsu, C.-W., et al.: A Practical Guide to Support Vector Classification. BJU Int. 101(1), 1396–1400 (2008)

    Google Scholar 

  21. Landis, J.R., Koch, G.G.: The measurement of observer agreement for categorical data. Biometrics 33(1), 159–174 (1977)

    Article  MATH  Google Scholar 

  22. Lintean, M., Rus, V.: Dissimilarity kernels for paraphrase identification. In: Proceedings of the 24th International Florida Artificial Intelligence Research Society Conference, Palm Beach, FL, pp. 263–268 (2011)

    Google Scholar 

  23. Madnani, N., et al.: Re-examining machine translation metrics for paraphrase identification. In: Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2012), PA, USA, pp. 182–190 (2012)

    Google Scholar 

  24. Madnani, N., et al.: Using paraphrases for parameter tuning in statistical machine translation. In: Proceedings of the Second Workshop on Statistical Machine Translation (WMT 2007), Prague, Czech Republic (2007)

    Google Scholar 

  25. Malakasiotis, P.: Paraphrase recognition using machine learning to combine similarity measures. In: Proceedings of the ACL-IJCNLP 2009 Student Research Workshop, Suntec, Singapore, pp. 27–35 (2009)

    Google Scholar 

  26. Max, A., Wisniewski, G.: Mining naturally-occurring corrections and paraphrases from wikipedia’s revision history. In: Proceeding of LREC, pp. 3143–3148 (2010)

    Google Scholar 

  27. Mihalcea, R., et al.: Corpus-based and knowledge-based measures of text semantic similarity. In: Proceedings of the 21st national conference on Artificial intelligence- Volume 1, pp. 775–780. AAAI Press (2006)

    Google Scholar 

  28. Owczarzak, K., et al.: Contextual bitext-derived paraphrases in automatic MT evaluation. In: StatMT 2006, Stroudsburg, PA, USA, pp. 86–93 (2006)

    Google Scholar 

  29. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., et al.: Scikit-learn: Machine Learning in Python. http://scikit-learn.org/stable/

  30. Quirk, C., et al.: Monolingual machine translation for paraphrase generation. In: EMNLP-2014, pp. 142–149 (2004)

    Google Scholar 

  31. Socher, R., et al.: Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In: Advances in Neural Information Processing Systems, pp. 801–809 (2011)

    Google Scholar 

  32. Stanovsky, G.: A Study in Hebrew Paraphrase Identification. Ben-Gurion University of Negev (2012)

    Google Scholar 

  33. Wubben, S. et al.: Creating and using large monolingual parallel corpora for sentential paraphrase generation, pp. 4292–4299 (2010)

    Google Scholar 

  34. Xu, W.: Data-driven approaches for paraphrasing across language variations. New York University (2014)

    Google Scholar 

  35. Xu, W., et al.: SemEval-2015 Task 1: paraphrase and semantic similarity in Twitter (PIT). In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval) (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Enginnering, Bartin University, 74100, Bartin, Turkey

    Asli Eyecioglu

  2. Department of Informatics, University of Sussex, Brighton, BN19QJ, UK

    Bill Keller

Authors
  1. Asli Eyecioglu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bill Keller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Asli Eyecioglu or Bill Keller .

Editor information

Editors and Affiliations

  1. CIC, Instituto Politécnico Nacional, Mexico City, Mexico

    Alexander Gelbukh

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Eyecioglu, A., Keller, B. (2018). Constructing a Turkish Corpus for Paraphrase Identification and Semantic Similarity. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2016. Lecture Notes in Computer Science(), vol 9623. Springer, Cham. https://doi.org/10.1007/978-3-319-75477-2_42

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-75477-2_42

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-75476-5

  • Online ISBN: 978-3-319-75477-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation