A 300 MB Turkish Corpus and Word Analysis
In order to determine some properties of a language, a corpus of that language should be created. To analyze Turkish language, at first, a Turkish corpus having ~300 MB capacity and more than 44 million words was prepared by using 10 different web sites having Turkish content. Most frequently used word statistics of Turkish were calculated by using this corpus. Frequencies of most frequently used first 7 words were compared with their equivalent in English, and it was found out that most frequently used words are not nouns in natural languages Most frequently used words having 1 to 5 letters were determined and they were applied onto a randomly selected text in order to test the validity of the process.
Unable to display preview. Download preview PDF.
- 1.Shannon, C.E.: Prediction and Entropy of Printed English, The Bell System Technical Journal, 30(1) (1951) 50–64Google Scholar
- 2.Jurafsky, D., Martin, J.H.: Speech and Language Processing, Prentice Hall (2000) 193–196Google Scholar
- 3.Garett, P.: Making-Breaking Codes, ISBN 0-13-030369-0, Prentice Hall (2001) 31–36Google Scholar
- 4.Dalkilic, G.: Some Statistical Properties of Contemporary Printed Turkish and A Text Copression Application, MSc Thesis, International Computing Institute, Ege University (2001)Google Scholar
- 5.Dalkilic, M.E., Dalkilic, G.: Some Measurable Language Characteristics of Printed Turkish, Proc. of the XVI. International Symposium on Computer and Information Sciences (2001) 217–224Google Scholar
- 6.Güngör, T.: Computer Processing of Turkish: Morphological and Lexical Investigation, PhD. Dissertation, Computer Engineering Dept., Bogazici University, Istanbul, Turkey (1995)Google Scholar
- 7.Teahan, W.J.: Modeling English Text, PhD. Dissertation, The University of Waikato, New Zeland (1998)Google Scholar