Advertisement

Abstract

Corpora are not easy to get a handle on. The usual way of getting to grips with text is to read it, but corpora are mostly too big to read (and not designed to be read). We show, with examples, how keyword lists (of one corpus vs. another) are a direct, practical and fascinating way to explore the characteristics of corpora, and of text types. Our method is to classify the top one hundred keywords of corpus1 vs. corpus2, and corpus2 vs. corpus1. This promptly reveals a range of contrasts between all the pairs of corpora we apply it to. We also present improved maths for keywords, and briefly discuss quantitative comparisons between corpora. All the methods discussed (and almost all of the corpora) are available in the Sketch Engine, a leading corpus query tool.

Keywords

corpora corpus similarity keywords keyword lists Sketch Engine 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Baroni, M., Bernardini, S., Ferraresi, A., Zanchetta, E.: The wacky wide web: a collection of very large linguistically processed web-crawled corpora. Language Resources and Evaluation Journal 43, 209–226 (2009)CrossRefGoogle Scholar
  2. 2.
    Pomikàlek, J., Rychlý, P., Kilgarriff, A.: Scaling to billion-plus word corpora. Advances in Computational Linguistics. Special Issue or Research in Computer Science 41 (2009)Google Scholar
  3. 3.
    Biber, D.: Variation across speech and writing. Cambridge University Press (1988)Google Scholar
  4. 4.
    Biber, D.: Dimensions of Register Variation: a cross-linguistic study. Cambridge University Press (2006)Google Scholar
  5. 5.
    Heylighen, F., Dewaele, J.M.: Formality of language: definition, measurement and behavioral determinants. Technical report, Free University of Brussels (1999)Google Scholar
  6. 6.
    Manning, C., Schütze, H.: Foundations of Statistical Natural Language Processing. MIT Press (1999)Google Scholar
  7. 7.
    Kilgarriff, A.: Language is never ever ever random. Corpus Linguistics and Linguistic Theory 1, 263–276 (2005)CrossRefGoogle Scholar
  8. 8.
    Heuboeck, A., Holmes, J., Nesi, H.: The BAWE corpus manual. Technical report, Universities of Warwick, Coventry and Reading (2007)Google Scholar
  9. 9.
    Partington, A.: Modern diachronic corpus-assisted discourse studies MD-CADS on UK newspapers: an overview of the project. Corpora 5, 83–108 (2010)CrossRefGoogle Scholar
  10. 10.
    Baker, P., Gabrielatos, C., McEnery, T.: Discourse Analysis and Media Bias: The representation of Islam in the British Press. Cambridge University Press (2012)Google Scholar
  11. 11.
    Biber, D.: University Language: A corpus-based study of spoken and written registers. John Benjamins (2006)Google Scholar
  12. 12.
    Paquot, M.: Academic Vocabulary in Learner Writing. Continuum (2010)Google Scholar
  13. 13.
    Kosem, I.: Designing a model for a corpus-driven dictionary of Academic English. Ph.D. thesis, Aston University, UK (2010)Google Scholar
  14. 14.
    Keller, F., Lapata, M.: Using the web to obtain frequencies for unseen bigrams. Computational Linguistics 29, 459–484 (2003)CrossRefGoogle Scholar
  15. 15.
    Sharoff, S.: Creating general-purpose corpora using automated search engine queries. In: Baroni, M., Bernardini, S. (eds.) WaCky! Working papers on the Web as Corpus, Gedit, Bologna (2006)Google Scholar
  16. 16.
    Leech, G.: New resources, or just better old ones? the holy grail of representativeness. In: Hundt, M., Nesselehauf, N., Biewer, C. (eds.) Corpus Linguistics and the Web, pp. 133–149. Rodopi, Amsterdam (2007)Google Scholar
  17. 17.
    Čermák, F., Schmiedtová, V., Křen, M.: Czech national corpus – syn. Technical report, Institute of the Czech National Corpus (Prague, Czech Republic) http://www.korpus.cz (accessed on June 08, 2012)
  18. 18.
    Kilgarriff, A.: Comparing corpora. Int. Jnl. Corpus Linguistics 6, 263–276 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Adam Kilgarriff
    • 1
  1. 1.Lexical Computing Ltd.BrightonUK

Personalised recommendations