Skip to main content

Towards Unrestricted, Large-Scale Acquisition of Feature-Based Conceptual Representations from Corpus Data

Abstract

In recent years a number of methods have been proposed for the automatic acquisition of feature-based conceptual representations from text corpora. Such methods could offer valuable support for theoretical research on conceptual representation. However, existing methods do not target the full range of concept-relation-feature triples occurring in human-generated norms (e.g. flute produce sound) but rather focus on concept-feature pairs (e.g. flute – sound) or triples involving specific relations only (e.g. is-a or part-of relations). In this article we investigate the challenges that need to be met in both methodology and evaluation when moving towards the acquisition of more comprehensive conceptual representations from corpora. In particular, we investigate the usefulness of three types of knowledge in guiding the extraction process: encyclopedic, syntactic and semantic. We present first a semantic analysis of existing, human-generated feature production norms, which reveals information about co-occurring concept and feature classes. We introduce then a novel method for large-scale feature extraction which uses the class-based information to guide the acquisition process. The method involves extracting candidate triples consisting of concepts, relations and features (e.g. deer have antlers, flute produce sound) from corpus data parsed for grammatical dependencies, and re-weighting the triples on the basis of conditional probabilities calculated from our semantic analysis. We apply this method to an automatically parsed Wikipedia corpus which includes encyclopedic information and evaluate its accuracy using a number of different methods: direct evaluation against the McRae norms in terms of feature types and frequencies, human evaluation, and novel evaluation in terms of conceptual structure variables. Our investigation highlights a number of issues which require addressing in both methodology and evaluation when aiming to improve the accuracy of unconstrained feature extraction further.

This is a preview of subscription content, access via your institution.

References

  • Almuhareb, A., & Poesio, M. (2004). Attribute-based and value-based clustering: An evaluation. In Proceedings of the conference on empirical methods in natural language processing (EMNLP 2004) (pp. 158–165). Barcelona, Spain.

  • Almuhareb, A., & Poesio, M. (2005). Concept learning and categorization from the web. In Proceedings of the 27th annual meeting of the cognitive science society (pp. 103–108). Stresa, Italy.

  • Andrews M., Vigliocco G., Vinson D. (2009) Integrating experiential and distributional data to learn semantic representations. Psychological Review 116(3): 463–498

    Article  Google Scholar 

  • Barbu, E. (2008). Combining methods to learn feature-norm-like concept descriptions. In Proceedings of the ESSLLI 2008 workshop on distributional lexical semantics (pp. 9–16). Hamburg, Germany.

  • Baroni, M., Evert, S., & Lenci, A. (Eds.). (2008). Proceedings of the ESSLLI 2008 workshop on distributional lexical semantics. Hamburg, Germany.

  • Baroni M., Lenci A. (2008) Concepts and properties in word spaces. From context to meaning: Distributional models of the lexicon in linguistics and cognitive science (Special issue of the Italian Journal of Linguistics 20(1): 55–88

    Google Scholar 

  • Baroni M., Murphy B., Barbu E., Poesio M. (2010) Strudel: A corpus-based semantic model based on properties and types. Cognitive Science 34(2): 222–254

    Article  Google Scholar 

  • Bird, S., Klein, E., & Loper, E. (2009). Natural language processing with Python. O’Reilly Media Inc.

  • Briscoe, E. (2006). An introduction to tag sequence grammars and the RASP system parser. Technical Report 662, University of Cambridge, Computer Laboratory.

  • Briscoe, E. & Carroll, J. (2002). Robust accurate statistical annotation of general text. In Proceedings of the 3rd international conference on language resources and evaluation (pp. 1499–1504). Las Palmas, Gran Canaria.

  • Briscoe, E., Carroll, J., & Watson, R. (2006). The second release of the RASP system. In Proceedings of the COLING/ACL on Interactive presentation sessions (pp. 77–80). Sydney, Australia.

  • Budanitsky A., Hirst G. (2006) Evaluating WordNet-based measures of lexical semantic relatedness. Computational Linguistics 32(1): 13–47

    Article  Google Scholar 

  • Burnard L. (2000) Users reference guide for the British National Corpus. Oxford University Computing Services, Technical report. Oxford

    Google Scholar 

  • Cree G. S., McNorgan C., McRae K. (2006) Distinctive features hold a privileged status in the computation of word meaning: Implications for theories of semantic memory. Journal of Experimental Psychology. Learning, Memory, and Cognition 32(4): 643–658

    Article  Google Scholar 

  • Cree G. S., McRae K., McNorgan C. (1999) An attractor model of lexical conceptual processing: Simulating semantic priming. Cognitive Science 23(3): 371–414

    Article  Google Scholar 

  • Farah M. J., McClelland J. L. (1991) A computational model of semantic memory impairment: Modality specificity and emergent category specificity. Journal of Experimental Psychology: General 120(4): 339–357

    Article  Google Scholar 

  • Fellbaum, C. (eds) (1998) WordNet: An electronic lexical database. MIT Press, Cambridge, MA

    Google Scholar 

  • Gabrilovich, E. & Markovitch, S. (2007). Computing semantic relatedness using Wikipedia-based explicit semantic analysis. In Proceedings of the 20th international joint conference on artificial intelligence (IJCAI’07) (pp. 1606–1611). Hyderabad, India.

  • Greer, M. J., van Castern, M., McLellan, S. A., Moss, H. E., Rodd, J., Rogers, T. T., & Tyler, L. K., (2001). The emergence of semantic catgeories from distributed featural representations. In Proceedings of the 23rd annual conference of the cognitive science society (pp. 358–363). Edinburgh, Scotland.

  • Grondin R., Lupker S. J., McRae K. (2009) Shared features dominate semantic richness effects for concrete concepts. Journal of Memory and Language 60(1): 1–19

    Article  Google Scholar 

  • Hinton G. E., Shallice T. (1991) Lesioning an attractor network: Investigations of acquired dyslexia. Psychological Review 98(1): 74–95

    Article  Google Scholar 

  • Jiang, J. J., Conrath, D. W. (1997). Semantic similarity based on corpus statistics and lexical taxonomy. InProceedings of the international conference on research in computational linguistics (ROCLING X), (pp. 19–33). Tapei, Taiwan.

  • Landauer T., Foltz P., Laham D. (1998) An introduction to latent semantic analysis. Discourse Processes 25: 259–284

    Article  Google Scholar 

  • Landis J. R., Koch G. G. (1977) The measurement of observer agreement for categorical data. Biometrics 33(1): 159–174

    Article  Google Scholar 

  • Leacock C., Miller G. A., Chodorow M. (1998) Using corpus statistics and WordNet relations for sense identification. Computational Linguistics 24(1): 147–165

    Google Scholar 

  • Lin, D. (1998). An information-theoretic definition of similarity. In Proceedings of the 15th international conference on machine learning (ICML-98) (pp. 296–304). Madison, Wisconsin.

  • Lund K., Burgess C. (1996) Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, and Computers 28(2): 203–208

    Google Scholar 

  • Masson M. E. J. (1995) A distributed memory model of semantic priming. Journal of Experimental Psychology: Learning, Memory, and Cognition 21(1): 3–23

    Article  Google Scholar 

  • McClelland J. L., Rumelhart D. E. (1985) Distributed memory and the representation of general and specific information. Journal of Experimental Psychology: General 114(2): 159–197

    Article  Google Scholar 

  • McRae K., Cree G. S., Seidenberg M. S., McNorgan C. (2005) Semantic feature production norms for a large set of living and nonliving things. Behavior Research Methods 37(4): 547–559

    Google Scholar 

  • McRae K., de Sa V. R., Seidenberg M. S. (1997) On the nature and scope of featural representations of word meaning. Journal of Experimental Psychology: General 126(2): 99–130

    Article  Google Scholar 

  • Mitchell T. M., Shinkareva S. V., Carlson A., Chang K.-M., Malave V. L., Mason R. A., Just M. A. (2008) Predicting human brain activity associated with the meanings of nouns. Science 320(5880): 1191–1195

    Article  Google Scholar 

  • Moss H.E., Tyler L.K., Devlin J. (2002) The emergence of category-specific deficits in a distributed semantic system. In: Forde E.M.E., Humphreys G.W. (eds) Category specificity in brain and mind. Psychology Press, Hove UK

    Google Scholar 

  • Moss H. E., Tyler L. K., Taylor K. I. (2007) Conceptual structure. In: Gaskell M. G. (eds) The Oxford handbook of psycholinguistics. Oxford University Press, Oxford, UK, pp 217–234

    Google Scholar 

  • Murphy, B., Baroni, M., & Poesio, M. (2009). EEG responds to conceptual stimuli and corpus semantics. InProceedings of the conference on empirical methods in natural language processing (EMNLP 2009) (pp. 619–627). East Stroudsburg, PA.

  • Murphy G. (2002) The big book of concepts. The MIT Press, Cambridge, MA

    Google Scholar 

  • Ó Séaghdha, D., & Copestake, A. (2008). Semantic classification with distributional kernels. In Proceedings of the 22nd international conference on computational linguistics (COLING-08) (pp. 649–656). Manchester, UK.

  • Pado S., Lapata M. (2007) Dependency-based construction of semantic space models. Computational Linguistics 33(2): 161–199

    Article  Google Scholar 

  • Pexman P., Holyk G., Monfils M. (2003) Number-of-features effects and semantic processing. Memory & Cognition 31: 842–855

    Google Scholar 

  • Pexman P., Lupker S., Hino Y. (2002) The impact of feedback semantics in visual word recognition: Number-of-features effects in lexical decision and naming tasks. Psychonomic Bulletin & Review 9: 542–549

    Google Scholar 

  • Plaut D. C. (1997) Structure and function in the lexical system: Insights from distributed models of word reading and lexical decision. Language and Cognitive Processes 12(5): 765–806

    Article  Google Scholar 

  • Randall B., Moss H. E., Rodd J. M., Greer M., Tyler L. K. (2004) Distinctiveness and correlation in conceptual structure: Behavioral and computational studies. Journal of Experimental Psychology: Learning, Memory & Cognition 30(2): 393–406

    Article  Google Scholar 

  • Randall, B., Taylor, K. I., Devereux, B., Acres, K., & Tyler, L.K. (2009). The differential engagement of shared and distinctive correlated feature information in picture categorisation and naming: evidence for the Conceptual Structure Account. In Proceedings of AMLaP (p. 33). Barcelona, Spain.

  • Resnik, P. (1995). Using information content to evaluate semantic similarity in a taxonomy. In Proceedings of the 14th international joint conference on artificial intelligence (IJCAI-95) (pp. 448–453). Montréal, Canada.

  • Rosch E., Mervis C. B. (1975) Family resemblance: Studies in the internal structure of categories. Cognitive Psychology 7(4): 573–605

    Article  Google Scholar 

  • Sun, L., Korhonen, A., and Krymolowski, Y. (2008). Verb class discovery from rich syntactic data. In Proceedings of the ninth international conference on intelligent text processing and computational linguistics (CICLing-2008) (pp. 16–27). Haifa, Israel.

  • Taylor K. I., Salamoura A., Randall B., Moss H., Tyler L. K. (2008) Clarifying the nature of the distinctiveness by domain interaction in conceptual structure: Comment on Cree, McNorgan, and McRae (2006). Journal of Experimental Psychology: Learning, Memory & Cognition 34(3): 719–725

    Article  Google Scholar 

  • Tyler L. K., Moss H. E. (2001) Towards a distributed account of conceptual knowledge. Trends in Cognitive Sciences 5(6): 244–252

    Article  Google Scholar 

  • Tyler L. K., Moss H. E., Durrant-Peatfield M. R., Levy J. P. (2000) Conceptual structure and the structure of concepts: A distributed account of category-specific deficits. Brain and Language 75(2): 195–231

    Article  Google Scholar 

  • Vinson D. P., Vigliocco G. (2008) Semantic feature production norms for a large set of objects and events. Behavior Research Methods 40(1): 183–190

    Article  Google Scholar 

  • Vlachos, A., Korhonen, A., & Ghahramani, Z. (2009). Unsupervised and constrained dirichlet process mixture models for verb clustering. In Proceedings of the workshop on geometrical models of natural language semantics (pp. 74–82). Athens, Greece.

  • Wu, Z., & Palmer, M. (1994). Verb semantics and lexical selection. In Proceedings of the 32nd annual meeting of the association for computational linguistics (pp. 133–138). Las Cruces, New Mexico.

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Barry Devereux.

About this article

Cite this article

Devereux, B., Pilkington, N., Poibeau, T. et al. Towards Unrestricted, Large-Scale Acquisition of Feature-Based Conceptual Representations from Corpus Data. Res on Lang and Comput 7, 137–170 (2009). https://doi.org/10.1007/s11168-010-9068-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11168-010-9068-8

Keywords

  • Property norms
  • Conceptual representations
  • Semantic features
  • Corpus-based acquisition