Abstract
In this article, we explore the feasibility of extracting suitable and unsuitable food items for particular health conditions from natural language text. We refer to this task as conditional healthiness classification. For that purpose, we annotate a corpus extracted from forum entries of a food-related website. We identify different relation types that hold between food items and health conditions going beyond a binary distinction of suitability and unsuitability and devise various supervised classifiers using different types of features. We examine the impact of different task-specific resources, such as a healthiness lexicon that lists the healthiness status of a food item and a sentiment lexicon. Moreover, we also consider task-specific linguistic features that disambiguate a context in which mentions of a food item and a health condition co-occur and compare them with standard features using bag of words, part-of-speech information and syntactic parses. We also investigate in how far individual food items and health conditions correlate with specific relation types and try to harness this information for classification.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
In our dataset, people mostly discuss food items that cause the outbreak of that disease rather than food items that protect them against it or alleviate the symptoms if they contract it.
Note that co-occurrences of a food item and a health condition are only an approximation of genuine food-health relations. In other words, not every of these co-occurrences necessarily conveys a proper food-health relationship. However, without manually annotating each of these co-occurrences, this is the best approximation we can produce.
For this corpus, we can only observe food-health co-occurrences within 5-grams and not entire sentences.
The low coverage on the 5-grams can be partially explained by the fact that this corpus only contains ngrams observed at least 40 times. We assume that there are considerably more co-occurrences of food items and health conditions among web-based 5-grams at lower frequencies.
In the following, we may also refer to category labels as classes or relation types.
We adopt the definition of semantic scope from Wiegand and Klakow (2010).
As there is not a one-to-one mapping between the English keywords from Girju (2003) and their German counterparts, the size of the English and the German lists varies slightly.
The fact that the target health condition is negated may indicate that this sentence expresses some type of conditional unhealthiness, however, there is no lexical cue that helps us to identify the subtype PREVENT.
By negative (data) instances, we mean those instances that have not been tagged with the relation type that is to be extracted.
Since the instances labeled as WORSEN only cover approximately 1 % of our entire dataset (Table 3), we are convinced that the choice of treating them as negative instances or as instances of type UNSUIT will not affect the overall results of our experiments.
We also experimented with variations restricting the co-occurrence to a fixed window size. However, we did not obtain better classifiers than with the plain (sentence-wise) co-occurrence.
In Sect. 3.2 we already speculated that the performance of scope may be affected by a bad parse quality due to the noise contained in our language data. Obviously, the noise really affects syntactic processing and some of the features that depend on that information, such as scope.
We assume that for the other three relation types, i.e. SUIT, UNSUIT and CAUSE, sentiment is not a predictive feature. Causation (as conveyed by CAUSE) is quite different from positive or negative sentiment, so it does not come as a surprise that sentiment features are not effective for this relation type. SUIT and UNSUIT will be discussed in a dedicated section of this error analysis (i.e. Sect. 5.6).
In order to match a polar expression, not only the word token listed in the sentiment lexicon has to match but also its part-of-speech tag.
More than 60 % of the shifters occur as singletons on the sentences labeled as BENEF.
The shifter lexicon from Wilson et al. (2005) just contains 67 shifters which, when translated to German, would have only a marginal impact on our dataset.
The counterintuitive result that helfen (help) appears on list of againstCond and gegen (against) appears on the list of synoHlthTS can be explained by the fact that the two words basically form a collocation helfen gegen (help against) (see also Sentences (30) and (33)).
We consider our unigram bag-of-words features as a typical example of lexical information.
Note that we do not consider acid as a negative polar expression. Acid is not harmful per se. (For example, our digestive system depends on gastric acid.) This is also reflected by the fact that it is not contained in our sentiment lexicon.
Unlike the lexical cues, some of our categories also consider additional syntactic information.
Example: I eat almonds because I suffer from dermatitis.
Notice that there is no direct correspondence between the categories from Table 16 stating the proportion of instances with manually annotated keywords and Tables 28 and 29. Table 16 only considers unigram keywords being either nouns, verbs or adjectives while the annotation in Tables 28 and 29 is unrestricted. In other words, keywords annotated in Table 16 are not the sum of categories excluding inferences in Tables 28 and 29. There will also be other constructions marked in those tables that were not captured by the restricted annotation from Table 16.
It may come as a surprise that the classes BENEF and CAUSE already produce high scores with only 25 % of the training data. This is due to the fact that we consider the full feature set that includes many features based on word lists. Typically, such features are particularly effective if only few training data are present. Such features generalize over individual word occurrences and are less sparse than bag of words.
The training data for those tools typically originate from the news domain, where such expressions cannot be found.
References
Beamer, B., & Girju, R. (2009). Using a bigram event model to predict causal potential. In Proceedings of the International Conference on Intelligent Text Processing and Computational Linguistics (CICLing) (pp. 430–441). Mexico City, Mexico.
Bunescu, R. C., & Mooney, R. J. (2005). Subsequence kernels for relation extraction. In Proceedings of the Conference on Neural Information Processing Systems (NIPS). Vancouver, Canada.
Chahuneau, V., Gimpel, K., Routledge, B. R., Scherlis, L., & Smith, N. A. (2012). Word salad: Relating food prices and descriptions. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP/CoNLL) (pp. 1357–1367). Jeju Island, Korea.
Church, K. W., & Hanks, P. (1990). Word association norms, mutual information, and lexicography. Computational Linguistics, 16(1), 22–29.
Cohen, A. M., & Hersh, W. R. (2005). A survey of current work in biomedical text mining. Briefings in Bioinformatics, 6, 57–71.
Deng, L., & Wiebe, J. (2014). Sentiment propagation via implicature constraints. In Proceedings of the Conference on European Chapter of the Association for Computational Linguistics (EACL) (pp. 377–385). Gothenburg, Sweden.
Deng, L., Wiebe, J., & Choi, Y. (2014). Joint inference and disambiguation of implicit sentiments via implicature constraints. In Proceedings of the International Conference on Computational Linguistics (COLING) (pp. 79–88). Dublin, Ireland.
Diaz-Aviles, E., Stewart, A., Velasco, E., Denecke, K., & Nejdl, W. (2012). Epidemic intelligence for the crowd, by the crowd. In Proceedings of the International AAAI Conference on Weblogs and Social Media (ICWSM). Dublin, Ireland.
Fisichella, M., Stewart, A., Cuzzocrea, A., & Denecke, K. (2011). Detecting health events on the social web to enable epidemic intelligence. In Proceedings of the International Symposium on String Processing and Information Retrieval (SPIRE) (pp. 87–103). Pisa, Italy.
Gildea, D., & Jurafsky, D. (2002). Automatic labeling of semantic roles. Computational Linguistics, 28(3), 245–288.
Girju, R. (2003). Automatic detection of causal relations for question answering. In Proceedings of the ACL 2003 Workshop on Multilingual Summarization and Question Answering (pp. 76–83). Sapporo, Japan.
Girju, R., & Moldovan, D. (2002). Text mining and causal relations. In Proceedings of the International FLAIRS Conference (FLAIRS) (pp. 360–364). Pensacola Beach, FL, USA.
Girju, R., Badulescu, A., & Moldovan, D. (2003). Learning semantic constraints for the automatic discovery of part-whole relations. In Proceedings of the Human Language Technology Conference (HLT) (pp. 80–87). Edmonton, Canada.
Hamp, B., & Feldweg, H. (1997). GermaNet—A lexical-semantic net for German. In Proceedings of ACL Workshop Automatic Information Extraction and Building of Lexical Semantic Resources for NLP Applications (pp. 9–15). Madrid, Spain.
Joachims, T. (1999). Making large-scale SVM learning practical. In B. Schölkopf, C. Burges & A. Smola (eds) Advances in kernel methods—Support vector learning (pp. 169–184). MIT Press.
Kessler, J. S., & Nicolov, N. (2009). Targeting sentiment expressions through supervised ranking of linguistic configurations. In Proceedings of the International AAAI Conference on Weblogs and Social Media (ICWSM). San Jose, CA, USA.
Klenner, M., Petrakis, S., & Fahrni, A. (2009). Robust compositional polarity classification. In Proceedings of Recent Advances in Natural Language Processing (RANLP) (pp. 180–184). Borovets, Bulgaria.
Kozareva, Z. (2012). Cause-effect relation learning. In Proceedings of the Text Graphs Workshop at ACL (pp 39–43). Jeju, Republic of Korea.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.
Miller, G., Beckwith, R., Fellbaum, C., Gross, D., & Miller, K. (1990). Introduction to wordnet: an on-line lexical database. International Journal of Lexicography, 3, 235–244.
Mintz, M., Bills, S., Snow, R., & Jurafsky, D. (2009). Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing (ACL/IJCNLP) (pp. 1003–1011). Singapore.
Moschitti, A., & Basili, R. (2006). Complex linguistic features for text classification: A comprehensive study. In Proceedings of the European Conference in Information Retrieval (ECIR) (pp. 181–196). Sunderland, United Kingdom.
Munro, R., Gunasekara, L., Nevins, S., Polepeddi, L., & Rosen, E. (2012). Tracking epidemics with natural language processing and crowdsourcing. In Proceedings of the Spring Symposium for Association for the Advancement of Artificial Intelligence (AAAI) (pp 52–58). Toronto, Canada.
Ng, V., Dasgupta, S., & Arifin, S. M. N. (2006). Examining the role of linguistic knowledge sources in the automatic identification and classification of reviews. In Proceedings of the International Conference on Computational Linguistics and Annual Meeting of the Association for Computational Linguistics (COLING/ACL) (pp. 611–618). Sydney, Australia.
Popowich, F. (2005). Using text mining and natural language processing for health care claims processing. SIGKDD Explorations, 7(1), 59–66.
Rafferty, A., & Manning, C. D. (2008). Parsing three German treebanks: Lexicalized and unlexicalized baselines. In Proceedings of the ACL Workshop on Parsing German (PaGe) (pp. 40–46). Columbus, OH, USA.
Roth, B., Barth, T., Wiegand, M., Singh, M., & Klakow, D. (2014). Effective slot filling based on shallow distant supervision methods. In Proceedings of the Text Analysis Conference (TAC). Gaithersburg, MD: NIST.
Sokolova, M., & Bobicev, V. (2011). Sentiments and opinions in health-related web messages. In Proceedings of Recent Advances in Natural Language Processing (RANLP) (pp. 180–184). Borovets, Bulgaria.
Torii, M., Yin, L., Nguyen, T., Mazumdar, C. T., Liu, H., Hartley, D. M., et al. (2011). An exploratory study of a text classification framework for internet-based surveillance of emerging epidemics. International Journal of Medical Informatics, 80(1), 56–66.
van Hage, W. R., Katrenko, S., & Schreiber, G. (2005). A method to combine linguistic ontology-mapping techniques. In Proceedings of International Semantic Web Conference (ISWC) (pp. 732–744). Galway: Springer.
van Hage, W. R., Kolb, H., & Schreiber, G. (2006). A method for learning part-whole relations. In Proceedings of International Semantic Web Conference (ISWC) (pp. 723–735). Athens, GA: Springer.
van Hage, W. R., Sini, M., Finch, L., Kolb, H., & Schreiber, G. (2010). The OAEI food task: An analysis of a thesaurus alignment task. Applied Ontology, 5(1), 1–28.
Wiegand, M., & Klakow, D. (2010). Convolution kernels for opinion holder extraction. In Proceedings of the Human Language Technology Conference of the North American Chapter of the ACL (HLT/NAACL) (pp. 795–803). Los Angeles, CA, USA.
Wiegand, M., & Klakow, D. (2013a). Towards contextual healthiness classification of food items—A linguistic approach. In Proceedings of the International Joint Conference on Natural Language Processing (IJCNLP) (pp. 19–27). Nagoya, Japan.
Wiegand, M., & Klakow, D. (2013b). Towards the detection of reliable food-health relationships. In Proceedings of the NAACL-Workshop on Language Analysis in Social Media (pp. 69–79). Atlanta, GA, USA.
Wiegand, M., Roth, B., & Klakow, D. (2012a). Data-driven knowledge extraction for the food domain. In Proceedings of KONVENS (pp. 21–29). Vienna, Austria.
Wiegand, M., Roth, B., & Klakow, D. (2012b). Web-based relation extraction for the food domain. In Proceedings of the International Conference on Applications of Natural Language Processing to Information Systems (NLDB) (pp. 222–227). Groningen: Springer.
Wiegand, M., Roth, B., Lasarcyk, E., Köser, S., & Klakow, D. (2012c). A gold standard for relation extraction in the food domain. In Proceedings of the Conference on Language Resources and Evaluation (LREC) (pp. 507–514). Istanbul, Turkey.
Wiegand, M., Roth, B., & Klakow, D. (2014). Automatic food categorization from large unlabeled corpora and its impact on relation extraction. In Proceedings of the Conference on European Chapter of the Association for Computational Linguistics (EACL) (pp. 673–682). Gothenburg, Sweden.
Wilson, T., Wiebe, J., & Hoffmann, P. (2005). Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing (HLT/EMNLP) (pp. 347–354). Vancouver, BC, Canada.
Witten, I., & Frank, E. (2005). Data mining: Practical machine learning tools and techniques. San Francisco: Morgan Kaufmann Publishers.
Acknowledgments
The authors would like to thank Stephanie Köser for annotating the new resources presented in this article. We would also like to thank Benjamin Roth and Marc Schulder for interesting discussions. Michael Wiegand was partially supported by the German Federal Ministry of Education and Research (BMBF) under Grant No. 01IC12SO1X and the German Research Foundation (DFG) under Grant No. WI 4204/2-1.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wiegand, M., Klakow, D. Detecting conditional healthiness of food items from natural language text. Lang Resources & Evaluation 49, 777–830 (2015). https://doi.org/10.1007/s10579-015-9314-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10579-015-9314-7