Abstract
Many domains in the field of Inductive Logic Programming (ILP) involve highly unbalanced data. Our research has focused on Information Extraction (IE), a task that typically involves many more negative examples than positive examples. IE is the process of finding facts in unstructured text, such as biomedical journals, and putting those facts in an organized system. In particular, we have focused on learning to recognize instances of the protein-localization relationship in Medline abstracts. We view the problem as a machine-learning task: given positive and negative extractions from a training corpus of abstracts, learn a logical theory that performs well on a held-aside testing set. A common way to measure performance in these domains is to use precision and recall instead of simply using accuracy. We propose Gleaner, a randomized search method which collects good clauses from a broad spectrum of points along the recall dimension in recall-precision curves and employs an “at least N of these M clauses” thresholding method to combine the selected clauses. We compare Gleaner to ensembles of standard Aleph theories and find that Gleaner produces comparable testset results in a fraction of the training time needed for ensembles.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aitken, S.: Learning Information Extraction Rules: An Inductive Logic Programming Approach. In: van Harmelen, F. (ed.) Proceedings of the 15th European Conference on Artificial Intelligence, Amsterdam (2002)
Ben Taskar, M.-F.W., Abbeeland, P., Koller, D.: Label and Link Prediction in Relational Data. In: IJCAI Workshop on Learning Statistical Models from Relational Data (2003)
Blaschke, C., Hirschman, L., Valencia, A.: Information Extraction in Molecular Biology. Briefings in Bioinformatics 3(2), 154–165 (2002)
Blaschke, C., Valencia, A.: Can Bibliographic Pointers for Known Biological Data be Found Automatically? Protein Interactions as a Case Study. Comparative and Functional Genomics 2, 196–206 (2001)
Bradley, A.: The Use of the Area Under the ROC Curve in the Evaluation of Machine Learning Algorithms. Pattern Recognition 30(7), 1145–1159 (1997)
Breiman, L.: Bagging Predictors. Machine Learning 24(2), 123–140 (1996)
Bunescu, R., Ge, R., Kate, R., Marcotte, E., Mooney, R., Ramani, A., Wong, Y.: Comparative Experiments on Learning Information Extractors for Proteins and their Interactions. Journal of Artificial Intelligence in Medicine (2004)
Califf, M., Mooney, R.: Relational Learning of Pattern-Match Rules for Information Extraction. In: Working Notes of AAAI Spring Symposium on Applying Machine Learning to Discourse Processing, pp. 6–11. AAAI Press, Menlo Park (1998)
Cortes, C., Mohri, M.: AUC Optimization vs. Error Rate Minimization. In: Neural Information Processing Systems NIPS2003 (2003)
Craven, M., Slattery, S.: Relational Learning with Statistical Predicate Invention: Better Models for Hypertext. Machine Learning 43(1/2), 97–119 (2001)
de Castro Dutra, I., Page, D., Costa, V.S., Shavlik, J.: An Empirical Evaluation of Bagging in Inductive Logic Programming. In: Twelfth International Conference on Inductive Logic Programming, Sydney, Australia, pp. 48–65 (2002)
Dietterich, T.: Machine-Learning Research: Four Current Directions. The AI Magazine 18(4), 97–136 (1998)
Eliassi-Rad, T., Shavlik, J.: A Theory-Refinement Approach to Information Extraction. In: Proceedings of the 18th International Conference on Machine Learning (2001)
Freitag, D., Kushmerick, N.: Boosted Wrapper Induction. In: AAAI/IAAI, pp. 577–583 (2000)
Hoche, S., Wrobel, S.: Relational Learning Using Constrained Confidence-Rated Boosting. In: 11th International Conference on Inductive Logic Programming, Strasbourg, France (2001)
Hu, Z.: Guidelines for Protein Name Tagging. Technical report, Georgetown University (2003)
Kauchak, D., Smarr, J., Elkan, C.: Sources of Success for Boosted Wrapper Induction. Journal of Machine Learning Research 5, 499–527 (2004)
Lavrac, N., Zelezny, F., Flach, P.: RSD: Relational Subgroup Discovery through First-order Feature Construction. In: Matwin, S., Sammut, C. (eds.) ILP 2002. LNCS (LNAI), vol. 2583, pp. 149–165. Springer, Heidelberg (2003)
Manning, C., Schutze, H.: Foundations of Statistical Natural Language Processing. MIT Press, Cambridge (1999)
Michalski, R., Larson, J.: Inductive Inference of VL Decision Rules. In: Proceedings of the Workshop in Pattern-Directed Inference Systems (May 1977)
Popescul, L., Ungar, S.: Lawrence, and D. Pennock. Statistical Relational Learning for Document Mining. In: IEEE International Conference on Data Mining, ICDM 2003 (2003)
Porter, M.: An Algorithm for Suffix Stripping. Program 14(3), 130–137 (1980)
Ray, S., Craven, M.: Representing Sentence Structure in Hidden Markov Models for Information Extraction. In: Proceedings of the 17th International Joint Conference on Artificial Intelligence, IJCAI 2001 (2001)
Riloff, E.: The Sundance Sentence Analyzer (1998), http://www.cs.utah.edu/projects/nlp/
Rindflesch, T., Tanabe, T., Weinstein, L., Hunter, J.: Edgar: Extraction of drugs, genes and relations from the biomedical literature. In: Proceedings of the Pacific Symposium on Biocomputing (2000)
Shatkay, H., Feldman, R.: Mining the Biomedical Literature in the Genomic Era: An Overview. Journal of Computational Biology 10(6), 821–855 (2003)
Srinivasan, A.: The Aleph Manual Version 4 (2003), http://web.comlab.ox.ac.uk/oucl/research/areas/machlearn/Aleph/
Srinivasan, A., King, R.: Feature Construction with Inductive Logic Programming: A Study of Quantitative Predictions of Biological Activity Aided by Structural Attributes. In: Muggleton, S. (ed.) Proceedings of the 6th International Workshop on Inductive Logic Programming, Stockholm University, Royal Institute of Technology, pp. 352–367 (1996)
Srinivasan, A., Muggleton, S., Sternberg, M., King, R.: Theories for Mutagenicity: A Study in First-Order and Feature-Based Induction. Artificial Intelligence 85(1- 2), 277–299 (1996)
Tang, L., Mooney, R., Melville, P.: Scaling up ILP to Large Examples: Results on Link Discovery for Counter-Terrorism. In: KDD Workshop on Multi-Relational Data Mining (2003)
Temkin, J., Gilder, M.: Extraction of Protein Interaction Information From Unstructured Text Using a Context-Free Grammar. Bioinformatics 19(16), 2046–2053 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Goadrich, M., Oliphant, L., Shavlik, J. (2004). Learning Ensembles of First-Order Clauses for Recall-Precision Curves: A Case Study in Biomedical Information Extraction. In: Camacho, R., King, R., Srinivasan, A. (eds) Inductive Logic Programming. ILP 2004. Lecture Notes in Computer Science(), vol 3194. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30109-7_11
Download citation
DOI: https://doi.org/10.1007/978-3-540-30109-7_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22941-4
Online ISBN: 978-3-540-30109-7
eBook Packages: Springer Book Archive