Improving Rule Evaluation Using Multitask Learning

  • Mark D. Reid
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3194)


This paper introduces Deft, a new multitask learning approach for rule learning algorithms. Like other multitask learning systems, the one proposed here is able to improve learning performance on a primary task through the use of a bias learnt from similar secondary tasks. What distinguishes Deft from other approaches is its use of rule descriptions as a basis for task similarity. By translating a rule into a feature vector or “description”, the performance of similarly described rules on the secondary tasks can be used to modify the evaluation of the rule for the primary task. This explicitly addresses difficulties with accurately evaluating, and therefore finding, good rules from small datasets. Deft is implemented on top of an existing ILP system and the approach is tested on a variety of relational learning tasks. Given appropriate secondary tasks, the results show that Deft is able to compensate for insufficient training examples.


Primary Task Secondary Task Base Learner Target Concept Inductive Logic Programming 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Srinivasan, A., King, R.D.: Feature construction with inductive logic programming: a study of quantative predictions of biological activity aided by structural attributes. Data Mining and Knowledge Discovery 3, 37–57 (1999)CrossRefGoogle Scholar
  2. 2.
    Caruana, R.: Multitask learning. Machine Learning 28, 41–75 (1997)CrossRefGoogle Scholar
  3. 3.
    Mitchell, T.M.: The need for biases in learning generalizations. Technical Report CBM-TR-117, Rutgers University, New Brunswick, New Jersey (1980)Google Scholar
  4. 4.
    Nédellec, C., Rouveirol, C., Adé, H., Bergadano, F., Tausend, B.: Declarative bias in ILP. In: Advances in ILP. Frontiers in AI and Applications, vol. 32, pp. 82–103. IOS Press, Amsterdam (1996)Google Scholar
  5. 5.
    Fürnkranz, J.: Separate-and-conquer rule learning. Artificial Intelligence Review 13, 3–54 (1999)zbMATHCrossRefGoogle Scholar
  6. 6.
    Datta, P., Kibler, D.F.: Concept sharing: A means to improve multi-concept learning. In: Proc. of the 10th ICML, pp. 89–96 (1993)Google Scholar
  7. 7.
    Khan, K., Muggleton, S., Parson, R.: Repeat learning using predicate invention. In: Proc. of the 8th ILP, pp. 165–174. Springer, Heidelberg (1998)Google Scholar
  8. 8.
    Morin, J.: Learning Relational Clichés with Contextual Generalization. PhD thesis, School of Information Technology and Engineering, University of Ottawa, Canada (1999)Google Scholar
  9. 9.
    Srinivasan, A.: ALEPH: A learning engine for proposing hypotheses. Prolog code (2001),
  10. 10.
    Silver, D.: Selective Transfer of Neural network Task Knowledge. PhD thesis, Graduate Program in Computer Science, University of Western Ontario, London, Ontario, Canada (2000)Google Scholar
  11. 11.
    Good, I.J.: The Estimation of Probabilities: An Essay on Modern Bayesian Methods. MIT Press, Cambridge (1965)zbMATHGoogle Scholar
  12. 12.
    Fürnkranz, J., Flach, P.A.: An analysis of rule evaluation metrics. In: Proc. of the 19th ICML, pp. 202–209. AAAI Press, Menlo Park (2003)Google Scholar
  13. 13.
    Muggleton, S.H.: Inverse entailment and progol. New Generation Computing 13, 245–286 (1995)CrossRefGoogle Scholar
  14. 14.
    Srinivasan, A.: A study of two sampling methods for analysing large datasets with ILP. Data Mining and Knowledge Discovery 3, 95–123 (1999)CrossRefGoogle Scholar
  15. 15.
    De Raedt, L., Bruynhooghe, M.: Interactive concept-learning and constructive induction by analogy. Machine Learning 8, 107–150 (1992)zbMATHCrossRefGoogle Scholar
  16. 16.
    Srinivasan, A., Muggleton, S., King, R.D., Sternberg, M.J.E.: Mutagenesis: ILP experiments in a non-determinate biological domain. In: Proc. of the 4th ILP (1994)Google Scholar
  17. 17.
    Srinivasan, A., King, R.D., Muggleton, S., Sternberg, M.J.E.: Carcinogenesis predictions using ILP. In: Proc. of the 7th ILP, pp. 273–287 (1997)Google Scholar
  18. 18.
    Holte, R.C., Acker, L.E., Porter, B.W.: Concept learning and the problem of small disjuncts. In: Proc. of the 11th IJCAI, pp. 813–818 (1989)Google Scholar
  19. 19.
    Srinivasan, A.: Extracting context-sensitive models in inductive logic programming. Machine Learning 44, 301–324 (2001)zbMATHCrossRefGoogle Scholar
  20. 20.
    Srinivasan, A.: Personal communication. Email regarding chemical data (2002)Google Scholar
  21. 21.
    Lavrač, N., Flach, P., Zupan, B.: Rule evaluation measures: A unifying view. In: Proc. of the 9th ILP, pp. 174–185. Springer, Heidelberg (1999)Google Scholar
  22. 22.
    Vilalta, R., Oblinger, D.: A quantification of distance-bias between evaluation metrics in classification. In: Proc. of the 17th ICML, pp. 1087–1094 (2000)Google Scholar
  23. 23.
    Cestnik, B.: Estimating probabilities: A crucial task in machine learning. In: Proc. of the 9th European Conference on AI, Pitman, pp. 147–149 (1990)Google Scholar
  24. 24.
    Baxter, J.: A model of inductive bias learning. Journal of Artificial Intelligence Research 12, 149–198 (2000)zbMATHMathSciNetGoogle Scholar
  25. 25.
    Heskes, T.: Empirical bayes for learning to learn. In: Proc. of the 17th ICML, pp. 367–374. Morgan Kaufmann, San Francisco (2000)Google Scholar
  26. 26.
    Cussens, J.: Using prior probabilities and density estimation for relational classification. In: Proc. of the 8th ILP, pp. 106–115. Springer, Heidelberg (1998)Google Scholar
  27. 27.
    McCreath, E., Sharma, A.: LIME: A system for learning relations. In: Richter, M.M., Smith, C.H., Wiehagen, R., Zeugmann, T. (eds.) ALT 1998. LNCS (LNAI), vol. 1501, pp. 336–374. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  28. 28.
    Lavrač, N., Flach, P.A.: An extended transformation approach to inductive logic programming. ACM Trans. on Computational Logic (TOCL) 2, 458–494 (2001)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Mark D. Reid
    • 1
  1. 1.School of Computer Science and EngineeringUniversity of New South WalesSydneyAustralia

Personalised recommendations