Beyond Trees: Adopting MITI to Learn Rules and Ensemble Classifiers for Multi-Instance Data

  • Luke Bjerring
  • Eibe Frank
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7106)

Abstract

MITI is a simple and elegant decision tree learner designed for multi-instance classification problems, where examples for learning consist of bags of instances. MITI grows a tree in best-first manner by maintaining a priority queue containing the unexpanded nodes in the fringe of the tree. When the head node contains instances from positive examples only, it is made into a leaf, and any bag of data that is associated with this leaf is removed. In this paper we first revisit the basic algorithm and consider the effect of parameter settings on classification accuracy, using several benchmark datasets. We show that the chosen splitting criterion in particular can have a significant effect on accuracy. We identify a potential weakness of the algorithm—subtrees can contain structure that has been created using data that is subsequently removed—and show that a simple modification turns the algorithm into a rule learner that avoids this problem. This rule learner produces more compact classifiers with comparable accuracy on the benchmark datasets we consider. Finally, we present randomized algorithm variants that enable us to generate ensemble classifiers. We show that these can yield substantially improved classification accuracy.

Keywords

Ensemble Member Gini Index Benchmark Dataset Priority Queue Positive Instance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Andrews, S., Tsochantaridis, I., Hofmann, T.: Support vector machines for multiple-instance learning. In: NIPS, pp. 561–568. MIT Press (2003)Google Scholar
  2. 2.
    Blockeel, H., Page, D., Srinivasan, A.: Multi-instance tree learning. In: ICML, pp. 57–64. ACM (2005)Google Scholar
  3. 3.
    Breiman, L.: Random forests. ML 45(1), 5–32 (2001)MATHGoogle Scholar
  4. 4.
    Chen, Y., Bi, J., Wang, J.Z.: MILES: Multiple-instance learning via embedded instance selection. IEEE PAMI 28(12), 1931–1947 (2006)CrossRefGoogle Scholar
  5. 5.
    Dietterich, T.G., Lathrop, R.H., Lozano-Perez, T.: Solving the multiple instance problem with axis-parallel rectangles. AI 89(1-2), 31–71 (1997)MATHGoogle Scholar
  6. 6.
    Dietterich, T.G.: An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. ML 40(2), 139–157 (2000)Google Scholar
  7. 7.
    Foulds, J., Frank, E.: Revisiting multiple-instance learning via embedded instance selection. In: AUS-AI, pp. 300–310. Springer, Berlin (2008)Google Scholar
  8. 8.
    Foulds, J.R., Frank, E.: Speeding Up and Boosting Diverse Density Learning. In: Pfahringer, B., Holmes, G., Hoffmann, A. (eds.) DS 2010. LNCS, vol. 6332, pp. 102–116. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  9. 9.
    Frank, E., Witten, I.H.: Generating accurate rule sets without global optimization. In: ICML, pp. 144–151. Morgan Kaufmann (1998)Google Scholar
  10. 10.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: an update. SIGKDD Explor. 11(1), 10–18 (2009)CrossRefGoogle Scholar
  11. 11.
    Leistner, C., Saffari, A., Bischof, H.: MIForests: Multiple-Instance Learning with Randomized Trees. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6316, pp. 29–42. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  12. 12.
    Maron, O., Lozano-Pérez, T.: A framework for multiple-instance learning. In: NIPS, pp. 570–576. MIT Press (1998)Google Scholar
  13. 13.
    Maron, O., Ratan, A.L.: Multiple-instance learning for natural scene classification. In: ICML, pp. 341–349. Morgan Kaufmann (1998)Google Scholar
  14. 14.
    Mayo, M.: Effective classifiers for detecting objects. In: CIRAS (2007)Google Scholar
  15. 15.
    Michie, D., Muggleton, S., Page, D., Srinivasan, A.: To the international computing community: A new East-West challenge. Tech. rep., Oxford University (1994)Google Scholar
  16. 16.
    Nadeau, C., Bengio, Y.: Inference for the Generalization Error. ML 52(3), 239–281 (2003)MATHGoogle Scholar
  17. 17.
    Opelt, A., Pinz, A., Fussenegger, M., Auer, P.: Generic object recognition with boosting. IEEE PAMI 28(3), 416–431 (2006)CrossRefMATHGoogle Scholar
  18. 18.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann (1993)Google Scholar
  19. 19.
    Srinivasan, A., Muggleton, S., King, R., Sternberg, M.: Mutagenesis: ILP experiments in a non-determinate biological domain. In: ILP, pp. 217–232. GMD (1994)Google Scholar
  20. 20.
    Wang, C., Scott, S., Zhang, J., Tao, Q., Fomenko, D., Gladyshev, V.: A study in modeling low-conservation protein superfamilies. Tech. rep., Department of Comp. Sci., University of Nebraska-Lincoln (2004)Google Scholar
  21. 21.
    Zucker, J., Chevaleyre, Y.: Solving multiple-instance and multiple-part learning problems with decision trees and decision rules. Application to the mutagenesis problem. In: Proc. Conf. of the Canadian Society for Computational Studies of Intelligence, pp. 204–214 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Luke Bjerring
    • 1
  • Eibe Frank
    • 1
  1. 1.Department of Computer ScienceUniversity of WaikatoNew Zealand

Personalised recommendations