MICCLLR: Multiple-Instance Learning Using Class Conditional Log Likelihood Ratio

  • Yasser EL-Manzalawy
  • Vasant Honavar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5808)


Multiple-instance learning (MIL) is a generalization of the supervised learning problem where each training observation is a labeled bag of unlabeled instances. Several supervised learning algorithms have been successfully adapted for the multiple-instance learning settings. We explore the adaptation of the Naive Bayes (NB) classifier and the utilization of its sufficient statistics for developing novel multiple-instance learning methods. Specifically, we introduce MICCLLR (multiple-instance class conditional log likelihood ratio), a method for mapping each bag of instances as a single meta-instance using class conditional log likelihood ratio statistics such that any supervised base classifier can be applied to the meta-data. The results of our experiments with MICCLLR using different base classifiers suggest that no single base classifier consistently outperforms other base classifiers on all data sets. We show that a substantial improvement in performance is obtained using an ensemble of MICCLLR classifiers trained using different base learners. We also show that an extra gain in classification accuracy is obtained by applying AdaBoost.M1 to weak MICCLLR classifiers. Overall, our results suggest that the predictive performance of the three proposed variants of MICCLLR are competitive to some of the state-of-the-art MIL methods.


multiple-instance learning image retrieval drug activity prediction ensemble of multiple-instance learning classifiers boosted multiple-instance learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Dietterich, T., Lathrop, R., Lozano-Pérez, T.: Solving the multiple instance problem with axis-parallel rectangles. Artif. Intell. 89, 31–71 (1997)CrossRefzbMATHGoogle Scholar
  2. 2.
    Wang, C., Scott, S., Zhang, J., Tao, Q., Fomenko, D., Gladyshev, V.: A Study in Modeling Low-Conservation Protein Superfamilies. Technical Report TR-UNL-CSE-2004-3, Dept. of Computer Science, University of Nebraska (2004)Google Scholar
  3. 3.
    Maron, O., Ratan, A.: Multiple-instance learning for natural scene classification. In: Proceedings of the Fifteenth International Conference on Machine Learning table of contents, pp. 341–349 (1998)Google Scholar
  4. 4.
    Zhang, Q., Goldman, S., Yu, W., Fritts, J.: Content-based image retrieval using multiple-instance learning. In: Proceedings of the Nineteenth International Conference on Machine Learning, pp. 682–689 (2002)Google Scholar
  5. 5.
    Bi, J., Chen, Y., Wang, J.: A sparse support vector machine approach to region-based image categorization. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1121–1128 (2005)Google Scholar
  6. 6.
    Fung, G., Dundar, M., Krishnapuram, B., Rao, R.: Multiple instance learning for computer aided diagnosis. In: Advances in Neural Information Processing Systems: Proceedings of the 2006 Conference, pp. 425–432. MIT Press, Cambridge (2007)Google Scholar
  7. 7.
    Ramon, J., De Raedt, L.: Multi instance neural networks. In: Proceedings of the ICML 2000 Workshop on Attribute-Value and Relational Learning (2000)Google Scholar
  8. 8.
    Wang, J., Zucker, J.D.: Solving the multiple-instance problem: a lazy learning approach. In: Proceedings 17th International Conference on Machine Learning, pp. 1119–1125 (2000)Google Scholar
  9. 9.
    Maron, O., Lozano-Perez, T.: A framework for multiple-instance learning. Adv. Neural. Inf. Process. Syst. 10, 570–576 (1998)Google Scholar
  10. 10.
    Zhang, Q., Goldman, S.A.: Em-dd: An improved multiple-instance learning technique. Neural. Inf. Process. Syst. 14 (2001)Google Scholar
  11. 11.
    Chen, Y., Wang, J.: Image categorization by learning and reasoning with regions. J. Mach. Learn. Res. 5, 913–939 (2004)MathSciNetGoogle Scholar
  12. 12.
    Ray, S., Craven, M.: Supervised versus multiple instance learning: An empirical comparison. In: Proceedings of the Twentieth-Second International Conference on Machine Learning, pp. 697–704 (2005)Google Scholar
  13. 13.
    Weidmann, N., Frank, E., Pfahringer, B.: A two-level learning method for generalized multi-instance problems. In: Lavrač, N., Gamberger, D., Todorovski, L., Blockeel, H. (eds.) ECML 2003. LNCS (LNAI), vol. 2837, pp. 468–479. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  14. 14.
    Tao, Q., Scott, S., Vinodchandran, N., Osugi, T.: SVM-based generalized multiple-instance learning via approximate box counting. In: Proceedings of the twenty-first international conference on Machine learning. ACM Press, New York (2004)Google Scholar
  15. 15.
    Chen, Y., Bi, J., Wang, J.: MILES: multiple-instance learning via embedded instance selection. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1931–1947 (2006)CrossRefGoogle Scholar
  16. 16.
    Zhou, Z.: Multi-instance learning from supervised view. Journal of Computer Science and Technology 21, 800–809 (2006)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Andrews, S., Tsochantaridis, I., Hofmann, T.: Support vector machines for multiple-instance learning. In: Adv. Neural. Inf. Process. Syst., vol. 15, pp. 561–568. MIT Press, Cambridge (2003)Google Scholar
  18. 18.
    Zhang, M., Zhou, Z.: Adapting RBF neural networks to multi-instance learning. Neural Process. Lett. 23, 1–26 (2006)CrossRefGoogle Scholar
  19. 19.
    Domingos, P., Pazzani, M.: On the optimality of the simple Bayesian classifier under zero-one loss. Machine learning 29, 103–130 (1997)CrossRefzbMATHGoogle Scholar
  20. 20.
    Mitchell, T.: Machine Learning. McGraw-Hill, New York (1997)zbMATHGoogle Scholar
  21. 21.
    Hellerstein, J., Jayram, T., Rish, I.: Recognizing End-User Transactions in Performance Management. In: Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence, pp. 596–602. AAAI Press/The MIT Press (2000)Google Scholar
  22. 22.
    Freund, Y., Schapire, R.: Experiments with a new boosting algorithm. In: Proceedings of 13th International Conference in Machine Learning, pp. 148–156 (1996)Google Scholar
  23. 23.
    Gartner, T., Flach, P., Kowalczyk, A., Smola, A.: Multi-instance kernels. In: Proceedings of the 19th International Conference on Machine Learning, pp. 179–186 (2002)Google Scholar
  24. 24.
    Friedman, J., Hastie, T., Tibshirani, R.: Special invited paper. additive logistic regression: A statistical view of boosting. Annals of statistics, 337–374 (2000)Google Scholar
  25. 25.
    Zhou, Z., Zhang, M.: Solving multi-instance problems with classifier ensemble based on constructive clustering. Knowl. Inf. Syst. 11, 155–170 (2007)CrossRefGoogle Scholar
  26. 26.
    Witten, I., Frank, E.: Data mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)zbMATHGoogle Scholar
  27. 27.
    Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)MathSciNetzbMATHGoogle Scholar
  28. 28.
    Le Cessie, S., Van Houwelingrn, J.: Ridge estimators in logistic regression. Appl. Stat. 41, 191–201 (1992)CrossRefzbMATHGoogle Scholar
  29. 29.
    Quinlan, J.: C4. 5: programs for machine learning. Morgan Kaufmann, San Francisco (1993)Google Scholar
  30. 30.
    Freund, Y., Mason, L.: The alternating decision tree learning algorithm. In: Proceedings of the Sixteenth International Conference on Machine Learning table of contents, pp. 124–133. Morgan Kaufmann, San Francisco (1999)Google Scholar
  31. 31.
    Platt, J.: Fast training of support vector machines using sequential minimal optimization. MIT Press, Cambridge (1998)Google Scholar
  32. 32.
    Ustün, B., Melssen, W., Buydens, L.: Facilitating the application of Support Vector Regression by using a universal Pearson VII function based kernel. Chemometrics and Intelligent Laboratory Systems 81, 29–40 (2006)CrossRefGoogle Scholar
  33. 33.
    Wolpert, D.: Stacked generalization. Neural Networks 5, 241–259 (1992)CrossRefGoogle Scholar
  34. 34.
    Andrews, S., Hofmann, T.: Multiple instance learning via disjunctive programming boosting. In: 2003 Conference on Advances in Neural Information Processing Systems, pp. 65–72. Bradford Book (2004)Google Scholar
  35. 35.
    Auer, P., Ortner, R.: A Boosting Approach to Multiple Instance Learning. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) ECML 2004. LNCS (LNAI), vol. 3201, pp. 63–74. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  36. 36.
    Xu, X., Frank, E.: Logistic regression and boosting for labeled bags of instances. In: Dai, H., Srikant, R., Zhang, C. (eds.) PAKDD 2004. LNCS (LNAI), vol. 3056, pp. 272–281. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  37. 37.
    Viola, P., Platt, J., Zhang, C.: Multiple Instance Boosting for Object Detection. In: Advances in Neural Information Processing Systems, pp. 1417–1424 (2006)Google Scholar
  38. 38.
    Frank, E., Xu, X.: Applying propositional learning algorithms to multi-instance data. Technical report, Dept. of Computer Science, University of Waikato (2003)Google Scholar
  39. 39.
    Wang, D., Li, J., Zhang, B.: Multiple-instance learning via random walk. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, p. 473. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  40. 40.
    Mangasarian, O., Wild, E.: Multiple instance classification via successive linear programming. Data Mining Institute Technical Report 05-02 (2005)Google Scholar
  41. 41.
    Han, F., Wang, D., Liao, X.: An Improved Multiple-Instance Learning Algorithm. In: 4th international symposium on Neural Networks, pp. 1104–1109. Springer, Heidelberg (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Yasser EL-Manzalawy
    • 1
    • 2
  • Vasant Honavar
    • 1
  1. 1.Department of Computer ScienceIowa State UniversityAmesUSA
  2. 2.Systems and Computer EngineeringAl-Azhar UniversityCairoEgypt

Personalised recommendations