Skip to main content
SpringerLink
Log in

MICCLLR: Multiple-Instance Learning Using Class Conditional Log Likelihood Ratio

  • Conference paper
Discovery Science (DS 2009)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 5808))

Included in the following conference series:

Abstract

Multiple-instance learning (MIL) is a generalization of the supervised learning problem where each training observation is a labeled bag of unlabeled instances. Several supervised learning algorithms have been successfully adapted for the multiple-instance learning settings. We explore the adaptation of the Naive Bayes (NB) classifier and the utilization of its sufficient statistics for developing novel multiple-instance learning methods. Specifically, we introduce MICCLLR (multiple-instance class conditional log likelihood ratio), a method for mapping each bag of instances as a single meta-instance using class conditional log likelihood ratio statistics such that any supervised base classifier can be applied to the meta-data. The results of our experiments with MICCLLR using different base classifiers suggest that no single base classifier consistently outperforms other base classifiers on all data sets. We show that a substantial improvement in performance is obtained using an ensemble of MICCLLR classifiers trained using different base learners. We also show that an extra gain in classification accuracy is obtained by applying AdaBoost.M1 to weak MICCLLR classifiers. Overall, our results suggest that the predictive performance of the three proposed variants of MICCLLR are competitive to some of the state-of-the-art MIL methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Dietterich, T., Lathrop, R., Lozano-Pérez, T.: Solving the multiple instance problem with axis-parallel rectangles. Artif. Intell. 89, 31–71 (1997)

    Article  MATH  Google Scholar 

  2. Wang, C., Scott, S., Zhang, J., Tao, Q., Fomenko, D., Gladyshev, V.: A Study in Modeling Low-Conservation Protein Superfamilies. Technical Report TR-UNL-CSE-2004-3, Dept. of Computer Science, University of Nebraska (2004)

    Google Scholar 

  3. Maron, O., Ratan, A.: Multiple-instance learning for natural scene classification. In: Proceedings of the Fifteenth International Conference on Machine Learning table of contents, pp. 341–349 (1998)

    Google Scholar 

  4. Zhang, Q., Goldman, S., Yu, W., Fritts, J.: Content-based image retrieval using multiple-instance learning. In: Proceedings of the Nineteenth International Conference on Machine Learning, pp. 682–689 (2002)

    Google Scholar 

  5. Bi, J., Chen, Y., Wang, J.: A sparse support vector machine approach to region-based image categorization. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1121–1128 (2005)

    Google Scholar 

  6. Fung, G., Dundar, M., Krishnapuram, B., Rao, R.: Multiple instance learning for computer aided diagnosis. In: Advances in Neural Information Processing Systems: Proceedings of the 2006 Conference, pp. 425–432. MIT Press, Cambridge (2007)

    Google Scholar 

  7. Ramon, J., De Raedt, L.: Multi instance neural networks. In: Proceedings of the ICML 2000 Workshop on Attribute-Value and Relational Learning (2000)

    Google Scholar 

  8. Wang, J., Zucker, J.D.: Solving the multiple-instance problem: a lazy learning approach. In: Proceedings 17th International Conference on Machine Learning, pp. 1119–1125 (2000)

    Google Scholar 

  9. Maron, O., Lozano-Perez, T.: A framework for multiple-instance learning. Adv. Neural. Inf. Process. Syst. 10, 570–576 (1998)

    Google Scholar 

  10. Zhang, Q., Goldman, S.A.: Em-dd: An improved multiple-instance learning technique. Neural. Inf. Process. Syst. 14 (2001)

    Google Scholar 

  11. Chen, Y., Wang, J.: Image categorization by learning and reasoning with regions. J. Mach. Learn. Res. 5, 913–939 (2004)

    MathSciNet  Google Scholar 

  12. Ray, S., Craven, M.: Supervised versus multiple instance learning: An empirical comparison. In: Proceedings of the Twentieth-Second International Conference on Machine Learning, pp. 697–704 (2005)

    Google Scholar 

  13. Weidmann, N., Frank, E., Pfahringer, B.: A two-level learning method for generalized multi-instance problems. In: Lavrač, N., Gamberger, D., Todorovski, L., Blockeel, H. (eds.) ECML 2003. LNCS (LNAI), vol. 2837, pp. 468–479. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  14. Tao, Q., Scott, S., Vinodchandran, N., Osugi, T.: SVM-based generalized multiple-instance learning via approximate box counting. In: Proceedings of the twenty-first international conference on Machine learning. ACM Press, New York (2004)

    Google Scholar 

  15. Chen, Y., Bi, J., Wang, J.: MILES: multiple-instance learning via embedded instance selection. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1931–1947 (2006)

    Article  Google Scholar 

  16. Zhou, Z.: Multi-instance learning from supervised view. Journal of Computer Science and Technology 21, 800–809 (2006)

    Article  MathSciNet  Google Scholar 

  17. Andrews, S., Tsochantaridis, I., Hofmann, T.: Support vector machines for multiple-instance learning. In: Adv. Neural. Inf. Process. Syst., vol. 15, pp. 561–568. MIT Press, Cambridge (2003)

    Google Scholar 

  18. Zhang, M., Zhou, Z.: Adapting RBF neural networks to multi-instance learning. Neural Process. Lett. 23, 1–26 (2006)

    Article  Google Scholar 

  19. Domingos, P., Pazzani, M.: On the optimality of the simple Bayesian classifier under zero-one loss. Machine learning 29, 103–130 (1997)

    Article  MATH  Google Scholar 

  20. Mitchell, T.: Machine Learning. McGraw-Hill, New York (1997)

    MATH  Google Scholar 

  21. Hellerstein, J., Jayram, T., Rish, I.: Recognizing End-User Transactions in Performance Management. In: Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence, pp. 596–602. AAAI Press/The MIT Press (2000)

    Google Scholar 

  22. Freund, Y., Schapire, R.: Experiments with a new boosting algorithm. In: Proceedings of 13th International Conference in Machine Learning, pp. 148–156 (1996)

    Google Scholar 

  23. Gartner, T., Flach, P., Kowalczyk, A., Smola, A.: Multi-instance kernels. In: Proceedings of the 19th International Conference on Machine Learning, pp. 179–186 (2002)

    Google Scholar 

  24. Friedman, J., Hastie, T., Tibshirani, R.: Special invited paper. additive logistic regression: A statistical view of boosting. Annals of statistics, 337–374 (2000)

    Google Scholar 

  25. Zhou, Z., Zhang, M.: Solving multi-instance problems with classifier ensemble based on constructive clustering. Knowl. Inf. Syst. 11, 155–170 (2007)

    Article  Google Scholar 

  26. Witten, I., Frank, E.: Data mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)

    MATH  Google Scholar 

  27. Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)

    MathSciNet  MATH  Google Scholar 

  28. Le Cessie, S., Van Houwelingrn, J.: Ridge estimators in logistic regression. Appl. Stat. 41, 191–201 (1992)

    Article  MATH  Google Scholar 

  29. Quinlan, J.: C4. 5: programs for machine learning. Morgan Kaufmann, San Francisco (1993)

    Google Scholar 

  30. Freund, Y., Mason, L.: The alternating decision tree learning algorithm. In: Proceedings of the Sixteenth International Conference on Machine Learning table of contents, pp. 124–133. Morgan Kaufmann, San Francisco (1999)

    Google Scholar 

  31. Platt, J.: Fast training of support vector machines using sequential minimal optimization. MIT Press, Cambridge (1998)

    Google Scholar 

  32. Ustün, B., Melssen, W., Buydens, L.: Facilitating the application of Support Vector Regression by using a universal Pearson VII function based kernel. Chemometrics and Intelligent Laboratory Systems 81, 29–40 (2006)

    Article  Google Scholar 

  33. Wolpert, D.: Stacked generalization. Neural Networks 5, 241–259 (1992)

    Article  Google Scholar 

  34. Andrews, S., Hofmann, T.: Multiple instance learning via disjunctive programming boosting. In: 2003 Conference on Advances in Neural Information Processing Systems, pp. 65–72. Bradford Book (2004)

    Google Scholar 

  35. Auer, P., Ortner, R.: A Boosting Approach to Multiple Instance Learning. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) ECML 2004. LNCS (LNAI), vol. 3201, pp. 63–74. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  36. Xu, X., Frank, E.: Logistic regression and boosting for labeled bags of instances. In: Dai, H., Srikant, R., Zhang, C. (eds.) PAKDD 2004. LNCS (LNAI), vol. 3056, pp. 272–281. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  37. Viola, P., Platt, J., Zhang, C.: Multiple Instance Boosting for Object Detection. In: Advances in Neural Information Processing Systems, pp. 1417–1424 (2006)

    Google Scholar 

  38. Frank, E., Xu, X.: Applying propositional learning algorithms to multi-instance data. Technical report, Dept. of Computer Science, University of Waikato (2003)

    Google Scholar 

  39. Wang, D., Li, J., Zhang, B.: Multiple-instance learning via random walk. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, p. 473. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  40. Mangasarian, O., Wild, E.: Multiple instance classification via successive linear programming. Data Mining Institute Technical Report 05-02 (2005)

    Google Scholar 

  41. Han, F., Wang, D., Liao, X.: An Improved Multiple-Instance Learning Algorithm. In: 4th international symposium on Neural Networks, pp. 1104–1109. Springer, Heidelberg (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Iowa State University, Ames, IA, 50011-1040, USA

    Yasser EL-Manzalawy & Vasant Honavar

  2. Systems and Computer Engineering, Al-Azhar University, Cairo, Egypt

    Yasser EL-Manzalawy

Authors
  1. Yasser EL-Manzalawy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vasant Honavar
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculty of Economics; Rua Dr. Roberto Frias, University of Porto, 4200-465, Porto, Portugal

    João Gama

  2. DCC-FC, Universidade do Porto, Portugal

    Vítor Santos Costa

  3. LIACC/FEP, Universidade do Porto, Portugal

    Alípio Mário Jorge

  4. LIAAD-INESC Porto L.A./Faculty of Economics, University of Porto, Rua de Ceuta, 118-6, 4050-190, Porto, Portugal

    Pavel B. Brazdil

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

EL-Manzalawy, Y., Honavar, V. (2009). MICCLLR: Multiple-Instance Learning Using Class Conditional Log Likelihood Ratio. In: Gama, J., Costa, V.S., Jorge, A.M., Brazdil, P.B. (eds) Discovery Science. DS 2009. Lecture Notes in Computer Science(), vol 5808. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04747-3_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-04747-3_9

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-04746-6

  • Online ISBN: 978-3-642-04747-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation