Machine Learning

, Volume 77, Issue 1, pp 61–102 | Cite as

Anytime classification for a pool of instances

  • Bei Hui
  • Ying Yang
  • Geoffrey I. Webb


In many real-world applications of classification learning, such as credit card transaction vetting or classification embedded in sensor nodes, multiple instances simultaneously require classification under computational resource constraints such as limited time or limited battery capacity. In such a situation, available computational resources should be allocated across the instances in order to optimize the overall classification efficacy and efficiency. We propose a novel anytime classification framework, Scheduling Anytime Averaged Probabilistic Estimators (SAAPE), which is capable of classifying a pool of instances, delivering accurate results whenever interrupted and optimizing the collective classification performance. Following the practice of our previous anytime classification system AAPE, SAAPE runs a sequence of very efficient Bayesian probabilistic classifiers to classify each single instance. Furthermore, SAAPE implements seven alternative scheduling schemes to decide which instance gets available computational resources next such that a new classifier can be applied to refine its classification. We formally present each scheduling scheme’s definition, rationale and time complexity. We conduct large-scale experiments using 60 benchmark data sets and diversified statistical tests to evaluate SAAPE’s performance on zero-one loss classification as well as on probability estimation. We analyze each scheduling scheme’s advantages and disadvantages according to both theoretical understandings and empirical observations. Consequently we identify effective scheduling schemes that enable SAAPE to accomplish accurate anytime classification for a pool of instances.


Anytime classification Computational resource constraints Bayesian probabilistic classifiers Ensemble learning 


  1. Asuncion, A., & Newman, D. J. (2007). UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences. Google Scholar
  2. Baeza-Yates, R., & Ribeiro-Neto, B. (1999). Modern information retrieval. Reading/Harlow: Addison-Wesley/Longman. Google Scholar
  3. Bartlett, P. L. (2007). Pattern classification and large margin classifiers. Machine Learning Summer School 2006, Taipei. Google Scholar
  4. Bekara, M., Knockaert, L., Seghouane, A.-K., & Fleury, G. (2006). A model selection approach to signal denoising using Kullback’s symmetric divergence. Signal Processing, 86, 1400–1409. CrossRefGoogle Scholar
  5. Chan, P., Fan, W., Prodromidis, A., & Stolfo, S. (1999). Distributed data mining in credit card fraud detection. IEEE Intelligent Systems, 14(6), 67–74. CrossRefGoogle Scholar
  6. DeCoste, D. (2002). Anytime interval-valued outputs for kernel machines: Fast support vector machine classification via distance geometry. In Proceedings of the 19th international conference on machine learning (pp. 99–106). Google Scholar
  7. Deitel, H. M., Deitel, P. J., & Choffnes, D. R. (2004). Operating systems (3rd ed.). New York: Prentice Hall. Google Scholar
  8. Demsar, J. (2006). Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7, 1–30. MathSciNetGoogle Scholar
  9. Duarte, M. F., & Hu, Y. H. (2004). Vehicle classification in distributed sensor networks. Journal of Parallel and Distributed Computing, 64(7), 826–838. CrossRefGoogle Scholar
  10. Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association, 32, 675–701. CrossRefGoogle Scholar
  11. Friedman, M. (1940). A comparison of alternative tests of significance for the problem of m rankings. Annals of Mathematical Statistics, 11, 86–92. zbMATHCrossRefMathSciNetGoogle Scholar
  12. Grass, J., & Zilberstein, S. (1996). Anytime algorithm development tools. M. Pittarelli (Ed.). SIGART Bulletin Special Issue on Anytime Algorithms and Deliberation Scheduling, 7(2), 20–27. Google Scholar
  13. Jeffreys, H. (1946). An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, 186, 453–461. zbMATHCrossRefMathSciNetGoogle Scholar
  14. Jitnah, N., & Nicholson, A. (1997). treeNets: A framework for anytime evaluation of belief networks. In Proceedings of the international joint conference on qualitative and quantitative practical reasoning (pp. 350–364). Google Scholar
  15. Keogh, E. J., & Pazzani, M. J. (2002). Learning the structure of augmented Bayesian classifiers. International Journal on Artificial Intelligence Tools, 11(40), 587–601. CrossRefGoogle Scholar
  16. Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the 14th international joint conference on artificial intelligence (pp. 1137–1145). Google Scholar
  17. Kohavi, R., & Provost, F. (1998). Glossary of terms, special issue on applications of machine learning and the knowledge discovery process. Machine Learning, 30, 271–274. CrossRefGoogle Scholar
  18. Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. Annals of Mathematical Statistics, 22(1), 79–86. zbMATHCrossRefMathSciNetGoogle Scholar
  19. Langley, P., Iba, W., & Thompson, K. (1992). An analysis of Bayesian classifiers. In Proceedings of the 10th national conference on artificial intelligence (pp. 223–228). Google Scholar
  20. Liu, C.-L., & Wellman, M. P. (1996). On state-space abstraction for anytime evaluation of Bayesian networks. ACM SIGART Bulletin, 7(2), 50–57. CrossRefGoogle Scholar
  21. Meesookho, C., Narayanan, S., & Raghavendra, C. S. (2002). Collaborative classification applications in sensor networks. In Proceedings of the sensor array and multichannel signal processing workshop (pp. 370–374). Google Scholar
  22. Resnick, P., & Varian, H. R. (1997). Recommender systems. Communications of the ACM, 40(3), 56–58. CrossRefGoogle Scholar
  23. Schapire, R., Freund, Y., Bartlett, P., & Lee, W. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of Statistics, 26(5), 1651–1686. zbMATHCrossRefMathSciNetGoogle Scholar
  24. Stallings, W. (2004). Operating systems: Internals and design principles (5th ed.). New York: Prentice Hall. Google Scholar
  25. Sun, Y., & Daigle, J. N. (2006). A PCA-based vehicle classification system in wireless sensor networks. In Proceedings of the wireless communications and networking conference (Vol. 4, pp. 2193–2198). Google Scholar
  26. Tanenbaum, A. S. (2001). Modern operating systems (2nd ed.). New York: Prentice Hall. Google Scholar
  27. Ueno, K., Xi, X., Keogh, E., & Lee, D.-J. (2006). Anytime classification using the nearest neighbor algorithm with applications to stream mining. In Proceedings of the 6th international conference on data mining (pp. 623–632). Google Scholar
  28. Webb, G. I., Boughton, J., & Wang, Z. (2005). Not so naive Bayes: Averaged one-dependence estimators. Machine Learning, 58(1), 5–24. zbMATHCrossRefGoogle Scholar
  29. Webb, G. I., Pazzani, M. J., & Billsus, D. (2001). Machine learning for user modeling. User Modeling and User-Adapted Interaction, 11(1–2), 19–29. zbMATHCrossRefGoogle Scholar
  30. Witten, I. H., & Frank, E. (2005). Data mining: Practical machine learning tools and techniques with java implementations (2nd ed.). San Mateo: Morgan Kaufmann. Google Scholar
  31. Yang, Y., Webb, G. I., Korb, K., & Ting, K. M. (2007). Classifying under computational resource constraints: anytime classification using probabilistic estimators. Machine Learning, 69, 35–53. CrossRefGoogle Scholar
  32. Ye, L., Wang, X., Yankov, D., & Keogh, E. (2008). The asymmetric approximate anytime join: A new primitive with applications to data mining. In Proceedings of the SIAM conference on data mining. Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.School of Computer Science and EngineeringUniversity of Electronic Science and Technology of ChinaChengDuChina
  2. 2.Australian Taxation OfficeMelbourneAustralia
  3. 3.Clayton School of Information TechnologyMonash UniversityClaytonAustralia

Personalised recommendations