Abstract
The standard prediction process of SVM requires acquisition of all the feature values for every instance. In practice, however, a cost is associated with the mere act of acquisition of a feature, e.g. CPU time needed to compute the feature out of raw data, the dollar amount spent for gleaning more information, or the patient wellness sacrificed by an invasive medical test, etc. In such applications, a budget constrains the classification process from using all of the features. We present, AceCost, a novel classification method that reduces the expected test cost of SVM without compromising from the classification accuracy. Our algorithm uses a cost efficient decision tree to partition the feature space for obtaining coarse decision boundaries, and local SVM classifiers at the leaves of the tree to refine them. The resulting classifiers are also effective in scenarios where several features share overlapping acquisition procedures, hence the cost of acquiring them as a group is less than the sum of the individual acquisition costs. Our experiments on the standard UCI datasets, a network flow detection application, as well as on synthetic datasets show that, the proposed approach achieves classification accuracy of SVM while reducing the test cost by 40%-80%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and regression trees. Wadsworth Brooks/Cole Advanced Books Software, Monterey, CA (1984) ISBN: 978-0412048418
Quinlan, J.R.: Bagging, boosting, and c4.5. In: Proceedings of the Thirteenth National Conference on Artificial Intelligence, pp. 725–730. AAAI Press, Menlo Park (1996)
Li, L., Topkara, U., Coskun, B., Memon, N.: CoCoST: A Computational Cost Efficient Classifier. In: Proceedings of the 9th International Conference on Data Mining (ICDM 2009), Miami, FL, December 6-9 (2009)
Kulesh Shanmugasundaram, M.K., Memon, N.: Nabs: A system for detecting resource abuses via characterization of flow content type. In: Computer Security Application Conference, Annual, pp. 316–325 (December 2004)
Ming Tan, J.C.S.: Two case studies in cost-sensitive concept acquisition. In: Proceedings of the Eighth National Conference on Artificial Intelligence (1990)
Tan, M.: Cost-sensitive learning of classification knowledge and its applications in robotics. Machine Learning 13, 7–33 (1993)
Nunez, M.: The use of background knowledge in decision tree induction. Machine Learning 6(3), 231–250 (1991)
Mansour, Y.: Pessimistic decision tree pruning based on tree size. In: Proc. 14th International Conference on Machine Learning (1997), pp. 195–201 (1997)
Murphy, P.M., Aha, D.W.: UCI Repository of Machine Learning Databases, University of California at Irvine, Department of Information and Computer Science (1994)
Kapoor, A., Greiner, R.: Learning and classifying under hard budgets. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 170–181. Springer, Heidelberg (2005)
Alspector, E.K.J.: Svm-based filtering of e-mail spam with content-specific misclassification costs. In: Proceedings of the Workshop on Text Mining, (TextDM 2001) (2001)
Osuna, E., Freund, R., Girosi, F.: Training Support Vector Machines: an Application to Face Detection. In: Proceedings of Computer Vision and Pattern Recognition 1997, pp. 130–136 (June 1997)
Abbes, T.: Protocol analysis in intrusion detection using decision tree. In: Proc. ITCC 2004, pp. 404–408 (2004)
Arun Kumar, M., Gopal, M.: A hybrid SVM based decision tree. Journal of Pattern Recogn. 43(12), 3977–3987 (2010)
Bennett, K.P., Blue, J.A.: A Support Vector Machine Approach to Decision Trees, Department of Mathematical Sciences Math Report No. 97-100, Rensselaer Polytechnic Institute (1997)
Madzarov, G., Gjorgjevikj, D., Chorbev, I.: A Multi-class SVM Classifier Utilizing Binary Decision Tree (2008)
Fei, B., Liu, J.: Binary Tree of SVM: A New Fast Multiclass Training and Classification Algorithm. IEEE Transaction on Neural Networks 17(3), 696–704 (2006)
Seewald, A.K., Petrak, J., Widmer, G.: Hybrid Decision Tree Learners with Alternative Leaf Classifiers: An Empirical Study. In: Proceedings of the 14th FLAIRS Conference (2000), pp. 407–411. AAAI Press, Menlo Park (2000)
Hyafil, L., Rivest, R.L.: Constructing Optimal Binary Decision Trees is NP-complete. Information Processing Letters 5(1), 15–17 (1976)
Grigoris, J.: Karakoulas: Cost-Effective Classification for Credit Decision Making Knowledge (1995)
Chen, Y.-w.: Combining SVMs with Various Feature Selection. Taiwan University, Springer (2005)
Esposito, F., Malerba, D., Semeraro, G.: A Comparative Analysis of Methods for Pruning Decision Trees. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 476–491 (1997)
Huang, J., Lu, J., Ling, C.X.: Comparing Naive Bayes, Decision Trees, and SVM with AUC and Accuracy. In: Third IEEE International Conference on Data Mining (ICDM 2003), Melbourne, Florida (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Li, L., Topkara, U., Memon, N. (2011). ACE-Cost: Acquisition Cost Efficient Classifier by Hybrid Decision Tree with Local SVM Leaves. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2011. Lecture Notes in Computer Science(), vol 6871. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23199-5_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-23199-5_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23198-8
Online ISBN: 978-3-642-23199-5
eBook Packages: Computer ScienceComputer Science (R0)