EasySVM: A visual analysis approach for open-box support vector machines
Support vector machines (SVMs) are supervised learning models traditionally employed for classification and regression analysis. In classification analysis, a set of training data is chosen, and each instance in the training data is assigned a categorical class. An SVM then constructs a model based on a separating plane that maximizes the margin between different classes. Despite being one of the most popular classification models because of its strong performance empirically, understanding the knowledge captured in an SVM remains difficult. SVMs are typically applied in a black-box manner where the details of parameter tuning, training, and even the final constructed model are hidden from the users. This is natural since these details are often complex and difficult to understand without proper visualization tools. However, such an approach often brings about various problems including trial-and-error tuning and suspicious users who are forced to trust these models blindly.
The contribution of this paper is a visual analysis approach for building SVMs in an open-box manner. Our goal is to improve an analyst’s understanding of the SVM modeling process through a suite of visualization techniques that allow users to have full interactive visual control over the entire SVM training process. Our visual exploration tools have been developed to enable intuitive parameter tuning, training data manipulation, and rule extraction as part of the SVM training process. To demonstrate the efficacy of our approach, we conduct a case study using a real-world robot control dataset.
Keywordssupport vector machines (SVMs) rule extraction visual classification high-dimensional visualization visual analysis
This work was supported in part by the National Basic Research Program of China (973 Program, No. 2015CB352503), the Major Program of National Natural Science Foundation of China (No. 61232012), and the National Natural Science Foundation of China (No. 61422211).
- Tzeng, F.-Y.; Ma, K.-L. Opening the black box— Data driven visualization of neural networks. In: Proceedings of the IEEE Visualization, 383–390, 2005.Google Scholar
- Núñez, H.; Angulo, C.; Català, A. Rule extraction from support vector machines. In: Proceedings of the European Symposium on Artificial Neural Networks, 107–112, 2002.Google Scholar
- Schölkopf, B.; Smola, A. J. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, 2002.Google Scholar
- Ladicky, L.; Torr, P. Locally linear support vector machines. In: Proceedings of the 28th International Conference on Machine Learning, 985–992, 2011.Google Scholar
- Ganti, R.; Gray, A. Local support vector machines: Formulation and analysis. arXiv preprint arXiv:1309.3699, 2013.Google Scholar
- Wahba, G. Support vector machines, reproducing kernel Hilbert spaces, and randomized GACV. In: Advances in Kernel Methods. Schölkopf, B.; Burges, C. J. C.; Smola, A. J. Eds. Cambridge, MA, USA: MIT Press, 69–88, 1999.Google Scholar
- Hsu, C.-W.; Chang, C.-C.; Lin, C.-J. A practical guide to support vector classification. 2016. Available at http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide. pdf.Google Scholar
- Mangasarian, O. L.; Wild, E. W. Proximal support vector machine classifiers. In: Proceedings of KDD- 2001: Knowledge Discovery and Data Mining, 77–86, 2001.Google Scholar
- Maji, S.; Berg, A. C.; Malik, J. Classification using intersection kernel support vector machines is efficient. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1–8, 2008.Google Scholar
- Blanzieri, E.; Melgani, F. An adaptive SVM nearest neighbor classifier for remotely sensed imagery. In: Proceedings of the IEEE International Symposium on Geoscience and Remote Sensing, 3931–3934, 2006.Google Scholar
- Yin, C.; Zhu, Y.; Mu, S.; Tian, S. Local support vector machine based on cooperative clustering for very largescale dataset. In: Proceedings of the 8th International Conference on Natural Computation, 88–92, 2012.Google Scholar
- Fung, G.; Sandilya, S.; Rao, R. B. Rule extraction from linear support vector machines. In: Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, 32–40, 2005.Google Scholar
- Caragea, D.; Cook, D.; Wickham, H.; Honavar, V. Visual methods for examining SVM classifiers. In: Visual Data Mining. Simoff, S. J.; Böhlen, M. H.; Mazeika, A. Eds. Springer Berlin Heidelberg, 2007.Google Scholar
- Aragon, C. R.; Bailey, S. J.; Poon, S.; Runge, K. J.; Thomas, R. C. Sunfall: A collaborative visual analytics system for astrophysics. In: Proceedings of the IEEE Symposium on Visual Analytics Science and Technology, 219–220, 2007.Google Scholar
- Ma, Y.; Chen, W.; Ma, X.; Xu, J.; Huang, X.; Maciejewski, R.; Tung, A. K. H. EasySVM: A visual analysis approach for open-box support vector machines. In: Proceedings of the IEEE VIS 2014 Workshop on Visualization for Predictive Analytics, 2014.Google Scholar
- Buja, A.; Cook, D.; Asimov, D.; Hurley, C. Computational methods for high-dimensional rotations in data visualization. In: Handbook of Statistics, Volume 24: Data Mining and Data Visualization. Rao, C. R.; Wegman, E. J.; Solka, J. L. Eds. Amsterdam, the Netherlands: North-Holland Publishing Co., 391–413, 2005.CrossRefGoogle Scholar
- Cook, D.; Buja, A. Manual controls for high-dimensional data projections. Journal of Computational and Graphical Statistics Vol. 6, No. 4, 464–480, 1997.Google Scholar
- Cleveland, W. C.; McGill, M. E. Dynamic Graphics for Statistics. Boca Raton, FL, USA: CRC Press, 1988.Google Scholar
- Inselberg, A.; Dimsdale, B. Parallel coordinates: A tool for visualizing multi-dimensional geometry. In: Proceedings of the 1st Conference on Visualization, 361–378, 1990.Google Scholar
- Quinlan, J. R. Induction of decision trees. Machine Learning Vol. 1, No. 1, 81–106, 1986.Google Scholar
- Teoh, S. T.; Ma, K.-L. PaintingClass: Interactive construction, visualization and exploration of decision trees. In: Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 667–672, 2003.Google Scholar
- Van den Elzen, S.; van Wijk, J. J. BaobabView: Interactive construction and analysis of decision trees. In: Proceedings of the IEEE Conference on Visual Analytics Science and Technology, 151–160, 2011.Google Scholar
- Höferlin, B.; Netzel, R.; Höferlin, M.; Weiskopf, D.; Heidemann, G. Inter-active learning of ad-hoc classifiers for video visual analytics. In: Proceedings of the IEEE Conference on Visual Analytics Science and Technology, 23–32, 2012.Google Scholar
- Freire, A. L.; Barreto, G. A.; Veloso, M.; Varela, A. T. Short-term memory mechanisms in neural network learning of robot navigation tasks: A case study. In: Proceedings of the 6th Latin American Robotics Symposium, 1–6, 2009.Google Scholar
Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.