Abstract
Support Vector Machines (SVM) are believed to be as powerful as Artificial Neural Networks (ANN) in modeling complex problems while avoiding some of the drawbacks of the latter such as local minimæ or reliance on architecture. However, a question that remains to be answered is whether SVM users may expect improvements in the interpretability of their models, namely by using rule extraction methods already available to ANN users. This study successfully applies the Orthogonal Search-based Rule Extraction algorithm (OSRE) to Support Vector Machines. The study evidences the portability of rules extracted using OSRE, showing that, in the case of SVM, extracted rules are as accurate and consistent as those from equivalent ANN models. Importantly, the study also shows that the OSRE method benefits from SVM specific characteristics, being able to extract less rules from SVM than from equivalent ANN models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Fisher, D.H., McKusick, K.B.: An Empirical Comparison of ID3 and Back-Propagation. In: 11th International Joint Conference on Artificial Intelligence, vol. 1, pp. 788–793. Morgan Kaufmann, Michigan (1989)
Shavlik, J.W., Mooney, R.J., Towell, G.G.: Symbolic and Neural Learning Algorithms: An Experimental Comparison. Machine Learning 6, 111–143 (1991)
Weiss, S.M., Kapouleas, I.: An Empirical Comparison of Pattern Recognition, Neural Nets, and Machine Learning Classification Methods. In: 11th International Joint Conference on Artificial Intelligence, vol. 1, pp. 781–787. Morgan Kaufmann, Michigan (1989)
Etchells, T.A., Lisboa, P.J.G.: Orthogonal Search-based Rule Extraction (OSRE) for Trained Neural Networks: A Practical and Efficient Approach. IEEE Transactions on Neural Networks 17, 374–384 (2006)
Andrews, R., Diederich, J., Tickle, A.B.: Survey and Critique of Techniques for Extracting Rules from Trained Artificial Neural Networks. Knowledge-Based Systems 8, 373–389 (1995)
Aung, M.S., Lisboa, P.J., Etchells, T.A., Testa, A.C., Calster, B., Huffel, S., Valentin, L., Timmerman, D.: Comparing Analytical Decision Support Models Through Boolean Rule Extraction: A Case Study of Ovarian Tumour Malignancy. In: Liu, D., Fei, S., Hou, Z., Zhang, H., Sun, C. (eds.) ISNN 2007. LNCS, vol. 4492, pp. 1177–1186. Springer, Heidelberg (2007)
Lisboa, P.J.G., Jarman, I.H., Etchells, T.A., Ramsey, P.: A Prototype Integrated Decision Support System for Breast Cancer Oncology. In: 9th International Work Conference on Artificial Neural Networks, pp. 996–1003. Springer, San Sebastián (2007)
Jarman, I.H., Etchells, T.A., Martín, J.D., Lisboa, P.J.G.: An Integrated Framework for Risk Profiling of Breast Cancer Patients Following Surgery. Artificial Intelligence in Medicine 42, 165–188 (2008)
Lisboa, P.J.G., Etchells, T.A., Jarman, I.H., Aung, M.S.H., Chabaud, S., Bachelot, T., Perol, D., Gargi, T., Bourdès, V., Bonnevay, S., Négrier, S.: Time-to-event Analysis with Artificial Neural Networks: An Integrated Analytical and Rule-Based Study for Breast Cancer. Neural Networks 21, 414–426 (2008)
Pop, E., Hayward, R., Diederich, J.: RULENEG: Extracting Rules from a Trained ANN by Stepwise Negation. Queensland University of Technology, Australia (1994)
Tsukimoto, H.: Extracting Rules from Trained Neural Networks. IEEE Transactions on Neural Networks 11, 377–389 (2000)
Thrun, S.B., Bala, J., Bloedorn, E., Bratko, I., Cestnik, B., Cheng, J., De Jong, K., Deroski, S., Fahlman, S.E., Fisher, D., Hamann, R., Kaufman, K., Keller, S., Kononenko, I., Kreuziger, J., Michalski, R.S., Mitchell, T., Pachowicz, P., Reich, Y., Vafaie, H., Van de Welde, W., Wenzel, W., Wnek, J., Zhang, J.: The MONK’s Problems: A Performance Comparison of Different Learning Algorithms. Technical Report CS-91-197, Computer Science Department, Carnegie Mellon University, Pittsburgh (1991)
Wisconsin Breast Cancer, http://archive.ics.uci.edu/ml/datasets.html
Iris Dataset, http://archive.ics.uci.edu/ml/datasets.html
Platt, J.C.: Fast Training of Support Vector Machines using Sequential Minimal Optimization. In: Platt, J.C. (ed.) Advances in Kernel Methods: Support Vector Learning, pp. 185–208. MIT Press, Cambridge (1999)
Etchells, T.A.: Rule Extraction from Neural Networks: A Practical and Efficient Approach. Ph.D Dissertation. John Moores University, Liverpool (2003)
Burges, C., Crisp, D.: Uniqueness of the SVM Solution. In: Advances in Neural Information Processing Systems, vol. 12, pp. 223–229. MIT Press, Cambridge (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Guerreiro, J., Trigueiros, D. (2010). A Unified Approach to the Extraction of Rules from Artificial Neural Networks and Support Vector Machines. In: Cao, L., Zhong, J., Feng, Y. (eds) Advanced Data Mining and Applications. ADMA 2010. Lecture Notes in Computer Science(), vol 6441. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17313-4_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-17313-4_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17312-7
Online ISBN: 978-3-642-17313-4
eBook Packages: Computer ScienceComputer Science (R0)