Abstract
The aim of the paper is to compare the prediction accuracies obtained using logistic regression, neural networks (NN), C5.0 and M5′ classification techniques on 4 freely available data sets. For this a feedforward neural network with a single hidden layer and using back propagation is built using a new algorithm. The results show that the training accuracies obtained using the new algorithm are better than that obtained using N2C2S algorithm. The cross-validation accuracies and the test prediction accuracies obtained by using both the algorithms are not statistically significantly different. Due to this and also since it is easy to understand and implement than N2C2S algorithm, the proposed algorithm should be preferred than the N2C2S algorithm. Along with this 3 different methods of obtaining weights for neural networks are also compared. The classification results show that NN is better than logistic regression over 2 data sets, equivalent in performance over 2 data sets and has low performance than logistic regression in case of 1 data set. It is observed that M5′ is a better classification technique than other techniques over 1 dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Zhang, G.P.: Neural Networks for classification: A Survey. IEEE Transactions on Systems, Man, and Cybernetics- Part C: Applications and Reviews 30(4), 451–462 (2000)
Liao, J.G., Chin: Logistic regression for disease classification using microarray data: model selection in a large p and small n case. Bioinformatics 23, 1945–1951 (2007)
Zhang, G.P.: Neural Networks for classification: A Survey. IEEE Transactions on Systems, Man, and Cybernetics- Part C: Applications and Reviews 30(4), 451–462 (2000)
Atiya, A.F.: Bankruptcy Prediction for Credit Risk Using Neural Networks: A Survey and New Results. IEEE Transactions on Neural Networks 12(4), 929–935 (2000)
Lacher, R.C., Coats, P.K., Sharma, S.C., Fant, L.F.: A neural network for classifying the financial health of a firm. Eur. J. Oper. Res. 85, 53–65 (1995)
Baxt, W.G.: Use of an artificial neural network for data analysis in clinical decision-making: The diagnosis of acute coronary occlusion. Neural Computing 2, 480–489 (1990)
Mazurowski, M.A., Habas, P.A., Zurada, J.M., Lo, J.Y., Baker, J.A., Tourassi, G.D.: Training neural network classifiers for medical decision making: The effects of imbalanced datasets on classification performance. Neural Networks 21, 427–436 (2007)
Guyon, I.: Applications of neural networks to character recognition. International Journal of Pattern Recognition and Artificial Intelligence 5, 353–382 (1991)
Bourlard, H., Morgan, N.: Continuous speech recognition by connectionist statistical methods. IEEE Transactions on Neural Networks 4, 893–909 (1993)
Ash, T.: Dynamic node creation in backpropagation networks. Connection Science 1(4), 365–375 (2002)
Setiono, R.: Feedforward Neural Network Construction Using Cross Validation. Neural Computation 13(12), 2865–2877 (2001)
Young, S., Downs, T.: CARVE -a constructive algorithm for real-valued examples. IEEE Transaction on Neural Networks 9(6), 1180–1190 (1998)
Setiono, R.: A Neural Network Construction Algorithm which Maximizes the Likelihood Function. Connection Science 7(2), 147–166 (1996)
Kwok, T.-Y., Yeung, D.-Y.: Experimental analysis of input weight freezing in constructing neural networks. In: IEEE International Conference on Neural Networks, vol. 1, pp. 511–516 (1993)
Frank, E., Wang, Y., Inglis, S., Holmes, G., Witten, I.H.: Using Model trees for classification. Machine Learning 32(1), 63–76 (1997)
Quinlan, J.R.: Learning with continuous classes. In: Proceedings Australian Joint Conference on Artificial Intelligence, pp. 343–348. World Scientific, Singapore (1992)
Wang, Y., Witten, I.H.: Induction of model trees for predicting continuous classes. In: Proceedings of the poster papers of the European Conference on Machine Learning. Faculty of Informatics and Statistics, Prague (1997)
Nguyen, D., Widrow, B.: Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights. In: Proceedings of the International Joint Conference on Neural Networks, vol. 3, pp. 21–26 (1990)
Demuth, H., Beale, M., Hagan, M.T.: Neural Network Toolbox 7, User’s Guide. The MathWorks, Inc., Natick, MA, Revised for Version 7.0 (Release 2010b) (September 2010), http://www.mathworks.com
UCI Machine learning repository data sets, http://archive.ics.uci.edu/ml/datasets.html
Welch, B.L.: The generalization of “Student’s” problem when several different population variances are involved. Biometrika 34(1-2), 28–35 (1947)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Thombre, A. (2012). Comparing Logistic Regression, Neural Networks, C5.0 and M5′ Classification Techniques. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2012. Lecture Notes in Computer Science(), vol 7376. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31537-4_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-31537-4_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-31536-7
Online ISBN: 978-3-642-31537-4
eBook Packages: Computer ScienceComputer Science (R0)