ROC Analysis as a Useful Tool for Performance Evaluation of Artificial Neural Networks
In many applications of neural networks, the performance of the network is given by the classification accuracy. While obtaining the classification accuracies, the total true classification is computed, but the number of classification rates of the classes and fault classification rates are not given. This would not be enough for a problem having fatal importance. As an implementation example, a dataset having fatal importance is classified by MLP, RBF, GRNN, PNN and LVQ networks and the real performances of these networks are found by applying ROC analysis.
KeywordsClassification Accuracy Radial Basis Function Receiver Operating Characteristic Receiver Operating Characteristic Curve Radial Basis Function Network
Unable to display preview. Download preview PDF.
- 1.Murphy, P.M., Aha, D.W.: UCI Repository of Machine Learning Databases. University of California, Department of Information and Computer Science (ics.uci.edu), Irvine, CAGoogle Scholar
- 2.Flach, P.A.: Tutorial on The Many Faces of ROC Analysis in Machine Learning. In: The Twenty-First International Conference on Machine Learning, Banff Canada (2004)Google Scholar
- 3.Sboner, A., Eccher, C., Blanzieri, E., Bauer, P., Cristofolini, M., Zumiani, G., Forti, S.: A Multiple Classifier System for Early Melanoma Diagnosis. AI in Medicine 27, 29–44 (2003)Google Scholar
- 4.Metz, C.E.: Basic Principles of ROC Analysis. Sem. Nuc. Med., 283–298 (1978)Google Scholar
- 5.Pinna-Pintor, P., Bobbio, M., Colangelo, S., Veglia, F., Giammaria, M., Cuni, D., Maisano, F., Alfieri, O.: Inaccuracy of Four Coronary Surgery Risk-Adjusted Models to Predict Mortality in Individual Patients. Eur. J. Cardiothorac. Surg., 199–204 (2002)Google Scholar
- 6.Karabulut, H., Toraman, F., Alhan, C., Camur, G., Evrenkaya, S., Dagdelen, S., Tarcan, S.: EuroSCORE Overestimates the Cardiac Operative Risk. Cardiovasc. Surg., 295–298 (2003)Google Scholar
- 7.Gefeller, O., Brenner, H.: How to Correct for Chance Agreement in the Estimation of Sensitivity and Specificity of Diagnostic Tests. Methods Inf. Med., 180–186 (1994)Google Scholar
- 8.Gennari, J.H., Langley, P., Fisher, D.: Models of Incremental Concept Formation. Artificial Intelligence, 11-61 (1989)Google Scholar
- 9.Li, T., Zhu, S., Ogihara, M.: A New Distributed Data Mining Model Based on Similarity. In: Proceedings of the 2003 ACM symposium on Applied computing, Melbourne Florida, pp. 432–436 (2003)Google Scholar
- 12.Prechelt, L.: Summary of Usenet Discussion on Unknown Input Values in Neural Networks (1994)Google Scholar
- 13.Vamplew, P., Adams, A.: Missing Values in Backpropagation Neural Net. Technical Report Department of Computer Science, University of Tasmania (1993)Google Scholar
- 14.Efron, B.: Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation. Journal of the American Statistical Association, 316-331 (1983)Google Scholar