Abstract
Recently there has been an explosion of work on the design of automated link discovery (LD) systems but little work has been done to investigate metrics to evaluate the performance of such systems. This paper states the link discovery system evaluation problem, explores the issues involved in evaluating the performance of link discovery systems by relating it to the traditional problems of evaluating classification systems, and describes metrics I derived to evaluate the LD systems being developed under DARPA’s EELD program.
This work was performed during author’s tenure at Information Extraction & Transport Inc. 1901 North Fort Myer Drive, Arlington, VA 22209, under a DARPA contract.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Feng, C., Sutherland, A. King, R., Muggleton, S. and Henry, R., 1993. Comparison of Machine Learning Classifiers to statistics and neural networks in Proceedings of the Fourth International Workshop on Artificial Intelligence and Statistics, pages 41–52, San Mateo, CA: Morgan Kaufmann.
Goldberg, H., and Senator, T. (1995). Restructuring Databases for Knowledge Discovery by Consolidation and Link Formation in Proceedings of the 1st Int. Conf. on Knowledge Discovery and Data Mining Menlo Park, CA: AAAI Press.
IET, 2002. Performance Evaluation Specifications for EELD, Technical Report, Information Extraction & Transport Inc. (http://www.iet.com/Projects/EELD)
Han, J. and Kamber, M., 2000. Data Mining: Concepts and Techniques, San Francisco, CA: Morgan Kaufmann.
Lenat, D. and Guha, R. 1990 Building large knowledge-based systems: representations and interface in the Cyc project, Reading MA: Addison-Wesley.
Salton, G., 1971. The SMART Information Retreival System. Englewood Cliffs, NJ: Prentice-Hall.
Upal, M. A. 1995. Monte Carlo Comparison of Non-hierarchical Unsupervised Classifiers, Masters Dissertation, Department of Computer Science, University of Saskatchewan.
Upal, M. A. and Neufeld, E. 1996 Comparison of non-hierarchical unsupervised classifiers in Proceedings of the International Conference on Information, Statistics and Induction in Science, pages 342–353, Singapore: World Scientific.
Weiss, S. and Kulikowski, C., 1991. Computer Systems That Learn: Classification and Prediction Methods From Statistics, Neural Networks, Machine Learning, and Expert Systems. San Mateo, CA: Morgan Kaufmann.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Upal, M.A. (2003). Performance Evaluation Metrics for Link Discovery Systems. In: Abraham, A., Franke, K., Köppen, M. (eds) Intelligent Systems Design and Applications. Advances in Soft Computing, vol 23. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-44999-7_26
Download citation
DOI: https://doi.org/10.1007/978-3-540-44999-7_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40426-2
Online ISBN: 978-3-540-44999-7
eBook Packages: Springer Book Archive