Using Classifier Performance Visualization to Improve Collective Ranking Techniques for Biomedical Abstracts Classification

  • Alexandre Kouznetsov
  • Nathalie Japkowicz
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6085)

Abstract

The purpose of this work is to improve on the selection of algorithms for classifier committees applied to reducing the workload of human experts in building systematic reviews used in evidence-based medicine. We focus on clustering pre-selected classifiers based on a multi-measure prediction performance evaluation expressed in terms of a projection from a high-dimensional space to a visualizable two-dimensional one. The best classifier was selected from each cluster and included in the committee. We applied the committee of classifiers to rank biomedical abstracts based on the predicted relevance to the topic under review. We identified a subset of abstracts that represents the bottom of the ranked list (predicted as irrelevant). We used False Negatives (relevant articles mistakenly ranked at the bottom) as a final performance measure. Our early experiments demonstrate that the classifier committee built using our new approach outperformed committees of classifiers arbitrary created from the same list of pre-selected classifiers.

Keywords

Machine Learning Automatic Text Classification Systematic Reviews Ranking Algorithms Scientific Visualization 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Sackett, D., Rosenberg, W., Gray, J., Haynes, R., Richardson, W.: Evidence based medicine: what it is and what it isn’t. BMJ 312 (7023): 71-2. PMID 8555924 (1996)Google Scholar
  2. 2.
    Kouznetsov, A., Matwin, S., Inkpen, D., Razavi, A., Frunza, O., Sehatkar, M., Seaward, L., O’Blenis, P.: Classifying Biomedical Abstracts Using Committees of Classifiers and Collective Ranking Techniques. In: Canadian Artificial Intelligence Conference (2009)Google Scholar
  3. 3.
    Alaiz-Rodriguez, R., Japkowicz, N., Tischer, P.: Visualizing Classifier Performance. In: Proceedings of the 20th IEEE International Conference on Tools for Artificial Intelligence, ICTAI 2008 (2008)Google Scholar
  4. 4.
    Alaiz-Rodriguez, R., Japkowicz, N., Tischer, P.: A Visualization-Based Exploratory Tool for Classifier Comparison with respect to Multiple Metrics and Multiple Domains. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part II. LNCS (LNAI), vol. 5212, pp. 660–665. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Japkowicz, N., Sanghi, P., Tischer, P.: A Projection-Based Framework for Classifier Performance Evaluation. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part I. LNCS (LNAI), vol. 5211, pp. 548–563. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  6. 6.
    Razavi, A.H., Matwin, S., Inkpen, D., Kouznetsov, A.: Parameterized Contrast in Second Order Soft Co-Occurrences: A Novel Text Representation Technique in Text Mining and Knowledge Extraction. In: Second International Workshop on Semantic Aspects in Data Mining (SADM 2009), USA, Miami (2009)Google Scholar
  7. 7.
    Software package Weka, http://www.cs.waikato.ac.nz/ml/weka/
  8. 8.
    Cox, T., Cox, M.: Multidimensional Scaling. Chapman and Hall, Boca Raton (October 1994)MATHGoogle Scholar
  9. 9.
    Visualization Software for Clasifier Evaluation, http://www.site.uottawa.ca/~nat/Visualization_Software/visualization.html

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Alexandre Kouznetsov
    • 1
  • Nathalie Japkowicz
    • 2
  1. 1.Department of Computer Science and Applied StatisticsUniversity of New Brunswick Saint John 
  2. 2.School of Information Technology and EngineeringUniversity of Ottawa 

Personalised recommendations