Using Negation and Phrases in Inducing Rules for Text Classification

  • Stephanie Chua
  • Frans Coenen
  • Grant Malcolm
  • Matías Fernando
  • García Constantino
Conference paper


An investigation into the use of negation in Inductive Rule Learning (IRL) for text classification is described. The use of negated features in the IRL process has been shown to improve effectiveness of classification. However, although in the case of small datasets it is perfectly feasible to include the potential negation of all possible features as part of the feature space, this is not possible for datasets that include large numbers of features such as those used in text mining applications. Instead a process whereby features to be negated can be identified dynamically is required. Such a process is described in the paper and compared with established techniques (JRip, NaiveBayes, Sequential Minimal Optimization (SMO), OlexGreedy). The work is also directed at an approach to text classification based on a “bag of phrases” representation; the motivation here being that a phrase contains semantic information that is not present in single keyword. In addition, a given text corpus typically contains many more key-phrase features than keyword features, therefore, providing more potential features to be negated.


Sequential Minimal Optimization Unique Positive Positive Document Negative Document Unique Negative 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Apté, C., Damerau, F. J., Weiss, S. M.: Automated learning of decision rules for text categorization. ACM Transactions on Information Systems 12, 233-251 (1994)CrossRefGoogle Scholar
  2. 2.
    Bakus, J., Kamel, M.: Document classification using phrases. Caelli, T. and Amin, A. and Duin, R. and de Ridder, D. and Kamel, M. (eds.): Structural, Syntactic, and Statistical Pattern Recognition, Lecture Notes in Computer Science, vol. 2396. Springer Berlin/Heidelberg, pp. 341-354 (2002)Google Scholar
  3. 3.
    Chang, M., Poon, C. K.: Using phrases as features in email classification. Journal of Systems and Software, Elsevier Science Inc., 82, pp. 1036-1045 (2009)Google Scholar
  4. 4.
    Chua, S., Coenen, F, Malcolm, G.: Classification Inductive Rule Learning with Negated Features. In: Proceedings of the 6th International Conference on Advanced Data Mining and Applications (ADMA’10), Part 1, Springer LNAI, pp. 125-136 (2010)Google Scholar
  5. 5.
    Cohen, W.: Fast effective rule induction. In: Proceedings of the 12th Int. Conf. on Machine Learning (ICML), pp. 115-123, Morgan Kaufmann (1995)Google Scholar
  6. 6.
    Fürnkranz, J., Mitchell, T., Riloff, E.: A case study in using linguistic phrases for text categorization on the WWW. In: Working Notes of the AAAI/ICML Workshop on Learning for Text Categorization, AAAI Press, pp. 5-12 (1998)Google Scholar
  7. 7.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I. H.: The WEKA data mining software: An update. SIGKDD Explorations 11 10-18 (2009)CrossRefGoogle Scholar
  8. 8.
    Holmes, G., Trigg, L.: A diagnostic tool for tree based supervised classification learning algorithms. In: Proceedings of the 6th Int. Conf. on Neural Information Processing (ICONIP), pp. 514-519 (1999)Google Scholar
  9. 9.
    Joachims, T.: Text categorization with support vector machines: Learning with many relevant features. In: Proceedings of the 10th European Conf. on Machine Learning (ECML), pp. 137-142 (1998)Google Scholar
  10. 10.
    Johnson, D. E., Oles, F. J., Zhang, T., Goetz, T.: A decision-tree-based symbolic rule induction system for text categorization. The IBM Systems Journal, 41 428-437 (2002)CrossRefGoogle Scholar
  11. 11.
    Lang, K.: Newsweeder: Learning to filter netnews. In: Proceedings of the 12th Int. Conf. on Machine Learning, pp. 331-339 (1995)Google Scholar
  12. 12.
    Lewis, D. D.: Reuters-21578 text categorization test collection, Distribution 1.0, README file (v 1.3). Available at (2004)
  13. 13.
    Li, Z., Li, P., Wei, W., Liu, H., He, J., Liu, T., Du, X.: AutoPCS: A phrase-based text categorization system for similar texts. In: Li, Q., Feng, L., Pei, J., Wang, S., Zhou, X., Zhu, Q.-M. (eds.): Advances in Data and Web Management, Lecture Notes in Computer Science, vol. 5446. Springer Berlin/Heidelberg, pp. 369-380 (2009)Google Scholar
  14. 14.
    McCallum, A., Nigam, K.: A comparison of event model for naive Bayes text classification. In: Proceedings of the AAAI-98 Workshop on Learning for Text Categorization, pp. 41-48 (1998)Google Scholar
  15. 15.
    Rullo, P., Cumbo, C., Policicchio, V. L.: Learning rules with negation for text categorization. In: Proceedings of the 22nd ACM Symposium on Applied Computing, pp. 409-416. ACM (2007)Google Scholar
  16. 16.
    Rullo, P., Policicchio, V., Cumbo, C., Iiritano, S.: Olex: Effective rule learning for text categorization. Transaction on Knowledge and Data Engineering, 21 1118-1132 (2009)CrossRefGoogle Scholar
  17. 17.
    Scott, S., Matwin, S.: Feature engineering for text classification. In: Proceedings of the 16th Int. Conf. on Machine Learning (ICML), pp. 379-388 (1999)Google Scholar
  18. 18.
    Wang, Y. J.: Language-independent pre-processing of large documentbases for text classifcation. PhD thesis (2007)Google Scholar
  19. 19.
    Yang, Y., Liu, X.: A re-examination of text categorization methods. In: Proceedings of the 22nd ACM Int. Conf. on Research and Development in Information Retrieval, pp. 42-49 (1999)Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Stephanie Chua
    • 1
  • Frans Coenen
    • 1
  • Grant Malcolm
    • 1
  • Matías Fernando
    • 1
  • García Constantino
    • 1
  1. 1.Department of Computer ScienceUniversity of LiverpoolLiverpoolUK

Personalised recommendations