Boosting for Text Classification with Semantic Features

  • Stephan Bloehdorn
  • Andreas Hotho
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3932)


Current text classification systems typically use term stems for representing document content. Semantic Web technologies allow the usage of features on a higher semantic level than single words for text classification purposes. In this paper we propose such an enhancement of the classical document representation through concepts extracted from background knowledge. Boosting, a successful machine learning technique is used for classification. Comparative experimental evaluations in three different settings support our approach through consistent improvement of the results. An analysis of the results shows that this improvement is due to two separate effects.


Noun Phrase Semantic Feature Feature Representation Lexical Entry Weak Learner 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bodner, R.C., Song, F.: Knowledge-Based Approaches to Query Expansion in Information Retrieval. In: Advances in Artificial Intelligence. Springer, New York (1996)Google Scholar
  2. 2.
    Bozsak, E., et al.: KAON – Towards a Large Scale Semantic Web. In: Bauknecht, K., Tjoa, A.M., Quirchmayr, G. (eds.) EC-Web 2002. LNCS, vol. 2455, pp. 304–313. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  3. 3.
    Cai, L., Hofmann, T.: Text Categorization by Boosting Automatically Extracted Concepts. In: Proc. of the 26th Annual Int. ACM SIGIR Conference on Research and Development in Informaion Retrieval, Toronto, Canada. ACM Press, New York (2003)Google Scholar
  4. 4.
    Freund, Y., Schapire, R.E.: A Decision Theoretic Generalization of On-Line Learning and an Application to Boosting. In: Vitányi, P.M.B. (ed.) EuroCOLT 1995. LNCS, vol. 904, pp. 23–37. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  5. 5.
    Handschuh, S., Staab, S. (eds.): Annotation for the Semantic Web. IOS Press, Amsterdam (2003)zbMATHGoogle Scholar
  6. 6.
    Hersh, W.R., Buckley, C., Leone, T.J., Hickam, D.H.: Ohsumed: An Interactive Retrieval Evealuation and new large Test Collection for Research. In: Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval. ACM Press, New York (1994)Google Scholar
  7. 7.
    Hotho, A., Staab, S., Stumme, G.: Wordnet improves Text Document Clustering. In: Proc. of the Semantic Web Workshop of the 26th Annual International ACM SIGIR Conference, Toronto, Canada (2003)Google Scholar
  8. 8.
    Ide, N., Véronis, J.: Introduction to the Special Issue on Word Sense Disambiguation: The State of the Art. Computational Linguistics 24(1), 1–40 (1998)Google Scholar
  9. 9.
    Joachims, T.: Text Categorization with Support Vector Machines: Learning With Many Relevant Features. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398. Springer, Heidelberg (1998)Google Scholar
  10. 10.
    Kehagias, A., Petridis, V., Kaburlasos, V.G., Fragkou, P.: A Comparison of Word- and Sense-Based Text Categorization Using Several Classification Algorithms. Journal of Intelligent Information Systems 21(3), 227–247 (2000)CrossRefGoogle Scholar
  11. 11.
    Lauser, B.: Semi-Automatic Ontology Engineering and Ontology Supported Document Indexing in a Multilingual Environment. Master’s thesis, University of Karlsruhe (2003)Google Scholar
  12. 12.
    Meir, R.: An Introduction to Boosting and Leveraging. In: Advanced Lectures on Machine Learning. LNCS. Springer, Heidelberg (2003)Google Scholar
  13. 13.
    Miller, G.A., Fellbaum, C., Gross, D., Miller, K.J.: Introduction to WordNet: an On-Line Lexical Database. International Journal of Lexicography 3(4), 235–244 (1990)CrossRefGoogle Scholar
  14. 14.
    Porter, M.F.: An algorithm for suffix stripping. Program 14(3), 130–137 (1980)CrossRefGoogle Scholar
  15. 15.
    Salton, G.: Automatic Text Processing. Addison-Wesley Publishing Inc., Boston, MA, USA (1989)Google Scholar
  16. 16.
    Schapire, R.E., Singer, Y.: BoosTexter: A Boosting-based System for Text Categorization. Machine Learning 39(2/3), 135–168 (2000)CrossRefzbMATHGoogle Scholar
  17. 17.
    Sebastiani, F.: Machine Learning in Automated Text Categorization. ACM Computing Surveys 34(1), 1–47 (2002)CrossRefGoogle Scholar
  18. 18.
    Wang, B.B., Mckay, R.I., Abbass, H.A., Barlow, M.: A comparative study for domain ontology guided feature extraction. In: Proceedings of the 26th Australian Computer Science Conference (ACSC 2003), pp. 69–78. Australian Computer Society (2003)Google Scholar
  19. 19.
    Yang, Y.: An Evaluation of Statistical Approaches to Text Categorization. Information Retrieval 1(1-2), 69–90 (1999)CrossRefGoogle Scholar
  20. 20.
    Yang, Y., Liu, X.: A re-examination of text categorization methods. In: Proceedings of SIGIR 1999, 22nd ACM International Conference on Research and Development in Information Retrieval, Berkeley, CA (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Stephan Bloehdorn
    • 1
  • Andreas Hotho
    • 2
  1. 1.Institute AIFB, Knowledge Management Research GroupUniversity of KarlsruheGermany
  2. 2.Knowledge and Data Engineering GroupUniversity of KasselGermany

Personalised recommendations