Weighting of Features by Sequential Selection

  • Urszula Stańczyk
Part of the Studies in Computational Intelligence book series (SCI, volume 584)


Constructing a set with characteristic features for supervised classification is a task which can be considered as preliminary for the intended purpose, just a step to take on the way, yet with its significance and bearing on the outcome, the level of difficulty and computational costs involved, the problem has evolved in time to constitute by itself a field of intense study. We can use statistics, available expert domain knowledge, specialised procedures, analyse the set of all accessible features and reduce them backward, we can examine them one by one and select them forward. The process of sequential selection can be conditioned by the performance of a classification system, while exploiting a wrapper model, and the observations with respect to selected variables can result in assignment of weights and ranking. The chapter illustrates weighting of features with the procedures of sequential backward and forward selection for rule and connectionist classifiers employed in the stylometric task of authorship attribution.


Weighting Ranking of features Sequential selection Forward selection Backward selection DRSA ANN Stylometry Authorship attribution 


  1. 1.
    Ahonen, H., Heinonen, O., Klemettinen, M., Verkamo, A.: Applying data mining techniques in text analysis. Technical Report C-1997-23, Department of Computer Science, University of Helsinki, Finland (1997)Google Scholar
  2. 2.
    Argamon, S., Burns, K., Dubnov, S. (eds.): The Structure of Style: Algorithmic Approaches to Understanding Manner and Meaning. Springer, Berlin (2010)Google Scholar
  3. 3.
    Argamon, S., Karlgren, J., Shanahan, J.: Stylistic analysis of text for information access. In: Proceedings of the 28th International ACM Conference on Research and Development in Information Retrieval, Brazil (2005)Google Scholar
  4. 4.
    Baayen, H., van Haltern, H., Tweedie, F.: Outside the cave of shadows: using syntactic annotation to enhance authorship attribution. Lit. Linguist. Comput. 11(3), 121–132 (1996)CrossRefGoogle Scholar
  5. 5.
    Bayardo Jr., R., Agrawal, R.: Mining the most interesting rules. In: Proceedings of the 5th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 145–154 (1999)Google Scholar
  6. 6.
    Berber Sardinha, T.: Using key words in text analysis: practical aspects (1999). Available on-line from
  7. 7.
    Burrows, J.: Textual analysis. In: Schreibman, S., Siemens, R., Unsworth, J. (eds.) A Companion to Digital Humanities. Blackwell, Oxford (2004)Google Scholar
  8. 8.
    Craig, H.: Stylistic analysis and authorship studies. In: Schreibman, S., Siemens, R., Unsworth, J. (eds.) A Companion to Digital Humanities. Blackwell, Oxford (2004)Google Scholar
  9. 9.
    Dash, M., Liu, H.: Feature selection for classification. Intell. Data Anal. 1, 131–156 (1997)CrossRefGoogle Scholar
  10. 10.
    Dash, M., Liu, H.: Consistency-based search in feature selection. Artif. Intell. 151, 155–176 (2003)CrossRefMathSciNetzbMATHGoogle Scholar
  11. 11.
    Fiesler, E., Beale, R.: Handbook of Neural Computation. Oxford University Press, Oxford (1997)CrossRefGoogle Scholar
  12. 12.
    Greco, S., Matarazzo, B., Słowiñski, R.: Advances in multiple criteria decision making. In: Gal, T., Hanne, T., Stewart, T. (eds.) The Use of Rough Sets and Fuzzy Sets in Multi Criteria Decision Making Chap. 14, pp. 14.1–14.59. Kluwer Academic Publishers, Boston (1999)Google Scholar
  13. 13.
    Greco, S., Matarazzo, B., Słowiński, R.: Rough set theory for multicriteria decision analysis. Eur. J. Oper. Res. 129(1), 1–47 (2001)CrossRefzbMATHGoogle Scholar
  14. 14.
    Greco, S., Matarazzo, B., Słowiński, R.: Dominance-based rough set approach as a proper way of handling graduality in rough set theory. Trans. Rough Sets 7, 36–52 (2007)Google Scholar
  15. 15.
    Greco, S., Słowiński, R., Stefanowski, J.: Evaluating importance of conditions in the set of discovered rules. Lect. Notes Artif. Intell. 4482, 314–321 (2007)Google Scholar
  16. 16.
    Greco, S., Słowiński, R., Stefanowski, J., Żurawski, M.: Incremental versus non-incremental rule induction for multicriteria classification. Trans. Rough Sets 2, 33–53 (2004)Google Scholar
  17. 17.
    Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)zbMATHGoogle Scholar
  18. 18.
    Jelonek, J., Krawiec, K., Stefanowski, J.: Comparative study of feature subset selection techniques for machine learning tasks. In: Proceedings of the 7th Workshop on Intelligent Information Systems (1998)Google Scholar
  19. 19.
    Jensen, R., Shen, Q.: Computational Intelligence and Feature Selection. Wiley, Hoboken (2008)CrossRefGoogle Scholar
  20. 20.
    John, G., Kohavi, R., Pfleger, K.: Irrelevant features and the subset selection problem. In: Cohen, W., Hirsh, H. (eds.) Machine Learning: Proceedings of the 11th International Conference, pp. 121–129. Morgan Kaufmann Publishers (1994)Google Scholar
  21. 21.
    Kavzoglu, T., Mather, P.: Assessing artificial neural network pruning algorithms. In: Proceedings of the 24th Annual Conference and Exhibition of the Remote Sensing Society, pp. 603–609. Greenwich (2011)Google Scholar
  22. 22.
    Khmelev, D., Tweedie, F.: Using Markov chains for identification of writers. Lit. Linguist. Comput. 16(4), 299–307 (2001)CrossRefGoogle Scholar
  23. 23.
    Kingston, G., Maier, H., Lambert, M.: A statistical input pruning method for artificial neural networks used in environmental modelling. In: Transactions of the 2nd Biennial Meeting of the International Environmental Modelling and Software Society, pp. 87–92. Osnabrueck (2004)Google Scholar
  24. 24.
    Liu, H., Motoda, H.: Computational Methods of Feature Selection. Chapman & Hall/CRC, Boca Raton (2008)Google Scholar
  25. 25.
    Lynam, T., Clarke, C., Cormack, G.: Information extraction with term frequencies. In: Proceedings of the Human Language Technology Conference, pp. 1–4. San Diego (2001)Google Scholar
  26. 26.
    Moshkov, M., Piliszczuk, M., Zielosko, B.: On partial covers, reducts and decision rules with weights. Trans. Rough Sets 6, 211–246 (2006)Google Scholar
  27. 27.
    Moshkow, M., Skowron, A., Suraj, Z.: On covering attribute sets by reducts. In: Kryszkiewicz, M., Peters, J., Rybinski, H., Skowron, A. (eds.) Rough Sets and Emerging Intelligent Systems Paradigms. LNCS (LNAI), vol. 4585, pp. 175–180. Springer, Berlin (2007)CrossRefGoogle Scholar
  28. 28.
    Munro, R.: A Queing-theory model of word frequency distributions. In: Proceedings of the 1st Australasian Language Technology Workshop, pp. 1–8. Melbourne (2003)Google Scholar
  29. 29.
    Pawlak, Z.: Rough sets. Int. J. Comput. Inform. Sci. 11(5), 341–356 (1982)CrossRefMathSciNetzbMATHGoogle Scholar
  30. 30.
    Pawlak, Z.: Rough sets and intelligent data analysis. Inf. Sci. 147, 1–12 (2002)CrossRefMathSciNetzbMATHGoogle Scholar
  31. 31.
    Peng, R.: Statistical aspects of literary style. Bachelor’s Thesis, Yale University (1999)Google Scholar
  32. 32.
    Peng, R., Hengartner, H.: Quantitative analysis of literary styles. Am. Stat. 56(3), 15–38 (2002)CrossRefMathSciNetGoogle Scholar
  33. 33.
    Shen, Q.: Rough feature selection for intelligent classifiers. Trans. Rough Sets 7, 244–255 (2006)Google Scholar
  34. 34.
    Sikora, M.: Rule quality measures in creation and reduction of data rule models. In: Greco, S., Hata, Y., Hirano, S., Inuiguchi, M., Miyamoto, S., Nguyen, H., Słowiński, R. (eds.) Rough Sets and Current Trends in Computing. Lecture Notes in Computer Science, vol. 4259, pp. 716–725. Springer (2006)Google Scholar
  35. 35.
    Słowiński, R., Greco, S., Matarazzo, B.: Dominance-Based Rough Set Approach to Reasoning About Ordinal Data. LNCS (LNAI), vol. 4585, pp. 5–11 (2007)Google Scholar
  36. 36.
    Stańczyk, U.: Relative reduct-based selection of features for ANN classifier. In: Cyran, K., Kozielski, S., Peters, J., Stańczyk, U., Wakulicz-Deja, A. (eds.) Man-Machine Interactions. AISC, vol. 59, pp. 335–344. Springer, Berlin (2009)CrossRefGoogle Scholar
  37. 37.
    Stańczyk, U.: DRSA decision algorithm analysis in stylometric processing of literary texts. In: Szczuka, M., Kryszkiewicz, M., Ramanna, S., Jensen, R., Hu, Q. (eds.) Rough Sets and Current Trends in Computing. LNCS (LNAI), vol. 6086, pp. 600–609. Springer, Berlin (2010)CrossRefGoogle Scholar
  38. 38.
    Stańczyk, U.: Reduct-based analysis of decision algorithms: application in computational stylistics. In: Corchado, M., Kurzyński, E., Woźniak, M.(eds.) Hybrid Artificial Intelligence Systems. Part 2. LNCS (LNAI), vol. 6679, pp. 295–302. Springer (2011)Google Scholar
  39. 39.
    Stańczyk, U.: Rule-based approach to computational stylistics. In: Bouvry, P., Kłopotek, M., Marciniak, M., Mykowiecka, A., Rybiński, H. (eds.) Security and Intelligent Information Systems. LNCS (LNAI), vol. 7053, pp. 168–179. Springer, Berlin (2012)CrossRefGoogle Scholar
  40. 40.
    Stańczyk, U.: On preference order of DRSA conditional attributes for computational stylistics. In: Decker, H., Lhotska, L., Link, S., Tjoa, B.J,A. (eds.) Database and Expert Systems Applications. LNCS, pp. 26–33. Springer, Berlin (2013)CrossRefGoogle Scholar
  41. 41.
    Stańczyk, U.: Relative reduct-based estimation of relevance for stylometric features. In: Catania, B., Guerrini, G., Pokorny, J. (eds.) Advances in Databases and Information Systems. LNCS, vol. 8133, pp. 135–147. Springer, Berlin (2013)CrossRefGoogle Scholar
  42. 42.
    Waugh, S., Adams, A., Tweedie, F.: Computational stylistics using artificial neural networks. Lit. Linguist. Comput. 15(2), 187–198 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Institute of InformaticsSilesian University of TechnologyGliwicePoland

Personalised recommendations