Advances in Spam Filtering Techniques

  • Tiago A. Almeida
  • Akebo Yamakami
Part of the Studies in Computational Intelligence book series (SCI, volume 394)


Nowadays e-mail spam is not a novelty, but it is still an important rising problem with a big economic impact in society. Fortunately, there are different approaches able to automatically detect and remove most of those messages, and the best-known ones are based on machine learning techniques, such as Naïve Bayes classifiers and Support Vector Machines. However, there are several different models of Naïve Bayes filters, something the spam literature does not always acknowledge. In this chapter, we present and compare seven different versions of Naïve Bayes classifiers, the well-known linear Support Vector Machine and a new method based on the Minimum Description Length principle. Furthermore, we have conducted an empirical experiment on six public and real non-encoded datasets. The results indicate that the proposed filter is easy to implement, incrementally updateable and clearly outperforms the state-of-the-art spam filters.


Support Vector Machine Collaborative Filter Minimum Description Length Linear Support Vector Machine Spam Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Almeida, T., Yamakami, A.: Content-Based Spam Filtering. In: Proceedings of the 23rd IEEE International Joint Conference on Neural Networks, Barcelona, Spain, pp. 1–7 (2010)Google Scholar
  2. 2.
    Almeida, T., Yamakami, A., Almeida, J.: Evaluation of Approaches for Dimensionality Reduction Applied with Naive Bayes Anti-Spam Filters. In: Proceedings of the 8th IEEE International Conference on Machine Learning and Applications, Miami, FL, USA, pp. 517–522 (2009)Google Scholar
  3. 3.
    Almeida, T., Yamakami, A., Almeida, J.: Filtering Spams using the Minimum Description Length Principle. In: Proceedings of the 25th ACM Symposium On Applied Computing, Sierre, Switzerland, pp. 1856–1860 (2010a)Google Scholar
  4. 4.
    Almeida, T., Yamakami, A., Almeida, J.: Probabilistic Anti-Spam Filtering with Dimensionality Reduction. In: Proceedings of the 25th ACM Symposium On Applied Computing, Sierre, Switzerland, pp. 1804–1808 (2010b)Google Scholar
  5. 5.
    Almeida, T., Almeida, J., Yamakami, A.: How the Dimensionality Reduction Affects the Accuracy of Naive-Bayes Classifiers. Journal of Internet Services and Applications 1(3), 183–200 (2011)CrossRefGoogle Scholar
  6. 6.
    Androutsopoulos, I., Paliouras, G., Michelakis, E.: Learning to Filter Unsolicited Commercial E-Mail. Technical Report 2004/2, National Centre for Scientific Research “Demokritos”, Athens, Greece (2004)Google Scholar
  7. 7.
    Baldi, P., Brunak, S., Chauvin, Y., Andersen, C., Nielsen, H.: Assessing the Accuracy of Prediction Algorithms for Classification: An Overview. Bioinformatics 16(5), 412–424 (2000)CrossRefGoogle Scholar
  8. 8.
    Barron, A., Rissanen, J., Yu, B.: The Minimum Description Length Principle in Coding and Modeling. IEEE Transactions on Information Theory 44(6), 2743–2760 (1998)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Cormack, G.: Email Spam Filtering: A Systematic Review. Foundations and Trends in Information Retrieval 1(4), 335–455 (2008)CrossRefMathSciNetGoogle Scholar
  10. 10.
    Drucker, H., Wu, D., Vapnik, V.: Support Vector Machines for Spam Categorization. IEEE Transactions on Neural Networks 10(5), 1048–1054 (1999)CrossRefGoogle Scholar
  11. 11.
    Forman, G., Scholz, M., Rajaram, S.: Feature Shaping for Linear SVM Classifiers. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France, pp. 299–308 (2009)Google Scholar
  12. 12.
    Grünwald, P.: A Tutorial Introduction to the Minimum Description Length Principle. In: Grünwald, P., Myung, I., Pitt, M. (eds.) Advances in Minimum Description Length: Theory and Applications, pp. 3–81. MIT Press (2005)Google Scholar
  13. 13.
    Hidalgo, J.: Evaluating Cost-Sensitive Unsolicited Bulk Email Categorization. In: Proceedings of the 17th ACM Symposium on Applied Computing, Madrid, Spain, pp. 615–620 (2002)Google Scholar
  14. 14.
    John, G., Langley, P.: Estimating Continuous Distributions in Bayesian Classifiers. In: Proceedings of the 11st International Conference on Uncertainty in Artificial Intelligence, Montreal, Canada, pp. 338–345 (1995)Google Scholar
  15. 15.
    Kolcz, A., Alspector, J.: SVM-based Filtering of E-mail Spam with Content-Specific Misclassification Costs. In: Proceedings of the 1st International Conference on Data Mining, San Jose, CA, USA, pp. 1–14 (2001)Google Scholar
  16. 16.
    Lemire, D.: Scale and Translation Invariant Collaborative Filtering Systems. Information Retrieval 8(1), 129–150 (2005)CrossRefMathSciNetGoogle Scholar
  17. 17.
    Liu, S., Cui, K.: Applications of Support Vector Machine Based on Boolean Kernel to Spam Filtering. Modern Applied Science 3(10), 27–31 (2009)zbMATHGoogle Scholar
  18. 18.
    Losada, D., Azzopardi, L.: Assessing Multivariate Bernoulli Models for Information Retrieval. ACM Transactions on Information Systems 26(3), 1–46 (2008)CrossRefGoogle Scholar
  19. 19.
    Marsono, M., El-Kharashi, N., Gebali, F.: Targeting Spam Control on Middleboxes: Spam Detection Based on Layer-3 E-mail Content Classification. Computer Networks 53(6), 835–848 (2009)CrossRefzbMATHGoogle Scholar
  20. 20.
    Matthews, B.: Comparison of the Predicted and Observed Secondary Structure of T4 Phage Lysozyme. Biochimica et Biophysica Acta 405(2), 442–451 (1975)Google Scholar
  21. 21.
    McCallum, A., Nigam, K.: A Comparison of Event Models for Naive Bayes Text Classication. In: Proceedings of the 15th AAAI Workshop on Learning for Text Categorization, Menlo Park, CA, USA, pp. 41–48 (1998)Google Scholar
  22. 22.
    Metsis, V., Androutsopoulos, I., Paliouras, G.: Spam Filtering with Naive Bayes - Which Naive Bayes? In: Proceedings of the 3rd International Conference on Email and Anti-Spam, Mountain View, CA, USA, pp. 1–5 (2006)Google Scholar
  23. 23.
    Rissanen, J.: Modeling by Shortest Data Description. Automatica 14, 465–471 (1978)CrossRefzbMATHGoogle Scholar
  24. 24.
    Sahami, M., Dumais, S., Hecherman, D., Horvitz, E.: A Bayesian Approach to Filtering Junk E-mail. In: Proceedings of the 15th National Conference on Artificial Intelligence, Madison, WI, USA, pp. 55–62 (1998)Google Scholar
  25. 25.
    Schneider, K.-M.: On word frequency information and negative evidence in naive bayes text classification. In: Vicedo, J.L., Martínez-Barco, P., Muńoz, R., Saiz Noeda, M. (eds.) EsTAL 2004. LNCS (LNAI), vol. 3230, pp. 474–485. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  26. 26.
    Sculley, D., Wachman, G.: Relaxed Online SVMs for Spam Filtering. In: Proceedings of the 30th International ACM SIGIR Conference on Research and Development in Information Retrieval, Amsterdam, The Netherlands, pp. 415–422 (2007)Google Scholar
  27. 27.
    Sculley, D., Wachman, G., Brodley, C.: Spam Filtering using Inexact String Matching in Explicit Feature Space with On-Line Linear Classifiers. In: Proceedings of the 15th Text REtrieval Conference, Gaithersburg, MD, USA, pp. 1–10 (2006)Google Scholar
  28. 28.
    Siefkes, C., Assis, F., Chhabra, S., Yerazunis, W.S.: Combining winnow and orthogonal sparse bigrams for incremental spam filtering. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) PKDD 2004. LNCS (LNAI), vol. 3202, pp. 410–421. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  29. 29.
    Song, Y., Kolcz, A., Gilez, C.: Better Naive Bayes Classification for High-precision Spam Detection. Software – Practice and Experience 39(11), 1003–1024 (2009)CrossRefGoogle Scholar
  30. 30.
    Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)zbMATHGoogle Scholar
  31. 31.
    Zadeh, L.: Fuzzy sets. Information and Control 8(3), 338–353 (1965)CrossRefzbMATHMathSciNetGoogle Scholar
  32. 32.
    Zhang, L., Zhu, J., Yao, T.: An Evaluation of Statistical Spam Filtering Techniques. ACM Transactions on Asian Language Information Processing 3(4), 243–269 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.School of Electrical and Computer EngineeringUniversity of Campinas – UNICAMPCampinasBrazil

Personalised recommendations