Advertisement

Spam Filtering Based on Latent Semantic Indexing

  • Wilfried N. Gansterer
  • Andreas G. K. Janecek
  • Robert Neumayer
Chapter

In this chapter, the classification performance of latent semantic indexing (LSI) applied to the task of detecting and filtering unsolicited bulk or commercial email (UBE, UCE, commonly called “spam”) is studied. Comparisons to the simple vector space model (VSM) and to the extremely widespread, de-facto standard for spam filtering, the SpamAssassin system, are summarized. It is shown that VSM and LSI achieve significantly better classification results than SpamAssassin.

Keywords

Information Gain Feature Selection Method Vector Space Model Email Message Attribute Selection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. I. Androutsopoulos, J. Koutsias, K. Chandrinos, and C.D. Spyropoulos. An experimental comparison of naive bayesian and keyword-based anti-spam filtering with personal e-mail messages. In SIGIR ’00: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 160-167, 2000. Available from World Wide Web: http://doi.acm.org/10.1145/345508.345569.
  2. Apache Software Foundation. SpamAssassin open-source spam filter, 2006. Available from World Wide Web: http://spamassassin.apache.org/.
  3. A. Back. Hashcash—a denial of service counter-measure, 2002. Available from World Wide Web: http://www.hashcash.org/papers/hashcash.pdf.
  4. M.W. Berry, Z. Drmac, and E.R. Jessup. Matrices, vector spaces, and information retrieval. SIAM Review, 41(2):335-362, 1999.zbMATHCrossRefMathSciNetGoogle Scholar
  5. G.V. Cormack and T.R. Lynam. TREC 2005 spam public corpora, 2005. Available from World Wide Web: http://plg.uwaterloo.ca/cgi-bin/cgiwrap/gvcormac/foo.
  6. N. Cristiani and B. Scholkopf. Support vector machines and kernel methods: the new generation of learning machines. AI Magazine, 23:31-41, 2002.Google Scholar
  7. K.R. Gee. Using latent semantic indexing to filter spam. In ACM Symposium on Applied Computing, Data Mining Track, pages 460-464, 2003. Available from World Wide Web: http://ranger.uta.edu/∼cook/pubs/sac03.ps.
  8. W.N. Gansterer, H. Hlavacs, M. Ilger, P. Lechner, and J. Strauß. Token buckets for outgoing spam prevention. In M.H. Hamza, editor, Proceedings of the IASTED International Conference on Communication, Network, and Information Security (CNIS 2005). ACTA Press, Anaheim, CA, November 2005.Google Scholar
  9. W.N. Gansterer, M. Ilger, A. Janecek, P. Lechner, and J. Strauß. Final report project ‘Spamabwehr II’. Technical Report FA384018-5, Institute of Distributed and Multimedia Systems, Faculty of Computer Science, University of Vienna, 05/2006.Google Scholar
  10. W.N. Gansterer, M. Ilger, P. Lechner, R. Neumayer, and J. Strauß. Anti-spam methods—state of the art. Technical Report FA384018-1, Institute of Distributed and Multimedia Systems, Faculty of Computer Science, University of Vienna, March 2005.Google Scholar
  11. W.N. Gansterer, A.G.K. Janecek, and P. Lechner. A reliable component-based architecture for e-mail filtering. In Proceedings of the Second International Conference on Availability, Reliability and Security (ARES 2007), pages 43-50. IEEE Computer Society Press, Los Alamitos, CA, 2007.CrossRefGoogle Scholar
  12. M. Ilger, J. Strauß, W.N. Gansterer, and C. Proschinger. The economy of spam. Technical Report FA384018-6, Institute of Distributed and Multimedia Systems, Faculty of Computer Science, University of Vienna, September 2006.Google Scholar
  13. A.N. Langville. The linear algebra behind search engines. In Journal of Online Mathematics and Its Applications (JOMA), 2005, Online Module, 2005. Available from World Wide Web: http://mathdl.maa.org/mathDL/4/?pa=content&sa=viewDocument&nodeId=636.
  14. A.N. Langville and C.D. Meyer. The use of linear algebra by web search engines. IMAGE Newsletter, 33:2-6, 2004.Google Scholar
  15. Y. Yang and J.O. Pedersen. A comparative study on feature selection in text categorization. In Douglas H. Fisher, editor, Proceedings of ICML-97, 14th International Conference on Machine Learning, pages 412-420, Nashville, TN, 1997. Morgan Kaufmann Publishers, San Francisco. Available from World Wide Web: citeseer.ist.psu.edu/yang97comparative.html.Google Scholar

Copyright information

© Springer-Verlag London Limited 2008

Authors and Affiliations

  • Wilfried N. Gansterer
    • 1
  • Andreas G. K. Janecek
    • 1
  • Robert Neumayer
    • 2
  1. 1.Research Lab for Computational Technologies and ApplicationsUniversity of ViennaViennaAustria
  2. 2.Institute of Software Technology and Interactive SystemsVienna University of TechnologyViennaAustria

Personalised recommendations