Spam Filtering Based on Latent Semantic Indexing
- 1.9k Downloads
In this chapter, the classification performance of latent semantic indexing (LSI) applied to the task of detecting and filtering unsolicited bulk or commercial email (UBE, UCE, commonly called “spam”) is studied. Comparisons to the simple vector space model (VSM) and to the extremely widespread, de-facto standard for spam filtering, the SpamAssassin system, are summarized. It is shown that VSM and LSI achieve significantly better classification results than SpamAssassin.
KeywordsInformation Gain Feature Selection Method Vector Space Model Email Message Attribute Selection
Unable to display preview. Download preview PDF.
- I. Androutsopoulos, J. Koutsias, K. Chandrinos, and C.D. Spyropoulos. An experimental comparison of naive bayesian and keyword-based anti-spam filtering with personal e-mail messages. In SIGIR ’00: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 160-167, 2000. Available from World Wide Web: http://doi.acm.org/10.1145/345508.345569.
- Apache Software Foundation. SpamAssassin open-source spam filter, 2006. Available from World Wide Web: http://spamassassin.apache.org/.
- A. Back. Hashcash—a denial of service counter-measure, 2002. Available from World Wide Web: http://www.hashcash.org/papers/hashcash.pdf.
- G.V. Cormack and T.R. Lynam. TREC 2005 spam public corpora, 2005. Available from World Wide Web: http://plg.uwaterloo.ca/cgi-bin/cgiwrap/gvcormac/foo.
- N. Cristiani and B. Scholkopf. Support vector machines and kernel methods: the new generation of learning machines. AI Magazine, 23:31-41, 2002.Google Scholar
- K.R. Gee. Using latent semantic indexing to filter spam. In ACM Symposium on Applied Computing, Data Mining Track, pages 460-464, 2003. Available from World Wide Web: http://ranger.uta.edu/∼cook/pubs/sac03.ps.
- W.N. Gansterer, H. Hlavacs, M. Ilger, P. Lechner, and J. Strauß. Token buckets for outgoing spam prevention. In M.H. Hamza, editor, Proceedings of the IASTED International Conference on Communication, Network, and Information Security (CNIS 2005). ACTA Press, Anaheim, CA, November 2005.Google Scholar
- W.N. Gansterer, M. Ilger, A. Janecek, P. Lechner, and J. Strauß. Final report project ‘Spamabwehr II’. Technical Report FA384018-5, Institute of Distributed and Multimedia Systems, Faculty of Computer Science, University of Vienna, 05/2006.Google Scholar
- W.N. Gansterer, M. Ilger, P. Lechner, R. Neumayer, and J. Strauß. Anti-spam methods—state of the art. Technical Report FA384018-1, Institute of Distributed and Multimedia Systems, Faculty of Computer Science, University of Vienna, March 2005.Google Scholar
- W.N. Gansterer, A.G.K. Janecek, and P. Lechner. A reliable component-based architecture for e-mail filtering. In Proceedings of the Second International Conference on Availability, Reliability and Security (ARES 2007), pages 43-50. IEEE Computer Society Press, Los Alamitos, CA, 2007.CrossRefGoogle Scholar
- M. Ilger, J. Strauß, W.N. Gansterer, and C. Proschinger. The economy of spam. Technical Report FA384018-6, Institute of Distributed and Multimedia Systems, Faculty of Computer Science, University of Vienna, September 2006.Google Scholar
- A.N. Langville. The linear algebra behind search engines. In Journal of Online Mathematics and Its Applications (JOMA), 2005, Online Module, 2005. Available from World Wide Web: http://mathdl.maa.org/mathDL/4/?pa=content&sa=viewDocument&nodeId=636.
- A.N. Langville and C.D. Meyer. The use of linear algebra by web search engines. IMAGE Newsletter, 33:2-6, 2004.Google Scholar
- WEKA, 2006. Available from World Wide Web: http://www.cs.waikato. ac.nz/ml/weka/. Data Mining Software in Java.
- Y. Yang and J.O. Pedersen. A comparative study on feature selection in text categorization. In Douglas H. Fisher, editor, Proceedings of ICML-97, 14th International Conference on Machine Learning, pages 412-420, Nashville, TN, 1997. Morgan Kaufmann Publishers, San Francisco. Available from World Wide Web: citeseer.ist.psu.edu/yang97comparative.html.Google Scholar