Clustering Ensemble for Spam Filtering
One of the main problems that modern e-mail systems face is the management of the high degree of spam or junk mail they recieve. Those systems are expected to be able to distinguish between legitimate mail and spam; in order to present the final user as much interesting information as possible. This study presents a novel hybrid intelligent system using both unsupervised and supervised learning that can be easily adapted to be used in an individual or collaborative system. The system divides the spam filtering problem into two stages: firstly it divides the input data space into different similar parts. Then it generates several simple classifiers that are used to classify correctly messages that are contained in one of the parts previously determined. That way the efficiency of each classifier increases, as they can specialize in separate the spam from certain types of related messages. The hybrid system presented has been tested with a real e-mail data base and a comparison of its results with those obtained from other common classification methods is also included. This novel hybrid technique proves to be effective in the problem under study.
KeywordsConcept Drift Cluster Ensemble Collaborative System Hybrid Intelligent System Apache Software Foundation
Unable to display preview. Download preview PDF.
- 1.Ruta, D., Gabrys, B.: An overview of classifier fusion methods. Computing and Information Systems 7(1), 1–10 (2000)Google Scholar
- 2.Schapire, R.E.: The strength of weak learnability. Machine Learning 5(2), 197–227 (1990)Google Scholar
- 4.Corchado, E., Baruque, B.: Wevos-visom: An ensemble summarization algorithm for enhanced data visualization. Neurocomputing ( in press, 2011)Google Scholar
- 11.Dara, R., Kremer, S.C., Stacey, D.A.: Clustering unlabelled data with SOMs improves classi cation of labelled real-world data. In: Proc. IEEE World Congress, on Computational Intelligence, pp. 2237–2242 ( May 2002)Google Scholar
- 12.Ultsch, A.: Self-organizing neural networks for visualization and classification. In: Proc. Conf. Soc. for Information and Classification (1992)Google Scholar
- 13.Ultsch, A.: U*-matrix: A tool to visualize clusters in high dimensional data. Tech. rep., Department of Computer Science, University of Marburg (2003)Google Scholar
- 14.Kuncheva, L.I.: Clustering-and-selection model for classifier combination. In: KES, pp. 185–188 (2000)Google Scholar
- 16.Breiman, L.: Bagging predictors. In: Machine Learning, vol. 24(2), pp. 123–140 (1996)Google Scholar
- 17.Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: International Conference on Machine Learning, vol. 156, p. 148 (1996)Google Scholar
- 18.Asuncion, A., Newman, D.J.: UCI machine learning repository (2007)Google Scholar
- 19.Apache Software Foundation. Spamassasin public corpus (2006)Google Scholar
- 20.Singhal, A.: Modern information retrieval: A brief overview. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering 24(4), 35–43 (2001)Google Scholar