A New Dimensionality Reduction Technique Based on HMM for Boosting Document Classification
Many classification problems, such as text classification, require the ability to handle the high dimension of a structured representation of the documents. The enormous size of the data would result in burdensome computations. Consequently, there is a strong need for reducing the quantity of handled information to develop the classification process. In this paper, we propose a dimensionality reduction technique on text datasets based on a clustering method to group documents with a simple Hidden Markov Model to represent them. We have applied the new method on the OHSUMED benchmark text corpora using the \(k\)-NN and SVM classifiers. The results obtained are very satisfactory and demonstrate the suitability of the proposed technique for the problem of dimensionality reduction and document classification.
KeywordsHidden markov model Text classification Dimensionality reduction Document clustering Similarity-based classification
This work has been funded from the European Union Seventh Framework Programme [FP7/REGPOT-2012-2013.1] under grant agreement n 316265, BIOCAPS, and the “Platform of integration of intelligent techniques for analysis of biomedical information” project (TIN2013-47153-C3-3-R) from Spanish Ministry of Economy and Competitiveness.
- 1.Sebastiani, F.: Text categorization. In: Text Mining and its Applications to Intelligence, CRM and Knowledge Management, pp. 109–129. WIT Press (2005)Google Scholar
- 2.Tsimboukakis, N., Tambouratzis, G.: Document classification system based on hmm word map. In Proceedings of the 5th International Conference on Soft Computing as Transdisciplinary Science and Technology, CSTST ’08, ACM, pp. 7–12, New York, NY, USA (2008)Google Scholar
- 3.Janecek, A.G., Gansterer, W.N., Demel, M.A., Ecker, G.F.: On the relationship between feature selection and classification accuracy. JMLR Workshop Conf. Proc. 4, 90–105 (2008)Google Scholar
- 6.Seara Vieira, A., Iglesias, E.L., Borrajo, L.: T-HMM: a novel biomedical text classifier based on hidden markov models. In: 8th International Conference on Practical Applications of Computational Biology and Bioinformatics (PACBB 2014), volume 294 of Advances in Intelligent Systems and Computing, pp. 225–234. Springer International Publishing (2014)Google Scholar
- 7.Pelleg, D., Moore, A.W.: X-means: extending k-means with efficient estimation of the number of clusters. In Proceedings of the Seventeenth International Conference on Machine Learning, ICML ’00, Morgan Kaufmann Publishers Inc, pp. 727–734, San Francisco, CA, USA (2000)Google Scholar
- 8.Rabiner, L.R.: Readings in speech recognition. Chapter A tutorial on hidden Markov models and selected applications in speech recognition, pp. 267–296. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1990)Google Scholar
- 9.Hersh, W.R., Buckley, C., Leone, T.J., Hickam, D.H.: Ohsumed: an interactive retrieval evaluation and new large test collection for research. In SIGIR, pp. 192–201 (1994)Google Scholar
- 10.Baeza-Yates, R.A., Ribeiro-Neto, B.: Modern Information Retrieval. Addison-Wesley Longman (1999)Google Scholar
- 11.Caporaso, J.G., Baumgartner, W.A., Cohen, K.B., Johnson, H.L., Paquette, J., Hunter, L.: Concept recognition and the trec genomics tasks. In: Voorhees, E.M., Buckland, L.P. (eds.), TREC, volume Special Publication 500–266. National Institute of Standards and Technology (NIST) (2005)Google Scholar
- 12.Chang, C., Lin, C.: Libsvm: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3):27:1–27:27 (2011)Google Scholar