A Probabilistic Model for Clustering Text Documents with Multiple Fields

  • Shanfeng Zhu
  • Ichigaku Takigawa
  • Shuqin Zhang
  • Hiroshi Mamitsuka
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4425)


We address the problem of clustering documents with multiple fields, such as scientific literature with the distinct fields: title, abstract, keywords, main text and references. By taking into consideration of the distinct word distributions of each field, we propose a new probabilistic model, Field Independent Clustering Model (FICM), for clustering documents with multiple fields. The benefits of FICM come not only from integrating the discrimination abilities of each field but also from the power of selecting the most suitable component probabilistic model for each field. We examined the performance of FICM on the problem of clustering biomedical documents with three fields (title, abstract and MeSH). From the genomics track data of TREC 2004 and TREC 2005, we randomly generated 60 datasets where the number of classes in each dataset ranged from 3 to 12. By applying the appropriate configuration of generative models for each field, FICM outperformed a classical multinomial model in 59 out of the total 60 datasets, of which 47 were statistically significant at the 95% level, and FICM also outperformed a multivariate Bernoulli model in 52 out of the total 60 datasets, of which 36 were statistically significant at the 95% level.


Probabilistic Model Multinomial Model Discrimination Ability Cluster Document Multiple Field 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baeza-Yates, R., Ribeiro-Neto, B.: Modern information retrieval. Addison Wesley, New York (1999)Google Scholar
  2. 2.
    Banerjee, A., et al.: Generative Model-based Clustering of Directional Data. In: The proceedings of the SIGKDD’03, pp. 19–28 (2003)Google Scholar
  3. 3.
    Ghosh, J.: Scalable clustering methods for data mining. In: Ye, N. (ed.) Handbook of data mining, Lawrence Erlbaum, Mahwah (2003)Google Scholar
  4. 4.
    Hersh, W.R., et al.: TREC 2004 Genomics Track Overview. In: Voorhees, E.M., Buckland, L.P. (eds.) The proceedings of the Thirteenth Text REtrieval Conference (TREC 2004) (2004)Google Scholar
  5. 5.
    Hersh, W.R., et al.: TREC 2005 Genomics Track Overview. In: Voorhees, E.M., Buckland, L.P. (eds.) The proceedings of the Fourteenth Text REtrieval Conference (TREC 2005) (2005)Google Scholar
  6. 6.
    Jain, A.K., Murty, M.N., Flynn, P.J.: Data Clustering: A Review. ACM Computing Surveys 31(3), 264–323 (1999)CrossRefGoogle Scholar
  7. 7.
    Meila, M., Heckerman, D.: An Experimental Comparison of Model-Based Clustering Methods. Machine Learning 42(1/2), 9–29 (2001)zbMATHCrossRefGoogle Scholar
  8. 8.
    McCallum, A., Nigam, K.: A comparsion of event models for naive Bayes text classification. In: AAAI Workshop on Learning for Text Categorization, pp. 41–48. AAAI Press, Menlo Park (1998)Google Scholar
  9. 9.
    Lewis, D.D.: Naive (Bayes) at forty: the independence assumption in information retrieval. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, Springer, Heidelberg (1998)CrossRefGoogle Scholar
  10. 10.
    Nelson, S.J., et al.: The MeSH Translation Maintenance System: Structure, Interface Design, and Implementation. In: Fieschi, M., et al. (eds.) Proceedings of the 11th World Congress on Medical Informatics, pp. 67–69 (2004)Google Scholar
  11. 11.
    Rigouste, L., Cappé, O., Yvon, F.: Evaluation of a Probabilistic Method for Unsupervised Text Clustering. In: International Symposium on Applied Stochastic Models and Data Analysis, Brest, France (2005)Google Scholar
  12. 12.
    Wheeler, D., et al.: Database resources of the National Center for Biotechnology Information. Nucl. Acids Res. 33, D39–D45 (2005)Google Scholar
  13. 13.
    Yoo, I., Hu, X.: A comprehensive comparison study of document clustering for a biomedical digital library MEDLINE. In: Marchionini, G., et al. (eds.) ACM/IEEE Joint Conference on Digital Libraries, JCDL 2006, pp. 220–229. IEEE Computer Society Press, Los Alamitos (2006)CrossRefGoogle Scholar
  14. 14.
    Zhong, S., Ghosh, J.: A unified framework for model-based clustering. Journal of Machine Learning Research 4, 1001–1037 (2003)CrossRefMathSciNetGoogle Scholar
  15. 15.
    Zhong, S., Ghosh, J.: Generative model-based document clustering: a comparative study. Knowledge and Information Systems 8(3), 374–384 (2005)CrossRefGoogle Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Shanfeng Zhu
    • 1
  • Ichigaku Takigawa
    • 1
  • Shuqin Zhang
    • 2
  • Hiroshi Mamitsuka
    • 1
  1. 1.Bioinformatics Center, Institute for Chemical Research, Kyoto UniversityJapan
  2. 2.Department of Mathematics, The University of Hong KongHong Kong

Personalised recommendations