Skip to main content
Download PDF

Consensus Clustering: A Resampling-Based Method for Class Discovery and Visualization of Gene Expression Microarray Data

Machine Learning volume 52, pages 91–118 (2003)Cite this article

Abstract

In this paper we present a new methodology of class discovery and clustering validation tailored to the task of analyzing gene expression data. The method can best be thought of as an analysis approach, to guide and assist in the use of any of a wide range of available clustering algorithms. We call the new methodology consensus clustering, and in conjunction with resampling techniques, it provides for a method to represent the consensus across multiple runs of a clustering algorithm and to assess the stability of the discovered clusters. The method can also be used to represent the consensus over multiple runs of a clustering algorithm with random restart (such as K-means, model-based Bayesian clustering, SOM, etc.), so as to account for its sensitivity to the initial conditions. Finally, it provides for a visualization tool to inspect cluster number, membership, and boundaries. We present the results of our experiments on both simulated data and real gene expression data aimed at evaluating the effectiveness of the methodology in discovering biologically meaningful clusters.

References

  • Banfield, J., & Raftery, A. E. (1993). Model-based Gaussian and non-Gaussian clustering. Biometrics, 49, 803–821.

    Google Scholar 

  • Bar-Joseph, Z., Demaine, E. D., Gifford, D. K., Hamel, A. M., Jaakkola, T. S., & Srebro, N. (2002). K-ary clustering with optimal leaf ordering for gene expression data. Bioinformatics, to appear.

  • Ben-Hur, A., Elisseeff, A., & Guyon, I. (2002). A stability based method for discovering structure in clustered data. In Pacific Symposium on Biocomputing 2002, vol. 7, pp. 6–17, Lihue, Hawaii.

    Google Scholar 

  • Bhattacharjee, A., Richards, W. G., Staunton, J., Li, C., Monti, S., Vasa, P., Ladd, C., Beheshti, J., Bueno, R., Gillette, M., Loda, M., Weber, G., Mark, E. J., Lander, E. S., Wong, W., Johnson, B. E., Golub, T. R., Sugarbaker, D. J., & Meyerson, M. (2001). Classification of human lung carcinomas by mRNA expression profiling reveals distinct adenocarcinomas sub-classes. In Proceedings of the National Academy of Sciences, 98:24, 13790–13795.

    Google Scholar 

  • Bock, H. (1985). On some significance tests in cluster analysis. Journal of Classification, 2, 77–108.

    Google Scholar 

  • Cheeseman, P., & Stutz, J. (1996), Bayesian classification (AutoClass): Theory and results. In U. M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, & R. Uthurasamy (Eds.), Advances in Knowledge Discovery and Data Mining, pp. 153–180, MIT Press.

  • Chickering, D. M., & Heckerman, D. (1997). Efficient approximation for the marginal likelihood of Bayesian networks with hidden variables. Machine Learning, 29, 181–212.

    Google Scholar 

  • Cowell, F. A. (1995). Measuring Inequality. New York: Prentice Hall.

    Google Scholar 

  • Duda, R. O., & Hart, P. E. (1973). Pattern Classification and Scene Analysis. John Wiley & Sons.

  • Dudoit, S., & Fridlyand, J. (2002). A prediction-based resampling method for estimating the number of clusters in a dataset. Genome Biology, 3:7,1–21.

    Google Scholar 

  • Efron, B., & Tibshirani, R. J. (1994). An Introduction to the Bootstrap, No. 57 in Monographs on Statistics and Applied Probability. CRC Press.

  • Eisen, M. B., Spellman, P. T., Brown, P. O., & Botstein, D. (1998). Cluster analysis and display of genome-wide expression patterns. Proceedings of the National Academy of Sciences, 95, 14863–14868.

    Google Scholar 

  • Golub, T. R., Slonim, D. K., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J. P., Coller, H., Loh, M., Downing, J., Caligiuri, M., Bloomfield, C., & Lander, E. (1999). Molecular classification of cancer: Class discovery and class prediction by gene expression. Science, 286:5439, 531–537.

    Google Scholar 

  • Hartigan, J. A. (1978). Asymptotic distributions for clustering criteria. Annals of Statistics, 6:1, 117–131.

    Google Scholar 

  • Hastie, T., Tibshirani, R., & Friedman, J. (2001). The Elements of Statistical Learning, Statistics. New York: Springer.

    Google Scholar 

  • Hubert, L., & Arabie, P. (1985). Comparing partitions. Journal of Classification, 2, 193–218.

    Google Scholar 

  • Jain, A. K., & Dubes, R. C. (1988). Algorithms for Clustering Data. Englewood Cliffs, NJ: Prentice Hall.

    Google Scholar 

  • Jain, A. K., & Moreau, J. (1988). Bootstrap techniques in cluster analysis. Pattern Recognition, 20, 547–568.

    Google Scholar 

  • Kass, R. E. & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795.

    Google Scholar 

  • Kohonen, T. (1990). The self-organizing map. Proceedings of the IEEE, 78:9, 1464–1480.

    Google Scholar 

  • Kohonen, T. (1997). Self-Organizing Maps, Information Sciences. Springer.

  • Levine, E., & Domany, E. (2001). Resampling method for unsupervised estimation of cluster validity. Neural Computation, 13:11, 2573–2593.

    Google Scholar 

  • Milligan, G., & Cooper, M. (1985). An examination of procedures for determining the number of clusters in a data set. Psyochometrika, 50, 159–179.

    Google Scholar 

  • Milligan, G. & Cooper, M. (1986). Astudy of the comparability of external criteria for hierarchical cluster analysis. Multivariate Behavioral Research, 21, 441–458.

    Google Scholar 

  • Pomeroy, S., Tamayo, P., Gaasenbeek, M., Angelo, L. M. S. M., McLaughlin, M. E., Kim, J. Y., Goumnerova, L. C., Black, P. M., Lau, C., Allen, J. C., Zagzag, D., Olson, J. M., Curran, T., Wetmore, C., Biegel, J. A., Poggio, T., Mukherjee, S., Rifkin, A., Califano, G., Stolovitzky, D. N., Louis, J. P., Mesirov, E. S., Lander, R., & Golub, T. R. (2002). Gene expression-based classification and outcome prediction of central nervous system embryonal tumors. Nature, 415:6870, 436–442.

    Google Scholar 

  • Ramaswamy, S., Tamayo, P., Rifkin, R., Mukherjee, S., Yeang, C.-H., Angelo, M., Ladd, C., Reich, M., Latulippe, E., Mesirov, J. P., Poggio, T., Gerald, W., Loda, M., Lander, E. S., & Golub, T. R. (2001). Multi-class cancer diagnosis using tumor gene expression signatures. Proceedings of the National Academy of Sciences, 98:26, 15149–15154.

    Google Scholar 

  • Ramoni, M., Sebastiani, P., & Kohane, I. S. (2002). Cluster analysis of gene expression dynamics. In Proceedings of the National Academy of Sciences, 99:14, 9121–9126.

    Google Scholar 

  • Slonim, D. K., Tamayo, P., Mesirov, J. P., Golub, T. R., & Lander, E. S. (2000). Class prediction and discovery using gene expression data. In RECOMB 2000: The Fourth Annual International Conference on Research in Computational Molecular Biology (pp. 263–272), Tokyo, Japan.

  • Su, A. I., Cooke, M. P., Ching, K. A., Hakak, Y., Walker, J. R., Wiltshire, T., Orth, A. P., Vega, R. G., Sapinoso, L. M., Moqrich, A., Patapoutian, A., Hampton, G. M., Schultz, P. G., & Hogenesch, J. B. (2002). Large-scale analysis of the human and mouse transcriptomes. Proceedings of the National Academy of Sciences, 99:7, 4465–447.

    Google Scholar 

  • Tamayo, P., Slonim, D., Mesirov, J., Zhu, Q., Kitareewan, S., Dmitrovsky, E., Lander, E. S., & Godlub, T. R. (1999), Interpreting patterns of gene expression with self-organizing maps: Methods and application to hematopoietic differentiation. Proceedings of the National Academy of Sciences, 96, 2907–2912.

    Google Scholar 

  • Tibshirani, R., Walther, G., Botstein, D., & Brown, P. (2001a). Cluster validation by prediction strength. Unpub-lished manuscript (http://www-stat.stanford.edu/~tibs/ftp/predstr.pdf).

  • Tibshirani, R., Walther, G., & Hastie, T. (2001b). Estimating the number of clusters in a dataset via the gap statistic. Journal of the Royal Statistical Society B, 63:2, 411–423.

    Google Scholar 

  • Titterington, D., Smith, A., & Makov, U. (1985). Statistical Analysis of Finite Mixture Distributions. New York: Wiley.

    Google Scholar 

  • Todd Golub et. al. (2002). GeneCluster 2.0. http://www-genome.wi.mit.edu/cancer/software/ genecluster2/gc2.html.

  • West, M. (2002). Bayesian factor regression models in the Large p, Small n Paradigm. Bayesian Statistics, 7,to appear.

  • West, M., Blanchette, C., Dressman, H., Huang, E., Ishida, S., Spang, R., Zuzan, H., Olson Jr., J. A., Marks, J. R., & Nevins, J. R. (2001). Predicting the clinical status of human breast cancer by using gene expression profiles. Proceedings of the National Academy of Sciences, 98:20, 11462–11467.

    Google Scholar 

  • Yeoh, E.-J., Ross, M. E., Shurtleff, S. A., Williams, W. K., Patel, D., Mahfouz, R., Behm, F. G., Raimondi, S. C., Relling, M. V., Patel, A., Cheng, C., Campana, D., Wilkins, D., Zhou, X., Li, J., Liu, H., Pui, C.-H., Evans, W. E., Naeve, C., Wong, L., & Downing, J. R. (2002). Classification, subtype discovery, and prediction of outcome in pediatric acute lymphoblastic leukemia by gene expression profiling. Cancer Cell, 1:2.

  • Yeung, K. Y., Fraley, C., Murua, A., Raftery, A. E., & Ruzzo, W. L. (2001a). Model-based clustering and data transformations for gene expression data. Bioinformatics, 17:10, 977–987.

    Google Scholar 

  • Yeung, K. Y., Haynor, D. R., & Ruzzo, W. L. (2001b) Validating clustering for gene expression data. Bioinformatics, 17:4.

Download references

Author information

Authors and Affiliations

  1. Whitehead Institute/MIT Center for Genome Research, One Kendall Square, Cambridge, MA, 02139, USA

    Stefano Monti, Pablo Tamayo, Jill Mesirov & Todd Golub

Authors
  1. Stefano Monti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pablo Tamayo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jill Mesirov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Todd Golub
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Monti, S., Tamayo, P., Mesirov, J. et al. Consensus Clustering: A Resampling-Based Method for Class Discovery and Visualization of Gene Expression Microarray Data. Machine Learning 52, 91–118 (2003). https://doi.org/10.1023/A:1023949509487

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1023949509487

  • unsupervised learning
  • class discovery
  • model selection
  • gene expression microarrays