A Hierarchical Clustering algorithm based on Silhouette Index for cancer subtype discovery from genomic data

  • N. NidheeshEmail author
  • K. A. Abdul Nazeer
  • P. M. Ameer
Original Article


Identifying potential novel subtypes of cancers from genomic data requires techniques to estimate the number of natural clusters in the data. Determining the number of natural clusters in a dataset has been a challenging problem in Machine Learning. Employing an internal cluster validity index such as Silhouette Index together with a clustering algorithm has been a widely used technique for estimating the number of natural clusters, which has limitations. We propose a Hierarchical Agglomerative Clustering algorithm which automatically estimates the numbers of natural clusters and gives the associated clustering solutions along with dendrograms for visualizing the clustering structure. The algorithm has a Silhouette Index-based criterion for selecting the pair of clusters to merge, in the process of building the clustering hierarchy. The proposed algorithm could find decent estimates for the number of natural clusters, and the associated clustering solutions when applied to a collection of cancer gene expression datasets and general datasets. The proposed method showed better overall performance when compared to that of a set of prominent methods for estimating the number of natural clusters, which are used for cancer subtype discovery from genomic data. The proposed method is deterministic. It can be a better alternative to contemporary approaches for identifying potential novel subtypes of cancers from genomic data.


Cluster analysis Hierarchical Clustering Silhouette Index Cluster number estimation Cancer subtype discovery Gene expression data Consensus Clustering 


Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Supplementary material

521_2019_4636_MOESM1_ESM.pdf (463 kb)
Supplementary material 1 (pdf 462 KB)


  1. 1.
    Azzalini A, Menardi G (2014) Clustering via nonparametric density estimation: the R package pdfCluster. J Stat Softw 57(11):1–26CrossRefGoogle Scholar
  2. 2.
    Barrett T, Wilhite SE, Ledoux P, Evangelista C, Kim IF, Tomashevsky M, Marshall KA, Phillippy KH, Sherman PM, Holko M et al (2012) NCBI GEO: archive for functional genomics data sets-update. Nucleic Acids Res 41(D1):D991–D995. CrossRefGoogle Scholar
  3. 3.
    Bhattacharjee A, Richards WG, Staunton J, Li C, Monti S, Vasa P, Ladd C, Beheshti J, Bueno R, Gillette M, Loda M, Weber G, Mark EJ, Lander ES, Wong W, Johnson BE, Golub TR, Sugarbaker DJ, Meyerson M (2001) Classification of human lung carcinomas by mRNA expression profiling reveals distinct adenocarcinoma subclasses. Proc Nat Acad Sci 98(24):13790–13795. CrossRefGoogle Scholar
  4. 4.
    Cai M, Li L (2017) Subtype identification from heterogeneous TCGA datasets on a genomic scale by multi-view clustering with enhanced consensus. BMC Med Genomics 10(4):75. CrossRefGoogle Scholar
  5. 5.
    Cancer Genome Atlas Research Network et al (2012) Comprehensive molecular portraits of human breast tumors. Nature 490(7418):61. CrossRefGoogle Scholar
  6. 6.
    Cancer Genome Atlas Research et al (2013) Integrated genomic characterization of endometrial carcinoma. Nature 497(7447):67–73. CrossRefGoogle Scholar
  7. 7.
    Cavalli FM, Remke M, Rampasek L, Peacock J, Shih DJ, Luu B, Garzia L, Torchia J, Nor C, Morrissy AS et al (2017) Intertumoral heterogeneity within Medulloblastoma subgroups. Cancer Cell 31(6):737–754. CrossRefGoogle Scholar
  8. 8.
    Chalise P, Fridley BL (2017) Integrative clustering of multi-level omic data based on non-negative matrix factorization algorithm. PLoS ONE 12(5):1–18. CrossRefGoogle Scholar
  9. 9.
    Ciriello G, Gatza M, Beck A, Wilkerson M, Rhie S, Pastore A, Zhang H, McLellan M, Yau C, Kandoth C, Bowlby R, Shen H, Hayat S, Fieldhouse R, Lester S, Tse G, Factor R, Collins L, Allison K et al (2015) Comprehensive molecular portraits of invasive lobular breast cancer. Cell 163(2):506–519. CrossRefGoogle Scholar
  10. 10.
    de Souto MC, Costa IG, de Araujo DS, Ludermir TB, Schliep A (2008) Clustering cancer gene expression data: a comparative study. BMC Bioinform 9(1):497. CrossRefGoogle Scholar
  11. 11.
    Galili T (2015) dendextend: an R package for visualizing, adjusting, and comparing trees of hierarchical clustering. Bioinformatics. CrossRefGoogle Scholar
  12. 12.
    Golub TR, Slonim DK, Tamayo P, Huard C, Gaasenbeek M, Mesirov JP, Coller H, Loh ML, Downing JR, Caligiuri MA et al (1999) Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286(5439):531–537. CrossRefGoogle Scholar
  13. 13.
    Gowrishankar B, Przybycin CG, Ma C, Nandula SV, Rini B, Campbell S, Klein E, Chaganti R, Magi-Galluzzi C, Houldsworth J (2015) A genomic algorithm for the molecular classification of common renal cortical neoplasms: development and validation. J Urol 193(5):1479–1485. CrossRefGoogle Scholar
  14. 14.
    Griesinger AM, Josephson RJ, Donson AM, Levy JMM, Amani V, Birks DK, Hoffman LM, Furtek SL, Reigan P, Handler MH et al (2015) Interleukin-6/STAT3 pathway signaling drives an inflammatory phenotype in Group A ependymoma. Cancer Immunol Res. CrossRefGoogle Scholar
  15. 15.
    Guo Y, Zheng J, Shang X, Li Z (2018) A similarity regression fusion model for integrating multi-omics data to identify cancer subtypes. Genes. CrossRefGoogle Scholar
  16. 16.
    Hubert L, Arabie P (1985) Comparing partitions. J Classif 2(1):193–218. CrossRefzbMATHGoogle Scholar
  17. 17.
    Karatzoglou A, Smola A, Hornik K, Zeileis A (2004) kernlab—an S4 package for kernel methods in R. J Stat Softw 11(9):1–20CrossRefGoogle Scholar
  18. 18.
    Karlsson A, Brunnström H, Micke P, Veerla S, Mattsson J, La Fleur L, Botling J, Jönsson M, Reuterswärd C, Planck M et al (2017) Gene expression profiling of large cell lung cancer links transcriptional phenotypes to the new histological WHO 2015 classification. J Thorac Oncol. CrossRefGoogle Scholar
  19. 19.
    Kaufman L, Rousseeuw P (1987) Clustering by means of medoids. In: Dodge Y (ed) Statistical data analysis based on the L1-norm and related methods. Elsevier Science Pub. Co., Amsterdam, pp 405–416Google Scholar
  20. 20.
    Laiho P, Kokko A, Vanharanta S, Salovaara R, Sammalkorpi H, Järvinen H, Mecklin J, Karttunen T, Tuppurainen K, Davalos V, Arango D, Aaltonen LA (2007) Serrated carcinomas form a subclass of colorectal cancer with distinct molecular basis. Oncogene 26(2):312CrossRefGoogle Scholar
  21. 21.
    Li Z, Chen Y, Hu S, Zhang J, Wu J, Ren W, Shao N, Ying X (2016) Integrative analysis of protein-coding and non-coding RNAs identifies clinically relevant subtypes of clear cell renal cell carcinoma. Oncotarget 7(50):82671. CrossRefGoogle Scholar
  22. 22.
    Lichman M (2013) UCI machine learning repository. Accessed 08 Mar 2018
  23. 23.
    Lloyd S (1982) Least squares quantization in PCM. IEEE Trans Inf Theory 28(2):129–137. MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Lu J, Getz G, Miska EA, Alvarez-Saavedra E, Lamb J, Peck D, Sweet-Cordero A, Ebert BL, Mak RH, Ferrando AA, Downing JR, Jacks T, Horwitz HR, Golub TR (2005) MicroRNA expression profiles classify human cancers. Nature 435(7043):834CrossRefGoogle Scholar
  25. 25.
    Maechler M, Rousseeuw P, Struyf A, Hubert M, Hornik K (2017) Cluster: cluster analysis basics and extensions. R package version 2.0.6Google Scholar
  26. 26.
    Mehmood R, El-Ashram S, Bie R, Dawood H, Kos A (2017) Clustering by fast search and merge of local density peaks for gene expression microarray data. Sci Rep. CrossRefGoogle Scholar
  27. 27.
    Mertins P, Mani DR, Ruggles KV, Gillette MA, Clauser KR, Wang P, Wang X, Qiao JW, Cao S, Petralia F, Kawaler E, Mundt F, Krug K, Tu Z, Lei JT, Gatza ML, Wilkerson M, Perou CM, Yellapantula V, Kl Huang et al (2016) Proteogenomics connects somatic mutations to signalling in breast cancer. Nature 534:55. CrossRefGoogle Scholar
  28. 28.
    Monti S, Tamayo P, Mesirov J, Golub T (2003) Consensus clustering: a resampling-based method for class discovery and visualization of gene expression microarray data. Mach Learn 52(1):91–118. CrossRefzbMATHGoogle Scholar
  29. 29.
    Mouselimis L (2018) ClusterR: Gaussian mixture models, K-means, mini-batch-KMeans and K-Medoids clustering. R package version 1.1.1
  30. 30.
    Ng AY, Jordan MI, Weiss Y (2002) On spectral clustering: analysis and an algorithm. In: Advances in neural information processing systems, pp 849–856Google Scholar
  31. 31.
    Nidheesh N, Abdul Nazeer KA, Ameer PM (2017) An enhanced deterministic K-means clustering algorithm for cancer subtype prediction from gene expression data. Comput Biol Med 91:213–221. CrossRefGoogle Scholar
  32. 32.
    R Core Team (2017) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna.
  33. 33.
    Reddy CK, Vinzamuri B (2013) A survey of partitional and hierarchical clustering algorithms. In: Aggarwal CC, Reddy CK (eds) Data clustering: algorithms and applications, Chap 4. Chapman & Hall/CRC, Boca Raton, pp 87–110Google Scholar
  34. 34.
    Rousseeuw PJ (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20:53–65CrossRefGoogle Scholar
  35. 35.
    Senbabaoğlu Y, Michailidis G, Li JZ (2014) Critical limitations of consensus clustering in class discovery. Sci Rep 4:6207. CrossRefGoogle Scholar
  36. 36.
    Shen R, Olshen AB, Ladanyi M (2009) Integrative clustering of multiple genomic data types using a joint latent variable model with application to breast and lung cancer subtype analysis. Bioinformatics 25(22):2906–2912. CrossRefGoogle Scholar
  37. 37.
    Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905. CrossRefGoogle Scholar
  38. 38.
    Shi Q, Zhang C, Peng M, Yu X, Zeng T, Liu J, Chen L (2017) Pattern fusion analysis by adaptive alignment of multiple heterogeneous omics data. Bioinformatics 33(17):2706–2714. CrossRefGoogle Scholar
  39. 39.
    Tan PN, Steinbach M, Kumar V (2005) Introduction to data mining, 1st edn. Addison-Wesley Longman Publishing Co., Inc., BostonGoogle Scholar
  40. 40.
    Verhaak RG, Hoadley KA, Purdom E, Wang V, Qi Y, Wilkerson MD, Miller CR, Ding L, Golub T, Mesirov JP, Alexe G, Lawrence M, O’Kelly M, Tamayo P, Weir BA, Gabriel S, Winckler W, Gupta S, Jakkula L, Feiler HS et al (2010) Integrated genomic analysis identifies clinically relevant subtypes of glioblastoma characterized by abnormalities in PDGFRA, IDH1, EGFR, and NF1. Cancer Cell 17(1):98–110. CrossRefGoogle Scholar
  41. 41.
    Wang B, Mezlini AM, Demir F, Fiume M, Tu Z, Brudno M, Haibe-Kains B, Goldenberg A (2014) Similarity network fusion for aggregating data types on a genomic scale. Nat Methods 11:333. CrossRefGoogle Scholar
  42. 42.
    Ward JH Jr (1963) Hierarchical grouping to optimize an objective function. J Am Stat Assoc 58(301):236–244MathSciNetCrossRefGoogle Scholar
  43. 43.
    Wilkerson MD, Hayes DN (2010) ConsensusClusterPlus: a class discovery tool with confidence assessments and item tracking. Bioinformatics 26(12):1572–1573CrossRefGoogle Scholar
  44. 44.
    Wiwie C, Baumbach J, Röttger R (2015) Comparing the performance of biomedical clustering methods. Nat Methods 12(11):1033. CrossRefGoogle Scholar
  45. 45.
    Wu D, Wang D, Zhang MQ, Gu J (2015) Fast dimension reduction and integrative clustering of multi-omics data using low-rank approximation: application to cancer molecular classification. BMC Genom 16(1):1022. CrossRefGoogle Scholar
  46. 46.
    Yu Z, Wongb HS, You J, Yang Q, Liao H (2011) Knowledge based cluster ensemble for cancer discovery from biomolecular data. IEEE Trans Nanobiosci 10(2):76–85CrossRefGoogle Scholar
  47. 47.
    Yu Z, Li L, You J, Wong HS, Han G (2012) Sc$^3$: triple spectral clustering-based consensus clustering framework for class discovery from cancer gene expression profiles. IEEE/ACM Trans Comput Biol Bioinform 9(6):1751–1765CrossRefGoogle Scholar
  48. 48.
    Yu Z, You J, Li L, Wong HS, Han G (2012) Representative distance: a new similarity measure for class discovery from gene expression data. IEEE Trans Nanobiosci 11(4):341–351CrossRefGoogle Scholar
  49. 49.
    Zheng S, Cherniack A, Dewal N, Moffitt R, Danilova L, Murray B, Lerario A, Else T, Knijnenburg T, Ciriello G, Kim S, Assie G, Morozova O, Akbani R et al (2016) Comprehensive pan-genomic characterization of adrenocortical carcinoma. Cancer Cell 29(5):723–736CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Electronics and Communication EngineeringNational Institute of Technology CalicutCalicutIndia
  2. 2.Department of Computer Science and EngineeringNational Institute of Technology CalicutCalicutIndia

Personalised recommendations