Skip to main content
SpringerLink
Log in

Unsupervised Learning Algorithms

  • Chapter
  • First Online:
Minimum Divergence Methods in Statistical Machine Learning

  • 840 Accesses

Abstract

In data analysis or data mining, there are mainly two fundamental types of methodologies, called unsupervised and supervised learning algorithms. This chapter explores principal component analysis, independent component analysis, density estimation and clustering analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Amari, S. (1998). Natural gradient works efficiently in learning. Neural Computation, 10, 251–276.

    Article  Google Scholar 

  • Amigó, E., Gonzalo, J., Artiles, J., & Verdejo, F. (2009). A comparison of extrinsic clustering evaluation metrics based on formal constraints. Information Retrieval, 12, 461–486.

    Article  Google Scholar 

  • Bezdek, J. C., Ehrlich, R., & Full, W. (1984). FCM: The fuzzy c-means clustering algorithm. Computers & Geoscience, 10, 191–2003.

    Article  Google Scholar 

  • Constantinopoulos, C., Titsias, M., & Likas, A. (2006). Bayesian feature and model selection for Gaussian mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 1013–1018.

    Article  Google Scholar 

  • Cox, D. R. (1957). Note on grouping. Journal of the American Statistical Association, 52, 543–547.

    Article  MATH  Google Scholar 

  • Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B, 39, 1–38.

    MathSciNet  MATH  Google Scholar 

  • Dunn, J. C. (1973). A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters. Journal of Cybernetics, 3, 32–57.

    Article  MathSciNet  MATH  Google Scholar 

  • Eguchi, S., Notsu, A., & Komori, O. (2017). Spontaneous learning for data distributions via minimum divergence. In F. Nielsen, F. Critchley, & C. Dodson (Eds.), Computational information geometry (pp. 79–99). Cham: Springer.

    Chapter  Google Scholar 

  • Fränti, P., & Sieranoja, S. (2018). K-means properties on six clustering benchmark datasets. Applied Intelligence, 48, 4743–4759.

    Article  MATH  Google Scholar 

  • Fränti, P., Rezaei, M., & Zhao, Q. (2014). Centroid index: Cluster level similarity measure. Pattern Recognition, 47, 3034–3045.

    Article  Google Scholar 

  • Ghosh, S., & Dubey, S. (2013). Comparative analysis of k-means and fuzzy c-means algorithms. International Journal of Advanced Computer Science and Applications, 4, 35–39.

    Google Scholar 

  • Hammersley, J. M., & Morton, K. W. (1954). Poor man’s Monte Carlo. Journal of the Royal Statistical Society. Series B, 16, 23–38.

    MathSciNet  MATH  Google Scholar 

  • Hathaway, R. J., & Bezdek, J. C. (1995). Optimization of clustering criteria by reformulation. IEEE Transactions on Fuzzy Systems, 3, 241–245.

    Article  Google Scholar 

  • Henderson, D., Jacobson, S. H., & Johnson, A. (2003). The theory and practice of simulated annealing. In F. Glover & G. A. Kochenberger (Eds.), Handbook of metaheuristics. Boston: Springer.

    Google Scholar 

  • Higuchi, I., & Eguchi, S. (1998). The influence function of principal component analysis by self-organizing rule. Neural Computation, 10, 1435–1444.

    Article  Google Scholar 

  • Higuchi, I., & Eguchi, S. (2004). Robust principal component analysis with adaptive selection for tuning parameters. Journal of Machine Learning Research, 5, 453–472.

    MathSciNet  MATH  Google Scholar 

  • Hosking, J. R., & Wallis, J. R. (1987). Parameter and quantile estimation for the generalized Pareto distribution. Technometrics, 29, 339–349.

    Article  MathSciNet  MATH  Google Scholar 

  • Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24, 417–441.

    Article  MATH  Google Scholar 

  • Huang, S., Yeh, Y., & Eguchi, S. (2009). Robust Kernel principal component analysis. Neural Computation, 21, 3179–3213.

    Article  MathSciNet  MATH  Google Scholar 

  • Hyvarinen, A. (1999). Gaussian moments for noisy independent component analysis. IEEE Signal Processing Letters, 6, 145–147.

    Article  Google Scholar 

  • Jain, A. K. (2010). Data clustering: 50 years beyond K-means. Pattern Recognition Letters, 31, 651–666.

    Article  Google Scholar 

  • Jolliffe, I. T. (2002). Principal component analysis. New York: Springer.

    MATH  Google Scholar 

  • Jolliffe, I. T. (2021). A 50-year personal journey through time with principal component analysis. Journal of Multivariate Analysis, 104820.

    Google Scholar 

  • Jolliffe, I. T., & Cadima, J. (2016). Principal component analysis: A review and recent developments. Philosophical Transactions of the Royal Society A, 374, 89–90.

    MathSciNet  MATH  Google Scholar 

  • Kamiya, H., & Eguchi, S. (2001). A class of robust principal component vectors. Journal of Multivariate Analysis, 77, 239–269.

    Article  MathSciNet  MATH  Google Scholar 

  • Komori, O., & Eguchi, S. (2021). A unified formulation of k-means, fuzzy c-means and Gaussian mixture model by the Kolmogorov-Nagumo average. Entropy, 23, 518.

    Article  MathSciNet  Google Scholar 

  • Lloyd, S. P. (1982). Least squares quantization in PCM. IEEE Transactions on Information Theory, 129–137.

    Google Scholar 

  • MacQueen, J. (1967). Some methods of classification and analysis of multivariate observations. In L. M. L. Cam & J. Neyman (Eds.), Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability (pp. 281–297). Berkeley: University of California Press.

    Google Scholar 

  • McNicholas, P. D., & Murphy, T. B. (2008). Parsimonious Gaussian mixture models. Statistics and Computing, 18, 285–296.

    Article  MathSciNet  Google Scholar 

  • Minami, M., & Eguchi, S. (2002). Robust blind source separation by beta divergence. Neural Computation, 14, 1859–1886.

    Article  MATH  Google Scholar 

  • Mollah, M. N. H., Minami, M., & Eguchi, S. (2006). Exploring latent structure of mixture ICA models by the minimum beta-divergence method. Neural Computation, 18, 166–190.

    Article  MATH  Google Scholar 

  • Mollah, M. N. H., Sultana, N., Minami, M., & Eguchi, S. (2010). Robust extraction of local structures by the minimum beta-divergence method. Neural Networks, 23, 226–238.

    Article  MATH  Google Scholar 

  • Notsu, A., & Eguchi, S. (2016). Robust clustering method in the presence of scattered observations. Neural Computation, 28, 1141–1162.

    Article  MathSciNet  MATH  Google Scholar 

  • Notsu, A., Komori, O., & Eguchi, S. (2014). Spontaneous clustering via minimum gamma-divergence. Neural Computation, 26, 421–448.

    Article  MathSciNet  MATH  Google Scholar 

  • Pearson, K. (1901). LIII. On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science,2, 559–572. https://doi.org/10.1080/14786440109462720.

  • Pickands, J. (1975). Statistical inference using extreme order statistics. The Annals of Statistics, 3, 119–131.

    MathSciNet  MATH  Google Scholar 

  • Rose, K., Gurewitz, E., & Fox, G. C. (1990). Statistical mechanics and phase transitions in clustering. Physical Review Letters, 65, 945–948.

    Article  Google Scholar 

  • Sofaer, H. R., Hoeting, J. A., & Jarnevich, C. S. (2019). The area under the precision-recall curve as a performance metric for rare binary events. Methods in Ecology and Evolution, 10, 565–577.

    Article  Google Scholar 

  • Steinhaus, H. (1957). Sur la division des corps matériels en parties. Bulletin L’Académie Polonaise des Science, 4, 801–804.

    MATH  Google Scholar 

  • Tibshirani, R., Walther, G., & Hastie, T. (2001). Estimating the number of clusters in a data set via the gap statistic. Journal of Royal Statistic Society Series B, 63, 411–423.

    Article  MathSciNet  MATH  Google Scholar 

  • Van Rijsbergen, C. (1974). Foundation of evaluation. Journal of Documentation, 30, 365–373.

    Article  Google Scholar 

  • Xu, R., & Wunsch, D. (2005). Survey of clustering algorithms. IEEE Transactions on Neural Networks, 16, 236–243.

    Article  Google Scholar 

  • Yu, J. (2005). General C-means clustering model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 1197–1211.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Statistical Mathematic, Tokyo, Japan

    Shinto Eguchi

  2. Seikei University, Tokyo, Japan

    Osamu Komori

Authors
  1. Shinto Eguchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Osamu Komori
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shinto Eguchi .

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Japan KK, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Eguchi, S., Komori, O. (2022). Unsupervised Learning Algorithms. In: Minimum Divergence Methods in Statistical Machine Learning. Springer, Tokyo. https://doi.org/10.1007/978-4-431-56922-0_5

Download citation

Publish with us

Policies and ethics

Search

Navigation