Proceedings of ELM-2014 Volume 1 pp 435-444 | Cite as
Extreme Learning Machine for Clustering
Conference paper
Abstract
Extreme Learning Machine (ELM) is originally introduced for regression and classification. This paper extends ELM for clustering using Extreme Learning Machine Auto Encoder (ELM-AE) which learn key features of the input data. The embedding created by multiplying the input data with the output weights of ELM-AE is shown to produce better clustering results than clustering the original data space. Furthermore, ELM-AE is used to find the starting cluster points for k-means clustering, which produces better results than randomly assigning the cluster start points. The experimental results show that the proposed clustering algorithm Extreme Learning Machine Auto Encoder Clustering (ELM-AEC) is better than k-means clustering and is competitive with Unsupervised Extreme Learning Machine (USELM).
Keywords
Extreme Learning Machine Auto Encoders Clustering K-meansPreview
Unable to display preview. Download preview PDF.
References
- 1.Huang, G.-B., Chen, L., Siew, C.-K.: Universal Approximation Using Incremental Constructive Feedforward Networks with Random Hidden Nodes. IEEE Transactions on Neural Networks 17(4), 879–892 (2006)CrossRefGoogle Scholar
- 2.Huang, G.-B., Chen, L.: Convex Incremental Extreme Learning Machine. Neurocomputing 70, 3056–3062 (2007)CrossRefGoogle Scholar
- 3.Zhang, R., Lan, Y., Huang, G.-B., Xu, Z.-B.: Universal Approximation of Extreme Learning Machine With Adaptive Growth of Hidden Nodes. IEEE Transactions on Neural Networks and Learning Systems 23(2), 365–371 (2012)CrossRefGoogle Scholar
- 4.Huang, G.-B., Chen, L.: Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16-18), 3460–3468 (2008)CrossRefGoogle Scholar
- 5.Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme Learning Machine: Theory and Applications. Neurocomputing 70, 489–501 (2006)CrossRefGoogle Scholar
- 6.Huang, G.-B., Zhou, H., Ding, X., Zhang, R.: Extreme Learning Machine for Regression and Multiclass Classification. IEEE Transactions on Systems, Man, and Cybernetics, Part B 42(2), 513–529 (2012)CrossRefGoogle Scholar
- 7.Huang, G., Song, S., Gupta, J.N.D., Wu, C.: Semi-Supervised and Unsupervised Extreme Learning Machines. IEEE Transactions on Cybernetics 99, 1 (2014)CrossRefGoogle Scholar
- 8.Huang, G.-B.: An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels. Cognitive Computation, 1–15 (2014)Google Scholar
- 9.Kasun, L.L.C., Zhou, H., Huang, G.-B., Vong, C.M.: Representational Learning with Extreme Learning Machines for Big Data. IEEE Intelligent Systems 28(6), 31–34 (2013)Google Scholar
- 10.Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323(6088), 533–536 (1986)CrossRefGoogle Scholar
- 11.Bartlett, P.L.: The Sample Complexity of Pattern Classification with Neural Networks: The Size of the Weights is More Important than the Size of the Network. IEEE Transactions on Information Theory 44(2), 525–536 (1998)CrossRefMATHMathSciNetGoogle Scholar
- 12.Serre, D.: Matrices: Theory and Applications. Springer-Verlag New York, Inc. (2002)Google Scholar
- 13.Rao, C.R., Mitra, S.K.: Generalized Inverse of Matrices and its Applications. John Wiley, New York (1971)MATHGoogle Scholar
- 14.Johnson, W., Lindenstrauss, J.: Extensions of Lipschitz mappings into a Hilbert space. In: Conference in Modern Analysis and Probability, vol. 26, pp. 189–206. American Mathematical Society (1984)Google Scholar
- 15.Hartigan, J.A., Wong, M.A.: Algorithm AS 136: A K-means clustering algorithm. Applied Statistics, 100–108 (1979)Google Scholar
- 16.Lloyd, S.: Least Squares Quantization in PCM. IEEE Trans. Inf. Theor. 28(2), 129–137 (2006)CrossRefMathSciNetGoogle Scholar
- 17.MacQueen, J.B.: Some Methods for Classification and Analysis of MultiVariate Observations. In: Cam, L.M.L., Neyman, J. (eds.) Proc. of the fifth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, pp. 281–297. University of California Press (1967)Google Scholar
- 18.Eckart, C., Young, G.: The approximation of one matrix by another of lower rank. Psychometrika 1(3), 211–218 (1936)CrossRefMATHGoogle Scholar
- 19.Ding, C., He, X.: K-means Clustering via Principal Component Analysis. In: Proceedings of the Twenty-First International Conference on Machine Learning, ICML 2004, p. 29. ACM (2004)Google Scholar
- 20.Bradley, P.S., Fayyad, U.M.: Refining initial points for k-means clustering. In: Proceedings of the Fifteenth International Conference on Machine Learning, ICML 1998, pp. 91–99. Morgan Kaufmann Publishers Inc., San Francisco (1998)Google Scholar
- 21.Grim, J., Novovicova, J., Pudil, P., Somol, P., Ferri, F.J.: Initializing normal mixtures of densities. In: Proceedings of the Fourteenth International Conference on Pattern Recognition, vol. 1, pp. 886–890 (1998)Google Scholar
- 22.Moore, A.W.: Very Fast EM-based Mixture Model Clustering Using Multiresolution Kd-trees. In: Proceedings of the 1998 Conference on Advances in Neural Information Processing Systems II, pp. 543–549. MIT Press, Cambridge (1999)Google Scholar
- 23.Bache, K., Lichman, M.: UCI machine learning repository (2013)Google Scholar
- 24.Samaria, F.S., Harter, A.C.: Parameterisation of a stochastic model for human face identification. In: Proceedings of the Second IEEE Workshop on Applications of Computer Vision, pp. 138–142 (December 1994)Google Scholar
Copyright information
© Springer International Publishing Switzerland 2015