ELM-NET, a closer to practice approach for classifying the big data using multiple independent ELMs

  • Amin Shokrzade
  • Fardin Akhlaghian TabEmail author
  • Mohsen Ramezani


In this paper, a new ELM based classification method is presented to deal with the large volume of data in an efficient way. By inspiration from both parallel and sequential ELMs, this method consists of some independent ELMs which are trained using data batches in parallel. The main goal of this method is preventing exponential training time by running some ELMs in parallel which similar to the sequential methods, are trained using different data chunks. Moreover, a new aggregation method is used here to outperform this structure which can relatively achieve stable results for the different number of the ELMs. The stable results can persuade us to use such classification method on regular platforms to decrease the cost of the big data analyzing. Our method is tested on different platforms to indicate that it can be used for reducing the costs of big data analyzing. Experimental results on MNIST, KDDCup99, KDDCup99_2, Susy, and Higgs datasets shows the better performance of our method than the state-of-the-art methods.


Big data ELM Hidden layer neurons Label space Correctness vector Summarization 



  1. 1.
    Suykens, J.A., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)CrossRefGoogle Scholar
  2. 2.
    Scholkopf, B., Smola, A.J.: Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press, Cambridge (2001)Google Scholar
  3. 3.
    Choi, J.K., Jeon, K.H., Won, Y., Kim, J.J.: Application of big data analysis with decision tree for the foot disorder. Clust. Comput. 18(4), 1399–1404 (2015)CrossRefGoogle Scholar
  4. 4.
    Adamo, J.M.: Fuzzy decision trees. Fuzzy Sets Syst. 4(3), 207–219 (1980)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Quinlan, J.R.: Decision trees and decision-making. IEEE Trans. Sys. Man Cybern. 20(2), 339–346 (1990)CrossRefGoogle Scholar
  6. 6.
    Quinlan, J.R.: Induction of decision trees. Mach. Learn. 1(1), 81–106 (1986)Google Scholar
  7. 7.
    Huang, G.B., Wang, D.H., Lan, Y.: Extreme learning machines: a survey. Int. J. Mach. Learn. Cybern. 2(2), 107–122 (2011)CrossRefGoogle Scholar
  8. 8.
    Huang, G.B., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. Part B 42(2), 513–529 (2012)CrossRefGoogle Scholar
  9. 9.
    Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: a new learning scheme of feedforward neural networks. Neural Netw. 2, 985–990 (2004)Google Scholar
  10. 10.
    Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(1), 489–501 (2006)CrossRefGoogle Scholar
  11. 11.
    Kirk, D.B., Wen-Mei, W.H.: Programming massively parallel processors: a hands-on approach. Elsevier, New York (2016)Google Scholar
  12. 12.
    Leighton, F.T.: Introduction to parallel algorithms and architectures: Arrays trees hypercubes. Elsevier, New York (2014)zbMATHGoogle Scholar
  13. 13.
    Chen, C., Li, K., Ouyang, A., Tang, Z., Li, K.: Gpu-accelerated parallel hierarchical extreme learning machine on flink for big data. IEEE Trans. Syst. Man Cybern. 47(10), 2740–2753 (2017)CrossRefGoogle Scholar
  14. 14.
    He, Q., Shang, T., Zhuang, F., Shi, Z.: Parallel extreme learning machine for regression based on MapReduce. Neurocomputing 102, 52–58 (2013)CrossRefGoogle Scholar
  15. 15.
    He, Y., Geng, Z., Zhu, Q.: Positive and negative correlation input attributes oriented subnets based double parallel extreme learning machine (PNIAOS-DPELM) and its application to monitoring chemical processes in steady state. Neurocomputing 165, 171–181 (2015)CrossRefGoogle Scholar
  16. 16.
    Wang, B., Huang, S., Qiu, J., Liu, Y., Wang, G.: Parallel online sequential extreme learning machine based on MapReduce. Neurocomputing 149, 224–232 (2015)CrossRefGoogle Scholar
  17. 17.
    Wang, Y., Dou, Y., Liu, X., Lei, Y.: PR-ELM: parallel regularized extreme learning machine based on cluster. Neurocomputing 173, 1073–1081 (2016)CrossRefGoogle Scholar
  18. 18.
    Roul, R.K., Nanda, A., Patel, V., Sahay, S.K.: Extreme learning machines in the field of text classification. In: 16th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, pp. 1–7 (2015)Google Scholar
  19. 19.
    Zheng, W., Qian, Y., Lu, H.: Text categorization based on regularization extreme learning machine. Neural Comput. Appl. 22(3), 447–456 (2013)CrossRefGoogle Scholar
  20. 20.
    He, B., Xu, D., Nian, R., van Heeswijk, M., Yu, Q., Miche, Y., Lendasse, A.: Fast face recognition via sparse coding and extreme learning machine. Cognit. Comput. 6(2), 264–277 (2014)Google Scholar
  21. 21.
    Marques, I., Graña, M.: Face recognition with lattice independent component analysis and extreme learning machines. Soft. Comput. 16(9), 1525–1537 (2012)CrossRefGoogle Scholar
  22. 22.
    Cecotti, H., Boumedine, C., Callaghan, M.: Hand-drawn symbol recognition in immersive virtual reality using deep extreme learning machines. In: International Conference on Recent Trends in Image Processing and Pattern Recognition, pp. 80–92 (2016)Google Scholar
  23. 23.
    Krawczyk, B.: GPU-accelerated extreme learning machines for imbalanced data streams with concept drift. Proc. Comput. Sci 80, 1692–1701 (2016)CrossRefGoogle Scholar
  24. 24.
    Xu, S., Wang, J.: Dynamic extreme learning machine for data stream classification. Neurocomputing 238, 433–449 (2017)CrossRefGoogle Scholar
  25. 25.
    Lu, S., Wang, X., Zhang, G., Zhou, X.: Effective algorithms of the Moore–Penrose inverse matrices for extreme learning machine. Intell. Data Anal. 19(4), 743–760 (2015)CrossRefGoogle Scholar
  26. 26.
    Pei, H., Wang, K., Lin, Q., Zhong, P.: Robust semi-supervised extreme learning machine. Knowl. Based Syst. 159, 203–220 (2018)CrossRefGoogle Scholar
  27. 27.
    Scardapane, S., Comminiello, D., Scarpiniti, M., Uncini, A.: Online sequential extreme learning machine with kernels. IEEE Trans. Neural Netw. Learn. Syst. 26(9), 2214–2220 (2015)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Vong, C.M., Tai, K.I., Pun, C.M., Wong, P.K.: Fast and accurate face detection by sparse Bayesian extreme learning machine. Neural Comput. Appl. 26(5), 1149–1156 (2015)CrossRefGoogle Scholar
  29. 29.
    Zhai, J.H., Xu, H.Y., Wang, X.Z.: Dynamic ensemble extreme learning machine based on sample entropy. Soft. Comput. 16(9), 1493–1502 (2012)CrossRefGoogle Scholar
  30. 30.
    Liao, S., Feng, C.: Meta-ELM: ELM with ELM hidden nodes. Neurocomputing 128, 81–87 (2014)CrossRefGoogle Scholar
  31. 31.
    Huang, G., Huang, G.B., Song, S., You, K.: Trends in extreme learning machines: a review. Neural Netw. 61, 32–48 (2015)zbMATHCrossRefGoogle Scholar
  32. 32.
    Duan, M., Li, K., Liao, X., Li, K.: A parallel multiclassification algorithm for big data using an extreme learning machine. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2337–2351 (2018)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Xin, J., Wang, Z., Qu, L., Yu, G., Kang, Y.: A-ELM: adaptive distributed extreme learning machine with MapReduce. Neurocomputing 174, 368–374 (2016)CrossRefGoogle Scholar
  34. 34.
    Lee, K.H., Lee, Y.J., Choi, H., Chung, Y.D., Moon, B.: Parallel data processing with MapReduce: a survey. AcM sIGMoD Record 40(4), 11–20 (2012)CrossRefGoogle Scholar
  35. 35.
    Liu, T., Fang, Z., Zhao, C., Zhou, Y.: Parallelization of a series of extreme learning machine algorithms based on spark. In: 15th International Conference on Computer and Information Science, pp. 1–5 (2016)Google Scholar
  36. 36.
    Han, M., Liu, B.: Ensemble of extreme learning machine for remote sensing image classification. Neurocomputing 149, 65–70 (2015)CrossRefGoogle Scholar
  37. 37.
    Jin, Y., Cao, J., Wang, Y., Zhi, R.: Ensemble based extreme learning machine for cross-modality face matching. Multimed. Tools Appl. 75(19), 11831–11846 (2016)CrossRefGoogle Scholar
  38. 38.
    Huang, S., Wang, B., Qiu, J., Yao, J., Wang, G., Yu, G.: Parallel ensemble of online sequential extreme learning machine based on MapReduce. Neurocomputing 174, 352–367 (2016)CrossRefGoogle Scholar
  39. 39.
    Tang, X., Chen, L.: Artificial bee colony optimization-based weighted extreme learning machine for imbalanced data learning. Clust. Comput., 1–16 (2018)Google Scholar
  40. 40.
    Luo, J., Vong, C.M., Wong, P.K.: Sparse Bayesian extreme learning machine for multi-classification. IEEE Trans. Neural Netw. Learn. Syst. 25(4), 836–843 (2014)CrossRefGoogle Scholar
  41. 41.
    Lu, H.J., An, C.L., Zheng, E.H., Lu, Y.: Dissimilarity based ensemble of extreme learning machine for gene expression data classification. Neurocomputing 128, 22–30 (2014)CrossRefGoogle Scholar
  42. 42.
    He, Q., Zhuang, F., Li, J., Shi, Z.: Parallel implementation of classification algorithms based on MapReduce. International Conference on Rough Sets and Knowledge Technology, pp. 655–662. Springer, Berlin (2010)Google Scholar
  43. 43.
    Serre, D.: Matrices. Graduate Texts in Mathematics, vol. 216. Springer, Berlin (2002)Google Scholar
  44. 44.
    Rao, C.R.: Generalized inverse of matrices and its applications. Elsevier, New York (1971)zbMATHGoogle Scholar
  45. 45.
    Huang, G.B., Bai, Z., Kasun, L.L.C., Vong, C.M.: Local receptive fields based extreme learning machine. IEEE Comput. Intell. Mag. 10(2), 18–29 (2015)CrossRefGoogle Scholar
  46. 46.
    Zhang, C., Li, F., Jestes, J.: Efficient parallel kNN joins for large data in MapReduce. In: Proceedings of the 15th International Conference on Extending Database Technology, pp. 38–49 (2012)Google Scholar
  47. 47.
    You, Z.H., Yu, J.Z., Zhu, L., Li, S., Wen, Z.K.A.: MapReduce based parallel SVM for large-scale predicting protein–protein interactions. Neurocomputing 145, 37–43 (2014)CrossRefGoogle Scholar
  48. 48.
    Panda, B., Herbach, J.S., Basu, S., Bayardo, R.J.: Planet: massively parallel learning of tree ensembles with mapreduce. Proc. VLDB Endow. 2(2), 1426–1437 (2009)CrossRefGoogle Scholar
  49. 49.
    Xin, J., Wang, Z., Chen, C., Ding, L., Wang, G., Zhao, Y.: ELM∗: distributed extreme learning machine with MapReduce. World Wide Web 17(5), 1189–1204 (2014)CrossRefGoogle Scholar
  50. 50.
    Xin, J., Wang, Z., Qu, L., Wang, G.: Elastic extreme learning machine for big data classification. Neurocomputing 149, 464–471 (2015)CrossRefGoogle Scholar
  51. 51.
    Inaba, F.K., Salles, E.O.T., Perron, S., Caporossi, G.: DGR-ELM-distributed generalized regularized ELM for classification. Neurocomputing 275, 1522–1530 (2018)CrossRefGoogle Scholar
  52. 52.
    Chen, J., Chen, H., Wan, X., Zheng, G.: MR-ELM: a MapReduce-based framework for large-scale ELM training in big data era. Neural Comput. Appl. 27(1), 101–110 (2016)CrossRefGoogle Scholar
  53. 53.
    Chen, C., Li, K., Ouyang, A., Li, K.: FlinkCL: an OpenCL-based in-memory computing architecture on heterogeneous CPU-GPU clusters for big data. IEEE Trans. Comput. 67(2), 1765–1779 (2018)MathSciNetzbMATHCrossRefGoogle Scholar
  54. 54.
    Van Heeswijk, M., Miche, Y., Oja, E., Lendasse, A.: GPU-accelerated and parallelized ELM ensembles for large-scale regression. Neurocomputing 74(16), 2430–2437 (2011)CrossRefGoogle Scholar
  55. 55.
    Liang, N.Y., Huang, G.B., Saratchandran, P., Sundararajan, N.: A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans. Neural Networks 17(6), 1411–1423 (2006)CrossRefGoogle Scholar
  56. 56.
    Huang, S., Wang, B., Chen, Y., Wang, G., Yu, G.: Efficient batch parallel online sequential extreme learning machine algorithm based on MapReduce. In: Proceedings of ELM 2015, pp. 13–25 (2016)Google Scholar
  57. 57.
    Huang, S., Wang, B., Chen, Y., Wang, G., Yu, G.: An efficient parallel method for batched OS-ELM training using MapReduce. Memet. Comput. 9(3), 183–197 (2017)CrossRefGoogle Scholar
  58. 58.
    Segatori, A., Bechini, A., Ducange, P., Marcelloni, F.: A distributed fuzzy associative classifier for big data. IEEE Trans. Cybern. 48(9), 2656–2669 (2018)CrossRefGoogle Scholar
  59. 59.
    Bechini, A., Marcelloni, F., Segatori, A.: A MapReduce solution for associative classification of big data. Inf. Sci. 332, 33–55 (2016)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Amin Shokrzade
    • 1
  • Fardin Akhlaghian Tab
    • 1
    Email author
  • Mohsen Ramezani
    • 2
  1. 1.Department of Computer Engineering, Faculty of EngineeringUniversity of KurdistanSanandajIran
  2. 2.Department of Computer ScienceUniversity of KurdistanSanandajIran

Personalised recommendations