CloudSVM: Training an SVM Classifier in Cloud Computing Systems

  • F. Ozgur Catak
  • M. Erdal Balaban
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7719)

Abstract

In conventional distributed machine learning methods, distributed support vector machines (SVM) algorithms are trained over pre-configured intranet/internet environments to find out an optimal classifier. These methods are very complicated and costly for large datasets. Hence, we propose a method that is referred as the Cloud SVM training mechanism (CloudSVM) in a cloud computing environment with MapReduce technique for distributed machine learning applications. Accordingly, (i) SVM algorithm is trained in distributed cloud storage servers that work concurrently; (ii) merge all support vectors in every trained cloud node; and (iii) iterate these two steps until the SVM converges to the optimal classifier function. Single computer is incapable to train SVM algorithm with large scale data sets. The results of this study are important for training of large scale data sets for machine learning applications. We provided that iterative training of splitted data set in cloud computing environment using SVM will converge to a global optimal classifier in finite iteration size.

Keywords

Support Vector Machines Distributed Computing Cloud Computing MapReduce 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Chang, E.Y., Zhu, K., Wang, H., Bai, H., Li, J., Qiu, Z., Cui, H.: PSVM: Parallelizing Support Vector Machines on Distributed Computers. In: Advances in Neural Information Processing Systems, vol. 20 (2007)Google Scholar
  2. 2.
    Tsang, I.W., Kwok, J.T., Cheung, P.M.: Core Vector Machines: Fast SVM Training on Very Large Data Sets. J. Mach. Learn. Res. 6, 363–392 (2005)MathSciNetMATHGoogle Scholar
  3. 3.
    Weston, J., Mukherjee, S., Chapelle, O., Pontil, M., Poggio, T., Vapnik, V.: Feature selection for SVMs. In: Advances in Neural Information Processing Systems, vol. 13, pp. 668–674 (2000)Google Scholar
  4. 4.
    Golub, G., Reinsch, C.E.: Singular value decomposition and least squares solutions. Numerische Mathematik 14, 403–420 (1970)MathSciNetMATHCrossRefGoogle Scholar
  5. 5.
    Jolliffe, I.T.: Principal Component Analysis, 2nd edn., New York. Springer Series in Statistics (2002)Google Scholar
  6. 6.
    Comon, P.: Independent Component Analysis, a new concept? Signal Processing 36, 287–314 (1994)MATHCrossRefGoogle Scholar
  7. 7.
    Hall, M.A.: Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning. In: Proceedings of the Seventeenth International Conference on Machine Learning, pp. 359–366. Morgan Kaufmann Publishers Inc., San Francisco (2000)Google Scholar
  8. 8.
    Lu, Y., Roychowdhury, V., Vandenberghe, L.: Distributed parallel support vector machines in strongly connected networks. IEEE Trans. Neural Networks 19, 1167–1178 (2008)CrossRefGoogle Scholar
  9. 9.
    Stefan, R.: Incremental Learning with Support Vector Machines. In: IEEE International Conference on Data Mining, p. 641. IEEE Computer Society, Los Alamitos (2001)Google Scholar
  10. 10.
    Syed, N.A., Liu, H., Sung, K.: Incremental learning with support vector machines. In: Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), San Diego, California (1999)Google Scholar
  11. 11.
    Caragea, C., Caragea, D., Honavar, V.: Learning support vector machine classifiers from distributed data sources. In: Proceedings of the Twentieth National Conference on Artificial Intelligence (AAAI), Student Abstract and Poster Program, pp. 1602–1603. AAAI Press, Pittsburgh (2005)Google Scholar
  12. 12.
    Collobert, R., Bengio, S., Bengio, Y.: A parallel mixture of SVMs for very large scale problems. Neural Computation 14, 1105–1114 (2002)MATHCrossRefGoogle Scholar
  13. 13.
    Vapnik, V.N.: The nature of statistical learning theory. Springer, NY (1995)MATHGoogle Scholar
  14. 14.
    Graf, H.P., Cosatto, E., Bottou, L., Durdanovic, I., Vapnik, V.: Parallel support vector machines: The cascade SVM. In: Proceedings of the Eighteenth Annual Conference on Neural Information Processing Systems (NIPS), pp. 521–528. MIT Press, Vancouver (2004)Google Scholar
  15. 15.
    Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 27–27 (2011)CrossRefGoogle Scholar
  16. 16.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 2278–2324 (1998)CrossRefGoogle Scholar
  17. 17.
    Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, Cambridge (1999)MATHGoogle Scholar
  18. 18.
    Dean, J., Ghemawat, S.: Mapreduce: Simplified data processing on large clusters. In: Proceedings of the 6th conference on Symposium on Operating Systems Design & Implementation(OSDI), p. 10. USENIX Association, Berkeley (2004)Google Scholar
  19. 19.
    Schatz, M.C.: CloudBurst: highly sensitive read mapping with MapReduce. Bioinformatics 25, 1363–1369 (2009)CrossRefGoogle Scholar
  20. 20.
    Rosasco, L., De Vito, E., Caponnetto, A., Piana, M., Verri, A.: Are loss functions all the same. Neural Computation 16, 1063–1076 (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • F. Ozgur Catak
    • 1
  • M. Erdal Balaban
    • 2
  1. 1.National Research Institute of Electronics and Cryptology (UEKAE)TubitakTurkey
  2. 2.Quantitative MethodsIstanbul UniversityTurkey

Personalised recommendations