Advertisement

Neural Processing Letters

, Volume 44, Issue 1, pp 161–184 | Cite as

A Resource Aware MapReduce Based Parallel SVM for Large Scale Image Classifications

  • Wenming Guo
  • Nasullah Khalid Alham
  • Yang Liu
  • Maozhen Li
  • Man Qi
Article
  • 360 Downloads

Abstract

Machine learning techniques have facilitated image retrieval by automatically classifying and annotating images with keywords. Among them support vector machines (SVMs) are used extensively due to their generalization properties. However, SVM training is notably a computationally intensive process especially when the training dataset is large. This paper presents RASMO, a resource aware MapReduce based parallel SVM algorithm for large scale image classifications which partitions the training data set into smaller subsets and optimizes SVM training in parallel using a cluster of computers. A genetic algorithm based load balancing scheme is designed to optimize the performance of RASMO in heterogeneous computing environments. RASMO is evaluated in both experimental and simulation environments. The results show that the parallel SVM algorithm reduces the training time significantly compared with the sequential SMO algorithm while maintaining a high level of accuracy in classifications.

Keywords

Parallel SVM MapReduce Image classification and annotation Load balancing 

Notes

Acknowledgments

This research is partially supported by the National Basic Research Program (973) of China under Grant 2014CB340404.

References

  1. 1.
    Smeulders A, Worring M, Santini S, Gupta A, Jain R (2000) Content based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380CrossRefGoogle Scholar
  2. 2.
    Tsai C, Hung C (2008) Automatically annotating images with keywords: a review of image annotation systems. Recent Pat Comput Sci 1:55–68CrossRefGoogle Scholar
  3. 3.
    Wong W, Hsu S (2006) Application of SVM and ANN for image retrieval. Eur J Oper Res 173(3):938–950MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Gao Y, Fan J (2005) Semantic image classification with hierarchical feature subset selection. In: Proceedings of the ACM multimedia workshop on multimedia information retrieval, pp 135–142Google Scholar
  5. 5.
    Boutell M, Luo J, Shen X, Brown M (2004) Learning multi-label scene classification. Pattern Recognit 37(9):1757–1771CrossRefGoogle Scholar
  6. 6.
    Chen Y, Wang JZ (2004) Image categorization by learning and reasoning with regions. J Mach Learn Res 5:913–939MathSciNetGoogle Scholar
  7. 7.
    Cusano C, Ciocca G, Schettini R (2004) Image annotation using SVM. In: Proceedings of SPIE conference on internet imaging, pp 330–338Google Scholar
  8. 8.
    Fan J, Gao Y, Luo H, Xu G (2004) Automatic image annotation by using concept-sensitive salient objects for image content representation. In: Proceedings of the 27th annual international conference on research and development in information retrieval (SIGIR), pp 361–368Google Scholar
  9. 9.
    Le Saux B, Amato G (2004) Image recognition for digital libraries. In: Proceedings of the ACM multimedia workshop on multimedia information retrieval (MIR), pp 91–98Google Scholar
  10. 10.
    Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, CambridgeCrossRefzbMATHGoogle Scholar
  11. 11.
    Abe S (2005) Support vector machines for pattern classification (advances in pattern recognition. Springer, New YorkGoogle Scholar
  12. 12.
    Waring C, Liu X (2005) Face detection using spectral histograms and SVMs. IEEE Trans Syst Man Cybern Part B 35(3):467–476CrossRefGoogle Scholar
  13. 13.
    Colas F, Brazdil P (2006) Comparison of SVM and some older classification algorithms in text classification tasks. In: Proceedings of IFIP-AI world computer congress, pp 169–178Google Scholar
  14. 14.
    Do T, Nguyen V, Poulet F (2008) Speed up SVM algorithm for massive classification tasks. In: Proceedings of the 4th international conference on advanced data mining and applications (ADMA), pp 147–157Google Scholar
  15. 15.
    Alham NK, Li M, Hammoud S, Qi H (2009) Evaluating machine learning techniques for automatic image annotations. In: Proceedings of the 6th international conference on fuzzy systems and knowledge discovery (FSKD), pp 245–249Google Scholar
  16. 16.
    Hazan T, Man A, Shashua A (2008) A parallel decomposition solver for SVM: distributed dual ascend using fenchel duality. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 1–8Google Scholar
  17. 17.
    Zanghirati G, Zanni L (2003) A parallel solver for large quadratic programs in training support vector machines. Parallel Comput 29(4):535–551MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Do T, Poulet F (2006) Classifying one billion data with a new distributed SVM algorithm. In: Proceedings of the international conference on research, innovation and vision for the future (RIVF), pp 59–66Google Scholar
  19. 19.
    Chang EY, Zhu K, Wang H, Bai H, Li J, Qiu Z (2007) PSVM: parallelizing support vector machines on distributed computers. In: Proceedings of advances in neural information processing systems, pp 257–264Google Scholar
  20. 20.
    Zhao HX, Magoules F (2011) Parallel support vector machines on multi-core and multiprocessor systems. Proceedings of the 11th International Conference on Artificial Intelligence and Applications (AIA 2011)Google Scholar
  21. 21.
    Zhu-Hong Y, Jian-Zhong Y, Lin Z, Shuai L, Zhen-Kun W (2014) A MapReduce based parallel SVM for large-scale predicting protein-protein interactions. Neurocomputing 145:37–43CrossRefGoogle Scholar
  22. 22.
    Cao L, Keerthi S, Chong-Jin O, Zhang J, Periyathamby U, Ju FuX, Lee H (2006) Parallel sequential minimal optimization for the training of support vector machines. IEEE Trans Neural Netw 17(4):1039–1049CrossRefGoogle Scholar
  23. 23.
    Bickson D, Yom-Tov E, Dolev D (2008) A Gaussian belief propagation solver for large scale support vector machines. In: Proceedings of the 5th European conference on complex systemsGoogle Scholar
  24. 24.
    Woodsend K, Gondzio J (2009) Hybrid MPI/OpenMP parallel linear support vector machine training. J Mach Learn Res 10:1937–1953MathSciNetzbMATHGoogle Scholar
  25. 25.
    Bottou L, Bordes A, Ertekin S (2012) LASVM. http://leon.bottou.org/projects/lasvm. Accessed 19 May 2015
  26. 26.
    Cao LJ, Keerthi SS, Ong CJ, Uvaraj P, Fu XJ, Lee HP (2006) Developing parallel sequential minimal optimization for fast training support vector machine. Neurocomputing 70(1–3):93–104CrossRefGoogle Scholar
  27. 27.
    Khalid Alham N (2011) Parallelizing support vector machines for scalable image annotation. PhD Thesis, Brunel UniversityGoogle Scholar
  28. 28.
    Platt J (1998) Sequential minimal optimization: a fast algorithm for training support vector machines. Technical report, MSR-TR-98-14, Microsoft ResearchGoogle Scholar
  29. 29.
    Dean J, Ghemawat S (2008) Mapreduce: simplified data processing on large clusters. Commun ACM 51(1):107–113CrossRefGoogle Scholar
  30. 30.
    Apache Hadoop!. http://hadoop.apache.org. Accessed 19 May 2015
  31. 31.
    Venner J (2009) Programming Hadoop. Springer, Berlin, pp 1–407CrossRefGoogle Scholar
  32. 32.
    Graf H, Cosatto E, Bottou L, Durdanovic I, Vapnik V (2004) Parallel support vector machines: the cascade SVM. In: Proceedings of advances in neural information processing systems (NIPS)Google Scholar
  33. 33.
    Yang J (2006) An improved cascade SVM training algorithm with crossed feedbacks. In: Proceedings of 1st international multi-symposium of computer and computational sciences (IMSCCS), pp 735–738Google Scholar
  34. 34.
    Ming-Hsuan Y, Moghaddam B (2000) Support vector machines for visual gender classification. In: Proceedings of the international conference on pattern recognition (ICOR), pp 5115–5118Google Scholar
  35. 35.
    Osuna E, Freund R, Girosit F (1997) Training support vector machines: an application to face detection. In: Proceedings of computer vision and pattern recognition (CVPR), pp 130–136Google Scholar
  36. 36.
    Bach FR, Jordan MI (2005) Predictive low-rank decomposition for kernel methods. In: Proceedings of the 22nd international conference on machine learning (ICML), pp 33–40Google Scholar
  37. 37.
    Suykens JK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Collobert R, Bengio S, Bengio Y (2002) A parallel mixture of SVMs for very large scale problems. Neural Comput 14(5):1105–1114CrossRefzbMATHGoogle Scholar
  39. 39.
    Huang G, Mao K, Siew C, Huang D (2005) Fast modular network implementation for support vector machines. IEEE Trans Neural Netw 16(6):1651–1663CrossRefGoogle Scholar
  40. 40.
    Lu Y, Roychowdhury V, Vandenberghe L (2008) Distributed parallel support vector machines in strongly connected networks. IEEE Trans Neural Netw 19(7):1167–1178CrossRefGoogle Scholar
  41. 41.
    Kun D, Yih L, Perera A (2003) Parallel SMO for training support vector machines, SMA 5505, project final reportGoogle Scholar
  42. 42.
    The Cilk Project. http://supertech.csail.mit.edu/cilk/. Accessed 19 May 2015
  43. 43.
    Kuszmaul BC (2007) Cilk provides the “best overall productivity” for high performance computing: (and won the HPC challenge award to prove it). In: Proceedings of the 9th ACM symposium on parallel algorithms and architectures (SPAA), pp 299–300Google Scholar
  44. 44.
    Catanzaro B, Sundaram N, Keutzer K (2008) Fast support vector machine training and classification on graphics processors. In: Proceedings of the 25th international conference on machine learning (ICML), pp 104—111Google Scholar
  45. 45.
    Dong J, Krzyżak A, Suen C (2003) A fast parallel optimization for training support vector machine. In: Proceedings of the 3rd international conference on machine learning and data mining in pattern recognition (MLDM), pp 96–105Google Scholar
  46. 46.
    Winters-Hilt S, Armond K (2012) Distributed SVM learning and support vector reduction. http://cs.uno.edu/~winters/SVM_SWH_preprint.pdf. Accessed 19 May 2015
  47. 47.
    Gillick D, Faria A, DeNero J (2012) MapReduce: distributed computing for machine learning. http://www.icsi.berkeley.edu/~arlo/publications/gillick_cs262a_proj.pdf. Accessed 19 May 2015
  48. 48.
    Chu C, Kim S, Lin Y, Yu Y, Bradski G, Ng A, Olukotun K (2006) Map-reduce for machine learning on multicore. In: Schölkopf B, Platt JC, Hoffman T (eds) NIPS. MIT Press, Cambridge, pp 281–288Google Scholar
  49. 49.
    Sun Z, Fox G (2014) Study on parallel SVM based on MapReduce. http://www.academia.edu/2929779/Study_on_Parallel_SVM_Based_on_MapReduce. Accessed 19 May 2015
  50. 50.
    Çatak FO, Balaban ME (2012) CloudSVM: training an SVM classifier in cloud computing systems. In: Proceedings of the pervasive computing and the networked world—joint international conference (ICPCA/SWS), pp 57–68Google Scholar
  51. 51.
    Keerthi S, Shevade S, Bhattacharyya C, Murthy K (2001) Improvements to Platt’s SMO algorithm for SVM classifier design. Neural Comput 13(3):637–649CrossRefzbMATHGoogle Scholar
  52. 52.
    Ghemawat S, Gobioff H, Leung S (2003) The Google file system. In: Proceedings of the 19th ACM symposium on operating systems principles (SOSP), pp 29–43Google Scholar
  53. 53.
    Liu Y, Li M, Khan M, Qi M (2014) A MapReduce based distributed LSI for scalable information retrieval. Comput Inf 33(2):259–280Google Scholar
  54. 54.
    Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, BostonzbMATHGoogle Scholar
  55. 55.
    Weka 3. http://www.cs.waikato.ac.nz/ml/weka. Accessed 19 May 2015
  56. 56.
    Sikora T (2001) The MPEG-7 visual standard for content description, an overview. IEEE Trans Circuits Syst Video Technol 11(6):696–702MathSciNetCrossRefGoogle Scholar
  57. 57.
    Corel Image Databases. http://www.corel.com. Accessed 19 May 2015
  58. 58.
    Lire, An open source Java content based image retrieval library. http://www.semanticmetadata.net/lire/. Accessed 19 May 2015
  59. 59.
    Liu Y, Li M, Alham NK, Hammoud S (2013) HSim: a MapReduce simulator in enabling cloud computing. Futur Gener Comput Syst 29(1):300–308CrossRefGoogle Scholar
  60. 60.
    Knerr S, Personnaz L, Dreyfus G (1990) Single-layer learning revisited: a stepwise procedure for building and training a neural network. In: Fogelman Soulié F, Hérault J (eds) Neurocomputing: algorithms, architectures and applications, volume F68 of NATO ASI series. Springer, Berlin, pp 41–50CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Wenming Guo
    • 1
  • Nasullah Khalid Alham
    • 2
  • Yang Liu
    • 3
  • Maozhen Li
    • 4
    • 5
  • Man Qi
    • 6
  1. 1.School of Software EngineeringBeijing University of Post and TelecommunicationBeijingChina
  2. 2.Nuffield Department of Clinical Laboratory SciencesUniversity of OxfordOxfordUK
  3. 3.School of Electrical Engineering and InformationSichuan UniversityChengduChina
  4. 4.Department of Electronic and Computer EngineeringBrunel University LondonUxbridgeUK
  5. 5.The Key Laboratory of Embedded Systems and Service ComputingTongji UniversityShanghaiChina
  6. 6.Department of ComputingCanterbury Christ Church UniversityCanterburyUK

Personalised recommendations