Advertisement

Performance Comparison of Distributed Pattern Matching Algorithms on Hadoop MapReduce Framework

  • C. P. SonaEmail author
  • Jaison Paul Mulerikkal
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 235)

Abstract

Creating meaning out of the growing Big Data is an insurmountable challenge data scientists face and pattern matching algorithms are great means to create such meaning from heaps of data. However, the available pattern matching algorithms are mostly tested with linear programming models whose adaptability and efficiency are not tested in distributed programming models such as Hadoop MapReduce, which supports Big Data. This paper explains an experience of parallelizing three of such pattern matching algorithms, namely - Knuth Morris Pratt Algorithm (KMP), Boyer Moore Algorithm (BM) and a lesser known Franek Jennings Smyth (FJS) Algorithm and porting them to Hadoop MapReduce framework. All the three algorithms are converted to MapReduce programs using key value pairs and experimented on single node as well as cluster Hadoop environment. The result analysis with the Project Gutenberg data-set has shown all the three parallel algorithms scale well on Hadoop environment as the data size increases. The experimental results prove that KMP algorithm gives higher performance for shorter patterns over BM, and BM algorithm gives higher performance than KMP for longer patterns. However, FJS algorithm, which is a hybrid of KMP and Boyer horspool algorithm which is advanced version of BM, outperforms both KMP and BM for shorter and longer patterns, and emerges as the most suitable algorithm for pattern matching in a Hadoop environment.

Keywords

Pattern matching Hadoop MapReduce Big Data Knuth Morris Pratt Algorithm Boyer Moore Algorithm Franek Jennings Smyth Algorithm 

Notes

Acknowledgement

This work was completed successfully using the infrastructure support provided by Śúnya Labs, Rajagiri School of Engineering and Technology, India.

References

  1. 1.
    Project gutenberg. https://www.gutenberg.org/
  2. 2.
    Ramya, A., Sivasankar, E.: Distributed pattern matching and document analysis on big data using Hadoop MapReduce model. In: International Conference on Parallel and Distributed Grid Computing (2014)Google Scholar
  3. 3.
    Al-Mazroi, A.A., Rashid, N.A.A.: A fast hybrid algorithm for the exact string matching problem. Am. J. Eng. Appl. Sci. 4(1), 102–107 (2011)CrossRefGoogle Scholar
  4. 4.
    Boyer, R.S.: A fast string searching algorithm. Commun. Assoc. Comput. Mach. 20, 762–772 (1977)zbMATHGoogle Scholar
  5. 5.
    Finnegan, M.: Boeing 787s to create half a terabyte of data per flight, says Virgin Atlantic. http://www.computerworlduk.com/data/. Accessed 12 Sep 2017
  6. 6.
    Hume, A., Sunday, D.: Fast string searching. Softw.: Pract. Exp. 21(11), 1221–1248 (1991)Google Scholar
  7. 7.
    Al-Khamaiseh, K., ALShagarin, S.: A survey of string matching algorithms. Int. J. Eng. Res. Appl. 4, 144–156 (2014). ISSN 2248–9622Google Scholar
  8. 8.
    Knuth, D.E., Morris, J.H., Pratt, V.R.: Fast pattern matching in strings. SIAM J. Comput. 6, 323–350 (1977)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Kofahi, N., Abusalama, A.: A framework for distributed pattern matching based on multithreading. Int. Arab J. Inf. Technol. 9(1), 30–38 (2012)Google Scholar
  10. 10.
    Cao, P., Wu, S.: Parallel research on KMP algorithm. In: International Conference on Consumer Electronics, Communications and Networks (CECNet) (2011)Google Scholar
  11. 11.
    Diwate, M.R.B., Alaspurkar, S.J.: Study of different algorithms for pattern matching. Int. J. Adv. Res. Comput. Sci. Softw. Eng. 3, 1–8 (2013). ISSN 2277 128XGoogle Scholar
  12. 12.
    Rajesh, S., Prathima, S., Reddy, L.S.S.: Unusual pattern detection in DNA database using KMP algorithm. Int. J. Comput. Appl. 1(22), 1–5 (2010)Google Scholar
  13. 13.
    Singh, S., Singh, N.: Big data analytics. In: 2012 International Conference on Communication, Information and Computing Technology (ICCICT), 13230053, IEEE, October 2012Google Scholar
  14. 14.
    Singh, A.: New York stock exchange oracle exadata - our journey. http://www.oracle.com/technetwork/database/availability/index.html. Accessed 12 Sep 2017
  15. 15.
    Sardjono, T.A., Al Kindhi, B.: Pattern matching performance comparisons as big data analysis recommendations for hepatitis C virus (HCV) sequence DNA. In: International Conference on Artificial Intelligence, Modelling and Simulation (AIMS) (2015). ISBN 978-1-4673-8675-3Google Scholar
  16. 16.
    Alzoabi, U.S., Alosaimi, N.M., Bedaiwi, A.S.: Parallelization of KMP string matching algorithm. In: World Congress on Computer and Information Technology (WCCIT). INSPEC Accession Number: 13826319 (2013)Google Scholar
  17. 17.
    Vance, A.: Hadoop, a free software program, finds uses beyond search, March 2009. http://www.nytimes.com/2009/03/17/technology/business-computing/17cloud.html
  18. 18.
    Vidanagama, D.: A comparative analysis of various string matching algorithms. In: International Research Conference, pp. 54–60 (2015)Google Scholar
  19. 19.
    Franek, F., Jennings, C.G., Smyth, W.F.: A simple fast hybrid pattern-matching algorithm. J. Discrete Algorithms 5, 682–695 (2007)MathSciNetCrossRefGoogle Scholar

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2018

Authors and Affiliations

  1. 1.Śúnya LabsRajagiri School of Engineering and TechnologyKochiIndia

Personalised recommendations