Skip to main content
SpringerLink
Log in

Addressing Stemming Algorithm for Arabic Text Using Spark Over Hadoop

  • Conference paper
  • First Online:
Advanced Intelligent Systems for Sustainable Development (AI2SD’2019) (AI2SD 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1102))

  • 418 Accesses

Abstract

Hadoop and Apache Spark are two popular big data frameworks that make two things different by working on two different logics. Hadoop distributes large amounts of data across multiple nodes which makes it not expensive in terms of hardware. Spark makes work with distributed data but does not rely on a distributed storage system. In this perspective we try to optimize existing stemming algorithm representing an essential step for language processing and analysis. We will focus on comparing between the results obtained using Hadoop and Spark, then implement a merger between the two frameworks. Towards the end of this paper a clear perception will be designed to benefit from advantage of the characteristics and performance of each of the two frameworks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Çakir, M.U., Güldamlasioğlu, S.: Text mining analysis in turkish language using big data tools. In: 2016 IEEE 40th Annual Computer Software and Applications Conference (COMPSAC), pp. 614–618. Atlanta, GA (2016). https://doi.org/10.1109/compsac.2016.203

  2. Bougar, M., Ziyati, E.H.: Stemming algorithm for arabic text using a parallel data processing. In: Yang, X.S., Sherratt, S., Dey, N., Joshi A. (eds.) Third International Congress on Information and Communication Technology. Advances in Intelligent Systems and Computing, vol. 797. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-1165-9_23

  3. Aydin, G., Hallac, I.R.: Document Classification Using Distributed Machine Learning. arXiv preprint arXiv:1802.03597 (2018)

  4. Ghazi, M.R., Gangodkar, D.: Hadoop, MapReduce and HDFS: A Developers Perspective

    Google Scholar 

  5. Madani, Y., Erritali, M., Bengourram, J.: Arabic stemmer based big data. J. Electron. Commer. Organ. (JECO) 16(1), 12 (2018). https://doi.org/10.4018/jeco.2018010102

    Article  Google Scholar 

  6. Zaharia, M., Xin, R.S., Wendell, P., Das, T., Armbrust, M., Dave, A., Meng, X., Rosen, J., Venkataraman, S., Michael, J.F., Ghodsi, A., Gonzalez, J., Shenker, S., Stoica, I.: Apache spark: a unified engine for big data processing. Commun. ACM 59, 56–65 (2016). https://doi.org/10.1145/2934664

    Article  Google Scholar 

  7. Karau, H., Konwinski, A., Wendell, P., Zaharia, M.: Learning Spark: Lightning-Fast Big Data Analysis, 276 pp. O’Reilly Media, Inc. 28 Jan 2015

    Google Scholar 

  8. Huang, W., Meng, L., Zhang, D., Zhang, W.: In-memory parallel processing of massive remotely sensed data using an apache spark on Hadoop Yarn Model. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10(1), 3–19 (2017). https://doi.org/10.1109/JSTARS.2016.2547020

    Article  Google Scholar 

  9. Zaharia, M., Chowdhury, M., Das, Tathagata, Dave, A., Ma, J., McCauley, M., Franklin, M., Shenker, S., Stoica, Ion: Fast and interactive analytics over Hadoop data with Spark. USENIX Login 37, 45–51 (2012)

    Google Scholar 

  10. Samadi, Y., Zbakh, M., Tadonki, C.: Comparative study between Hadoop and spark based on Hibench benchmarks. In: 2016 2nd International Conference on Cloud Computing Technologies and Applications (CloudTech), pp. 267–275. Marrakech (2016). https://doi.org/10.1109/cloudtech.2016.7847709

  11. Taran, V., Alienin, O., Stirenko, S., Gordienko, Y., Rojbi, A.: Performance evaluation of distributed computing environments with Hadoop and spark frameworks. In: 2017 IEEE International Young Scientists Forum on Applied Physics and Engineering (YSF), pp. 80–83. Lviv (2017). https://doi.org/10.1109/ysf.2017.8126655

  12. Verma, A., Mansuri, A.H., Jain, N.: Big data management processing with Hadoop MapReduce and spark technology: a comparison. In: 2016 Symposium on Colossal Data Analysis and Networking (CDAN), pp. 1–4. Indore (2016). https://doi.org/10.1109/cdan.2016.7570891

Download references

Author information

Authors and Affiliations

  1. ENSEM, RITM Laboratory, Université Hassan II, Casablanca, Maroc

    Marieme Bougar

Authors
  1. Marieme Bougar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. El Houssaine Ziyati
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marieme Bougar .

Editor information

Editors and Affiliations

  1. Faculty of Sciences and Techniques of Tangier, Abdelmalek Essaâdi University, Tangier, Morocco

    Mostafa Ezziyyani

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bougar, M., Ziyati, E.H. (2020). Addressing Stemming Algorithm for Arabic Text Using Spark Over Hadoop. In: Ezziyyani, M. (eds) Advanced Intelligent Systems for Sustainable Development (AI2SD’2019). AI2SD 2019. Advances in Intelligent Systems and Computing, vol 1102. Springer, Cham. https://doi.org/10.1007/978-3-030-36653-7_7

Download citation

Publish with us

Policies and ethics

Search

Navigation