Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Artificial Intelligence and Robotics

ISAIR 2018: Cognitive Internet of Things: Frameworks, Tools and Applications pp 121–133Cite as

Performance Modeling of Spark Computing Platform

  • Chapter
  • First Online:

Part of the book series: Studies in Computational Intelligence ((SCI,volume 810))

Abstract

Big Data has been widely used in all aspects of society. For solving the problem of massive data storing and analyzing, many big data solutions have been proposed. Spark is the newer solution of the universal parallel framework which like Hadoop MapReduce. Compare the Hadoop, Spark’s performance has been increased significantly. As a data analysis framework, researchers are particularly concerned about its performance. So in this paper, we use a stochastic process algebra (PEPA) to model the Spark architecture. This model will give the usability of the compositional approach in modeling and analysis Spark architecture. This research obtains an algorithm that generated the service flow of the PEPA model. In the end, we will state the benefit of this compositional method in modeling a large parallel system.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. What Is Apache Hadoop? May 2018. http://hadoop.apache.org/

  2. Chandakanna, V.R.: REHDFS: a random read/write enhanced HDFS. J. Netw. Comput. Appl. 103, 85–100 (2018)

    Article  Google Scholar 

  3. Won, H., Nguyen, M.C., Gil, M.S., Moon, Y.S.: Advanced resource management with access control for multitenant hadoop. J. Commun. Netw. 17(6), 592–601 (2015)

    Article  Google Scholar 

  4. Serikawa, S., Huimin, L.: Underwater image dehazing using joint trilateral filter. Comput. Electr. Eng. 40(1), 41–50 (2014)

    Article  Google Scholar 

  5. Lu, H., Li, Y., Mu, S., Wang, D., Kim, H., Serikawa, S.: Motor anomaly detection for unmanned aerial vehicles using reinforcement learning. IEEE Internet Things J. (2017). https://doi.org/10.1109/JIOT.2017.2737479

  6. Lu, H., Li, Y., Chen, M., Kim, H., Serikawa, S.: Brain intelligence: go beyond artificial intelligence. Mob. Netw. Appl. 1–8 (2017)

    Google Scholar 

  7. Lu, H., Bin, L., Zhu, J., Serikawa, S.: Wound intensity correction and segmentation with convolutional neural networks. Concurr. Comput.: Pract. Exp. (2017). https://doi.org/10.1002/cpe.3927

  8. Lu, H., Li, Y., Uemura, T., Kim, H., Serikawa, S.: Low illumination underwater light field images reconstruction using deep convolutional neural networks. Future Gener. Comput. Syst. 142–148 (2018). https://doi.org/10.1016/j.future.2018.01.001

  9. Xu, X., He, L., Lu, H., Gao, L., Ji, Y.: Deep adversarial metric learning for cross-modal retrieval. World Wide Web J. (2018). https://doi.org/10.1007/s11280-018-0541-x

  10. Hillston, J.: A Compositional Approach to Performance Modelling. Cambridge University Press (1996)

    Google Scholar 

  11. Ding, J., Wang, R., Chen, X.: Performance modeling and evaluation of real-time traffic status query for intelligent traffic systems. In: Proceedings of 2016 22nd Asia-Pacific Conference on Communications (APCC) (2016)

    Google Scholar 

  12. Ding, J.: A comparison of fluid approximation and stochastic simulation for evaluating content adaptation systems. Wirel. Pers. Commun. 84, 231–250 (2015)

    Article  Google Scholar 

  13. Ding, J., Zhu, X., Wang, M.: Fluid analysis for a PEPA model. In: Proceedings of the 2015 Chinese Intelligent Systems Conference, vol. 2, pp. 181–190 (2015)

    Google Scholar 

  14. Hillston, J.: Fluid flow approximation of PEPA models. In: Second International Conference on the Quantitative Evaluation of Systems (2005)

    Google Scholar 

  15. Molloy, M.K.: Performance analysis using stochastic Petri Nets. IEEE Trans. Comput. 31, 913–917 (1982)

    Article  Google Scholar 

Download references

Acknowledgements

The authors acknowledge the financial support by the National Natural Science Foundation of China under Grant 61472343.

Author information

Authors and Affiliations

  1. School of Information Engineering, Yangzhou University, Yangzhou, 225000, China

    Jie Ding, Yunyue Xie & Meihua Zhou

Authors
  1. Jie Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yunyue Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Meihua Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yunyue Xie .

Editor information

Editors and Affiliations

  1. Department of Mechanical and Control Engineering, Kyushu Institute of Technology, Kitakyushu, Japan

    Huimin Lu

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Ding, J., Xie, Y., Zhou, M. (2020). Performance Modeling of Spark Computing Platform. In: Lu, H. (eds) Cognitive Internet of Things: Frameworks, Tools and Applications. ISAIR 2018. Studies in Computational Intelligence, vol 810. Springer, Cham. https://doi.org/10.1007/978-3-030-04946-1_13

Download citation

Publish with us

Policies and ethics

Search

Navigation