Advertisement

SECapacity: A Secure Capacity Scheduler in YARN

  • Chuntao Dong
  • Qingni ShenEmail author
  • Lijing Cheng
  • Yahui Yang
  • Zhonghai Wu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9977)

Abstract

In this paper, aiming to the requirement that isolation of user’s job and data security, we deeply analyze the mainstream computing framework Hadoop YARN, and start with the core module of YARN - resource scheduler. Using the existing label-based scheduling policy, we design and implement a SECapacity scheduler. Our main work including: First, according to the principle of least privilege, we propose a user-classification based scheduling policy, which divided users to several levels based on their attributes, then restrict which nodes could be used by this user according to the user level. Second, we design and implement a SECapacity scheduler to implement user-classification based scheduling. Third, we verify and analyze the effectiveness and efficiency of SECapacity scheduler, the results shows that SECapacity scheduler can ensure 100% isolation of users at different levels, and the performance overhead is about 6.95%.

Keywords

Big data platform Hadoop User-classification based scheduling SECapacity scheduler 

Notes

Acknowledgments

This work is supported by the National High Technology Research and Development Program (“863” Program) of China under Grant No. 2015AA016009, the National Natural Science Foundation of China under Grant No. 61232005, 61672062, and the Science and Technology Program of Shen Zhen, China under Grant No. JSGG20140516162852628.

References

  1. 1.
    Apache hadoop. http://hadoop.apache.org
  2. 2.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. In: Conference on Symposium on Operating Systems Design & Implementation, vol. 51, pp. 107–113. USENIX Association (2004)Google Scholar
  3. 3.
    Denning, P.J.: Fault tolerant operating systems. ACM Comput. Surv. 8(4), 359–389 (1976)CrossRefzbMATHGoogle Scholar
  4. 4.
    Dinh, T.T.A., Saxena, P., Chang, E.C., et al.: M2R: enabling stronger privacy in mapreduce computation (2015)Google Scholar
  5. 5.
    Dong, C., Shen, Q., Li, W., Yang, Y., Wu, Z., Wan, X.: Eavesdropper: a framework for detecting the location of the processed result in hadoop. In: Qing, S., Okamoto, E., Kim, K., Liu, D. (eds.) ICICS 2015. LNCS, vol. 9543, pp. 458–466. Springer, Heidelberg (2016). doi: 10.1007/978-3-319-29814-6_39 CrossRefGoogle Scholar
  6. 6.
    Ohrimenko, O., Costa, M., Fournet, C., et al.: Observing and preventing leakage in MapReduce. In: ACM SIGSAC Conference, pp. 1570–1581 (2015)Google Scholar
  7. 7.
    Roy, I., Setty, S.T.V., Kilzer, A., et al.: Airavat: security and privacy for MapReduce. In: Usenix Symposium on Networked Systems Design and Implementation, NSDI 2010, San Jose, pp. 297–312 (2010)Google Scholar
  8. 8.
    Vavilapalli, V.K., Murthy, A.C., Douglas, C., et al.: Apache hadoop YARN: yet another resource negotiator. In: Symposium on Cloud Computing, pp. 1–16 (2013)Google Scholar
  9. 9.
    Wei, W., Du, J., Yu, T., et al.: SecureMR: a service integrity assurance framework for MapReduce. In: Computer Security Applications Conference, pp. 73–82. IEEE (2009)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Chuntao Dong
    • 1
    • 2
  • Qingni Shen
    • 1
    • 2
    Email author
  • Lijing Cheng
    • 1
    • 2
  • Yahui Yang
    • 1
    • 2
  • Zhonghai Wu
    • 1
    • 2
  1. 1.School of Software and MicroelectronicsPeking UniversityBeijingChina
  2. 2.MoE Key Lab of Network and Software AssurancePeking UniversityBeijingChina

Personalised recommendations